Do not buy RAM, compress (curiousbits.nl)

194 pointsby Ecno9212 hours ago163 comments

varenc 5 hours ago

As others have said, macOS does this by default and it's quite impressive. I have a long running daemon with a memory leak. By its nature, the leaking memory is very repetitive and easily compressed. Right now this daemon has tried to claim 2.5GB of memory, but it's been compressed to a mere 450MB of actual memory! And because it's a memory leak, it never accesses this old stuff, so it never gets decompressed. Only 20MB of RAM is held that isn't compressed.

Of course, it would be far better to just not have the memory leak...but in the mean time, macOS memory compression is making the leak nearly a none issue.

trollbridge 4 hours ago

Wait until people find out about swapping.

Sharlin 3 hours ago

Paging to disk has also become vastly more transparent, and less of a last resort, with M.2 SSDs. No more things slowing down 10x and endless HD noise when out of RAM.

bayindirh 3 hours ago

Oh we found out about it. In the old days, a machine with a load of 57380 (probably not the highest number I have seen) means the machine has run out of memory and started swapping heavily.

Fun days.

Now, we find out when the SSD wears out, in the worst case.

Lerc 17 minutes ago

How long does a wear leveling SSD take to fail if it is set to constantly write at it's fastest possible bandwidth?

dotancohen 2 hours ago

Why the downvotes? I also was thinking that's a perfect candidate for swapping to disk.

hobo123 an hour ago

You can also combine both. Linux has zswap.

davidkwast 4 hours ago

It's like having a room that shrinks everything to 1/5th of its size but you never come back to get anything. It will work until gets full.

01HNNWZ0MV43FF 3 hours ago

If the user was feeling stochastic, the OS could theoretically just drop pages that haven't been accessed in, say, 2 days, and send the process a segfault if it tries to access them.

Thus solving memory leaks forever...

bayindirh 3 hours ago

I believe one can create a kernel module called leaky_ram to make this possible. No?

kccqzy 13 minutes ago

No need for a kernel module. Create a FUSE filesystem in user space on which to create a swap file. Once any written block is more than 2 days old just discard it. When they are accessed give EIO. This IIRC translates to a SIGBUS signal in the process.

cpeterso 3 hours ago

A new take on the OOM killer: if a long-running daemon hasn't touched some heap pages for a very long time, perhaps the OS could assume it is leaked and unmap the pages. If the daemon tries to access that memory, let it crash.

kevincox 2 hours ago

This sounds like a bad idea. No one has logged in via SSH for a while so the daemon hasn't touched any pages (or only pre-auth relevant pages) then when you log in it gets killed. Or if someone sends a rarely used request to an HTTP server then it crashes accessing some data unique to that request.

Sure, they will get restarted, but not great.

cxr 2 hours ago

It's a good example of perverse outcomes. Rather than simply following the contract and functioning correctly, we would be:

going through extra effort to accommodate/enable buggy, poorly behaving programs, and meanwhile—

punishing those (and the programmers of those) that are well-behaved and functioning exactly as intended but that now have to contend with evading some heuristic of badness.

dotancohen 2 hours ago

Or swap it to a less expensive medium, such as the SSD.

TheSoftwareGuy 2 hours ago

That’s actually a very old idea, called paging or swapping. And it totally works

InsidiousWombat 9 minutes ago

I'd rather not have it waste write cycles

... but reading say libraries directly and only putting in RAM the parts often used, now we're talking

fl0ki 18 minutes ago

The BackRAMs

Lio 10 hours ago

RAM compression sounds very useful but the thing I really need extra RAM for is local inference.

Since the models are effectively highly compressed data already my assumption is that RAM compression won't help improve that much (I'm assuming unified memory here as per the Framework Desktop). A bit like trying to compress a JPEG in a zip file.

Could anyone comment on that?

rft 10 hours ago

I haven't tried it in RAM, but your reasoning matches what I saw when I looked at compressing weights for archiving them. The weights don't compress nicely with standard, especially fast, algorithms. Some more background in this repo [1] including example compression ratios. I would also worry about decompression speed, that gives you a hard bound on the throughput. If the numbers in this repo are accurate, you likely hit the throughput limit of decompression way before you hit the memory limit, even considering the space savings and thus higher effective throughput.

[1] https://github.com/zipnn/zipnn#results

kalcode 10 hours ago

I don't know at all. But I've been playing with oMlx and Splash. Using Pi there are compression extensions like the billion-context. I suspect your right.

But all seem to have massive issue boards. What I did see was oMlx release clustering so you can string 3 macs and have 3 times the memory or use one for prefilling accelerating.

Which I thought was neat and what I'll likely poke at instead of getting more ran.

0123456789ABCDE 9 hours ago

perhaps i'm not pilled enough for this conversation, but aren't you conflating two disparate concepts — model-weights and context-window?

kalcode 5 hours ago

Yah I think I am conflating it. I was thinking using the ram more effectively to begin to squeeze more, one of those is context size. Less VRAM the less context you can handle, so there always a lot of tuning to find that sweet spot.

So my mind on that.

matja 8 hours ago

> A bit like trying to compress a JPEG in a zip file

Interesting comparison, I wonder if there are loss-less compression schemes that are "weight-aware", about the statistical properties of model weights.

Back to that line - because (Huffman coded) JPEG images can be lossless recoded using arithmetic coding, which makes them smaller than if they were compressed with the best general-purpose compression algorithms/tools (e.g. `jpegtran -arithmetic < input.jpg > output.jpg` beats `xz -9 < input.jpg > output.jpg`). AC takes advantage of the properties of the image coding better than general-purpose compression tools on the output of the Huffman coding.

rft 7 hours ago

The library I use for archiving compressed raw LLM weights ([1] , posted in my other comment) afaik at least reorders the weights to more easily compress them. They also have a paper linked, which I admittedly didn't read, that might have more background and tradeoffs. Likely they will have discussed methods to take advantage of the structure of weights.

[1] https://github.com/zipnn/zipnn#results

strobe 5 hours ago

it interesting because usually on workstation the browser & electron apps it's that holding much of the memory hostage (it's very common to see that browser consuming 20-30gb on 64gb system), so may be if it will be significant compression win on that side it leads to more memory for model but how much performance penalty would be for inference in that case?

throw-qqqqq 5 hours ago

The max theoretical ratio of (byte-independent) compression can be approximated using the Shannon entropy.

Most quantized GGUFs have a very high Shannon entropy, so are almost perfectly compressed already. A Q4_K_M quantization is often around +7.90 bits/byte and so is virtually incompressible.

Unquantized models (F16) are often around +7bits/byte for comparison.

https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...

vlovich123 5 hours ago

You are correct, compression doesn’t help reduce a model. That’s why quantization is inherently a lossy operation although there’s tricks to limit the perceived loss. That’s the only way to further compress the data.

One way to think about it is that if the weights were compressible, it’s a clear signal that better training could make the model smarter as there’s redundant information in the structure that could be used to store other meaning.

The other challenge with compressing weights of course is that you have to decompress them before you do math with them. Dequantizing weights is significantly cheaper and supports random access too (not that that matters too much but it helps in terms of leveraging SIMD techniques to do it in an embarrassingly parallel way)

Powdering7082 4 hours ago

> redundant information in the structure that could be used to store other meaning.

Consider the induction heads, their W_O are generally expected to be the identity matrix and it is useful to have many of them at many layers (e.g. if you only had one induction head in the whole network you'd only be able to perform one induction circuit per token emitted)

That being said the attention heads are a small portion of the total network size

hobo123 an hour ago

Since you need to apply CPU time to compress memory, and also CPU before you can map pages back into process memory (by uncompressing), this is only useful for not frequently used memory, like swap only a bit faster.

I'm not too familiar with LLM memory access patterns, but since the model files are stored on disk, you wouldn't ever keep them in memory anyway (except caching) and just reread them on demand.

RossBencina 31 minutes ago

> I'm not too familiar with LLM memory access patterns

Dense architectures hit every byte of the model for every token. It's the ultimate memory bandwidth nightmare.

MoE models use subsections for each token and are more amenable to keeping parts in secondary storage, and indeed there are some local llm engines that do this.

jakozaur 10 hours ago

> Memory compression is not new. If you have a Windows or Mac device you are already using it.

Yes, macOS has had it enabled by default since OS X Mavericks 10.9, released in October 2013. I’m surprised some desktop Linux distributions still require manual configuration to enable it.

For browser tabs, though, I think a better solution is automatic tab closing, using something like the Tab Wrangler extension for Chrome. Collecting tabs comes naturally; we’re much worse at closing them.

jve 9 hours ago

Wait, Edge on windows automatically puts tabs to sleep when they are unused (https://explore.microsoft.com/en-us/edge/features/sleeping-t...). I think edge weren't even the first one, thus being kinda surprised I see chrome needs extension for that?

evenfrost 9 hours ago

For Chrome I recommend The Marvellous Suspender, which can auto-suspend unused tabs. Has been a life-saver for me for quite a while.

jamesgeck0 5 hours ago

I thought suspend plugins became redundant when Chrome started shipping a built-in auto-suspend. Is there still a reason to use them?

kasabali 4 hours ago

They provide more granular options on when and what to suspend compared to crude heuristics of the built in feature.

aa-jv 9 hours ago

>I’m surprised some desktop Linux distributions still require manual configuration to enable it.

Its a user choice and should be a user-facing knob because there are times it is inappropriate for use, such as in safety-critical circumstances, certain realtime realities, and so on. It should probably be on by default for desktop Linux, but off by default for every other application a distribution may be targeting ...

xnx 6 hours ago

Chrome has had a built-in tab-sleeping Memory Saver feature for 4 years: https://blog.google/products-and-platforms/products/chrome/n...

benjiro29 5 hours ago

"For browser tabs, though, I think a better solution is automatic tab closing" ... the better solution is to actually offload tabs. The issue with Chrome/Edge/... is not your tabs itself but the huge amount of memory loaded for GPU operations.

Do Shift+Esc and watch how large the GPU memory usage is. Often 2/3 to 50% of your actual tab usage.

Ironically, its not the closing of tabs that bothers me, but the fact that tab order changes, what slows my productivity/muscle memory down. And its the same reason for a lot of people why they keep tabs open.

I never understood why we do not have a basic snapshot / sleep that offloads a tabs memory on disk (and dumps the GPU memory), aka a ghost tab. And fast loads when you view the tab. Sure, there may be a small delay but it can save so much memory, while people can keep their tabs open/organized.

Some extensions try to mimic that but its often a crude solution because they do not have deeper access to the webview2 engine.

hobo123 an hour ago

Brave on Mac (but not Linux IIRC) does this, you can choose how eagerly it will suspend background tabs, although this is somewhat erratic - I've seen it suspend a paused YT video that has been idle for maybe an hour (and not much memory pressure on the system), and then sometimes it takes its time.

In Firefox there's about:unloads where you can suspend tabs in I believe LRU order. I'd really like to see this as a default option (like: only keep N tabs active). Maybe there's an extension for that.

hedgehog 4 hours ago

As others have noted zswap is usually a better choice than zram, additionally depending on your situation it may be worth checking which algorithm it's using as there are a few choices (zstd, lz4, etc) and the defaults vary by distro. zstd is the right choice IMO unless you're in a situation where you know better.

flowerlad 3 hours ago

In the 1990s there was a third-party add-on for Windows called SoftRAM95 [1] that doubled your RAM using compression. That one turned out to be a scam though [2].

[1] https://en.wikipedia.org/wiki/SoftRAM

[2] https://devblogs.microsoft.com/oldnewthing/20211111-00/?p=10...

cogman10 6 hours ago

Don't use Zram, use Zswap. Especially if you want to swap to disk.

https://chrisdown.name/2026/03/24/zswap-vs-zram-when-to-use-...

ciupicri 5 hours ago

I've tried for running bsdiff and zswap was slower and spilled out data to the disk (which would wear an SSD).

mrob 4 hours ago

I never want to swap to disk. If I'm ever in that situation, it means I did something wrong, so I want my userspace OOM killer to trigger and inform me of my mistake immediately instead of wasting time and SSD endurance delaying the inevitable. But zram with lz4 compression is effectively free RAM in most circumstances, so there's little reason not to enable it.

dsp_person 4 hours ago

Good point, dumping to SSD isn't ideal. I think I'm gonna start trying to set memory limits on programs with systemd-run so they can self-limit themselves.

01HNNWZ0MV43FF 3 hours ago

For today's 10,000, do `sudo apt-get install earlyoom` and thank me later

quotemstr an hour ago

You absolutely do want to swap to disk. There are lots of non-file-backed cold pages that are pointless to keep in RAM forever. You could argue that these pages shouldn't exist, but they do, so why not shove them to a disk?

mrob an hour ago

I want instant access to my rarely used pages just as much as my frequently used pages. It's only pointless to keep in RAM if it's literally never used. If we knew how to identify those pages we could just avoid allocating them in the first place.

quotemstr 27 minutes ago

Your system is still swapping all your file-backed pages (e.g. .so files) unless you're mlockall()-ing everything. If you pin non-file-backed pages in RAM by running without a swap file, the system has a harder time keeping your frequently-accessed file-backed pages in RAM and can't do as good a job as optimizing your usage patterns.

All you're doing by running without swap is pessimizing yourself.

InsidiousWombat 7 minutes ago

wasting SSD writes for one. Especially if something leaks and it starts trashing

teravor 3 hours ago

> Especially if you want to swap to disk.

if you don't want to swap to disk you should not use zswap.

zram is ideal in that scenario, though it may complicate OOM killer considerations (you may want a userspace one).

jmakov 2 hours ago

Looking at the solution AMD bought, looks like Linux has a lot of room to improve regarding moving (compressed) inactive pages to disk: https://www.amd.com/en/products/software/mext.html.

If my understanding is correct, both ZRAM and ZSWAP write uncompressed pages to disk. And both don't pack the pages together meaning even if we'd write compressed pages (let's say they compress 5:1), we'd still write the original size to disk. This is crazy.

quotemstr an hour ago

FWIW, DAMON (https://docs.kernel.org/mm/damon/design.html#operation-actio...) has supported explicit control over Linux working sets for a while now. For whatever reason, ISTM few people know about it, but it's a nice subsystem with a lot of policy flexibility.

zamalek an hour ago

> If my understanding is correct, both ZRAM and ZSWAP write uncompressed pages to disk.

ZRAM added compressed_writeback in 7.0, it needs to be explicitly enabled.

jmakov an hour ago

Yes, but if a 4k page compresses to 1k, a 4k block on the disk still gets used: 1k + 3k of zero padding. Meaning if 100GB of data compresses 5:1, 100GB needs to be written to the disk instead of 20GB.

noodlesUK an hour ago

I think it's a great shame that kubernetes really doesn't like systems with any kind of swap on (including zswap). If memory compression were more commonly used I think many workloads could survive on smaller systems.

zamalek an hour ago

> Compressing memory does come with small, but not really noticeable, downsides. It takes a little bit more from your CPU and adds latency to some processes.

I recall reading an article around the time the Windows team added it (back when I still used the OS). Their compression algorithm was typically faster than the latency to the prevailing disk type at the time (SATA SSD, most likely) - at the time it had zero downsides. I'm not sure if that still holds true.

On Linux it generally doesn't: off-the-shelf compression is used, which do not outpace storage access latency.

sammatilda 10 hours ago

I just discovered zram thanks to claude after installing fedora on a new computer and man I was blown away by both zram and fedora. Fedora amongst the linux distributions for some reasons has really got everything that an end user needs and the whole experience is just way smoother than anything I have used in 25 years on linux.

I am hoping to switch my dad from ubuntu to fedora in a month or so because he has 16 GB ram and this seems like something most people dont know.

nico 10 hours ago

Wow, very cool. Is this like the old "RAM doubler" for Mac OS (or like Johnny Mnemonic's "Pemex MemDoubler")? :)

Lio 10 hours ago

It's funny, I switch from Fedora back to Ubuntu last year.

I really like Fedora in that it seems very up to date package wise but I found it requires a reboot every time I update using the UI updater.

It's better using the dnf command but it still felt a bit lumpy compared to apt.

mritzmann 10 hours ago

Install "needrestart", and you'll see that a reboot would be useful in Ubuntu, too. Many running programs still use the old version of a shared library even after it has been updated. With so many security updates these days, it almost always makes sense to reboot.

nosrepa 2 hours ago

You can turn that off in the settings for discover.

speed_spread 10 hours ago

I like Fedora too but I'm pretty sure Ubuntu also has zram. You just need to enable it.

amelius 9 hours ago

Could this be one of the tricks the M-series cpus are pulling behind the scenes to make them look faster?

jrmg 6 hours ago

It’s definitely one of the tricks the Neo is pulling behind the scenes to make 8Gb be ‘enough’.

swiftcoder 5 hours ago

It's nothing to do with the CPUs. Apple implemented this OS-wide back in the 2010s, long before the M-series was on anyone's radar.

mkesper 5 hours ago

zram has inherent problems and is superseded by zswap. https://chrisdown.name/2026/03/24/zswap-vs-zram-when-to-use-...

anthk 4 hours ago

Not in my case. Both a NUC with 8GB of RAM and a netbook with 1. Zram ran better. The least you touch the disk, the better.

kasabali 4 hours ago

superseded when? both have inherent problems and their usecases and I got sick of this meme of pushing zswap as the superior solution.

quotemstr 31 minutes ago

Doesn't seem like a "meme" as much as a well-argued technical position. What's the problem?

hofiflo 4 hours ago

I have a laptop with soldered RAM, a whole whopping 16GB of it. Without zswap or zram, it’s borderline unusable with web browsing, a bunch of terminals, mail client, etc. open. Maybe it’s a skill issue on my side, but only zram makes it work properly without hickups due to the OOM killer taking too long to kick in. Since switching from zswap to zram, I basically don’t encounter OOM situations and corresponding lockups anymore at all. Just my personal anecdote :)

adgjlsfhk1 3 hours ago

zswap+earlyoom/systemd-oomd is the optimal combo. Unfortunately, very few desktops ship this/configure it correctly.

hofiflo an hour ago

I guess that’s where the skill issue is coming in! That’s what I tried but probably my configuration wasn’t ideal. Maybe I should give it another shot!

hobo123 an hour ago

Not hating on Fedora, but do you really want your dad on a distro that switches to new packages (or tries new stuff) very eagerly, and that basically requires you to update to the new version every time? Also, 16GB on Linux go a long way, unless you do mem intensive stuff.

InsidiousWombat 5 minutes ago

Fedora is beta test environment for RedHat, I wouldn't use it for work machine.

Juvination 5 hours ago

What a fun loop through time. I remember downloading programs to free up ram in the early 2000's, then as RAM became more ubiqutous and programs were still small you could run them directly on RAMDisk. (I did this with Path of Exile about 10 years ago to avoid lag.) Now we are working back to getting the most out of our ram.

2001zhaozhao an hour ago

I do this and also buy old used 16GB Optane sticks to use as swap. These are $7 a pop on ebay. They're way lower latency than normal swap and you can swap data as much as you like without worrying about wear.

myrandomcomment 5 hours ago

MacOS does this by default for inactive pages. On my current system with 18GB of RAM I show 4.80GB for Apps, 3.16GB Wired and 5.06GB Compressed.

certyfreak 10 hours ago

I switched to Fedora some time ago and only recently learned about zram. I was hitting full system memory(16GB) during normal use, and after increasing my zram to 16GB, I was blown away. I had initially planned to buy additional ram.

nukanse 9 hours ago

For a programmer, on many cases, it pays to compress the data, to improve RAM IO, because the alu may already be doing nothing meanwhile waiting for the data from ram, so you can use that time to decompress, process, compress, and store the result. A lot of code today is bottlenecked by IO. You spend a lot of effort optimizing with SIMD, processing the data super fast, and then the speed it's wssted because there is not enough bandwidth to even send the result to ram. Modern cpu should include more hardware support for memory compression

bullish-bearr 39 minutes ago

Doesn't this only work if you have a bunch of stuff you do not need to access frequently in RAM? i wonder what kind of algorithm optimization goes behind the scenes to make sure pages that least recently used get compressed with other pages

Lerc 6 hours ago

I have to wonder if the better solution would be for browsers (and apps that are effectively broswers) to compress their data proactively.

I find that the ratio of browserish memory use to that of other programs is enough to render the latter insignificant most of the time.

anthk 4 hours ago

Browsers can just suspend tabs. If they are separate processes, just send a SIGSTOP except for the actual active tab and run SIGCONT upon switching.

roosgit 12 hours ago

MacOS has been doing this for quite a while now. I was a bit surprised this wasn’t the default for most Linux distros, although I think Pop!_OS (what a name!) has had zram enabled for a few years already. I turned it on on one of my servers (1GB RAM) and it works fine.

kachnuv_ocasek 10 hours ago

Fedora has had zram on by default for at least five years now.

ChocolateGod 4 hours ago

zram relies on the swap subsystem, macOS memory compression does not rely on pages being swapped.

anthk 4 hours ago

You can enforce the swappiness to 100 and virtually work in the same way.

mrob 3 hours ago

Maximum swappiness is 200. Setting it higher than 100 is possibly a good idea when using zram.

aktenlage 9 hours ago

So, if it is used as swap, that means to use the data, it first needs to go to the CPU, then back to regular RAM, then back to the CPU.

Isn't that wasteful? The CPU's caches may help a bit, but I would guess first a huge block is going through the decompression, so only the last bits are still in the cache, when the actual usage starts.

altcognito 5 hours ago

It's funny that MS didn't integrate memory compression until 2015. From ye olden days, we used "RAM doublers" in MSDOS, which were a bit of a different beast and took a performance hit due to simpler implementation, so some of us aren't "excited" by the cost of packing things in with compression. It is a bandaid for processes that lie somewhere between being needed and not (otherwise it would goto swap right?)

If it avoids hitting the disk I get it, but to some of us, that just means it's time to buy more RAM

rustyhancock 10 hours ago

I've used zswap on most of my systems and generally find it beneficial especially as (de)compression can be very fast these days.

What's interesting to note is that I've occasionally wondered why isn't the swap partition compressed.

If you have it on an NVMe drive it probably is as is all data! The controller usually transparently does it as it writes to/from NAND.

So the kernel sees uncompressed fixed block pages and the speed boosts happen with data arriving faster when read or written to flash.

close04 6 hours ago

> I've used zswap on most of my systems and generally find it beneficial

I started using zswap recently but didn't have the time yet to measure the benefit. Does it benefit machines with low amounts of RAM and wimpy CPU (like a RPi3)?

> The controller usually transparently does it as it writes to/from NAND.

In the consumer SSD space the last controllers I know did on-the-fly compression were SandForce [1], probably 15 years ago, when minimizing writes and saving the little capacity available was imperative. I don't think any other desktop SSDs do this now. A bit more common in enterprise models like ScaleFlux CSD.

[1] https://en.wikipedia.org/wiki/SandForce#Products

mkesper 5 hours ago

Where did you get that info? Sounds suspicious.

deltoidmaximus 3 hours ago

I didn't think controllers generally did any compression but I remember once that on old sandforce controllers if you did something like write all zeros it wouldn't actually overwrite anything instead just mapping to another block of zeros. Which sounded more like on device dedupping than compression.

pyaamb 5 hours ago

I wonder what the theoretical limits are of semantic compression for LLM input/output (eg. "Can you please send me the final report by Friday?" -> "Send final report by Friday." -> sr(FINAL, DEADLINE=FRIDAY)) that can still be recovered back to English

NegativeLatency 5 hours ago

https://github.com/microsoft/LLMLingua one such library I’m aware of

Tried porting it to swift, was hoping it would make mobile local models more useful for a project I was working on but it was hurting the accuracy too much. Can be very task dependent on how much you can compress. https://github.com/nburns/llmlingua-swift

My specific use case was using a LLM on segments of labeled data from a different faster model as a final pass.

grandlimelabs 3 hours ago

Make sure to stress-test compression and decompression for whatever you're doing before you go jumping into relying on it, though.

IronWolve 4 hours ago

I remember the compression rage back on my amiga, had both ram and file compression, since you had to get as much room as you could with limited ram and a floppy.

1970-01-01 3 hours ago

This feels like the kind of thing that leads to 0day. Compression can defeat security. Delta frames. Compression bombs.

https://www.youtube.com/watch?v=4icQwducz68

https://www.youtube.com/watch?v=IXkX2ojrKZQ