One big difference that I’ve noticed between Windows and Linux is that Windows does a much better job ensuring that the system stays responsive even under heavy load.
For instance, I often need to compile Rust code. Anyone who writes Rust knows that the Rust compiler is very good at using all your cores and all the CPU time it can get its hands on (which is good, you want it to compile as fast as possible after all). But that means that for a time while my Rust code is compiling, I will be maxing out all my CPU cores at 100% usage.
When this happens on Windows, I’ve never really noticed. I can use my web browser or my code editor just fine while the code compiles, so I’ve never really thought about it.
However, on Linux when all my cores reach 100%, I start to notice it. It seems like every window I have open starts to lag and I get stuttering as the programs struggle to get a little bit of CPU that’s left. My web browser starts lagging with whole seconds of no response and my editor behaves the same. Even my KDE Plasma desktop environment starts lagging.
I suppose Windows must be doing something clever to somehow prioritize user-facing GUI applications even in the face of extreme CPU starvation, while Linux doesn’t seem to do a similar thing (or doesn’t do it as well).
Is this an inherent problem of Linux at the moment or can I do something to improve this? I’m on Kubuntu 24.04 if it matters. Also, I don’t believe it is a memory or I/O problem as my memory is sitting at around 60% usage when it happens with 0% swap usage, while my CPU sits at basically 100% on all cores. I’ve also tried disabling swap and it doesn’t seem to make a difference.
EDIT: Tried nice -n +19, still lags my other programs.
EDIT 2: Tried installing the Liquorix kernel, which is supposedly better for this kinda thing. I dunno if it’s placebo but stuff feels a bit snappier now? My mouse feels more responsive. Again, dunno if it’s placebo. But anyways, I tried compiling again and it still lags my other stuff.
The Linux kernel uses the CPU default scheduler, CFS, a mode that tries to be fair to all processes at the same time - both foreground and background - for high throughput. Abstractly think “they never know what you intend to do” so it’s sort of middle of the road as a default - every CPU cycle of every process gets a fair tick of work unless they’ve been intentionally nice’d or whatnot. People who need realtime work (classic use is for audio engineers who need near-zero latency in their hardware inputs like a MIDI sequencer, but also embedded hardware uses realtime a lot) reconfigure their system(s) to that to that need; for desktop-priority users there are ways to alter the CFS scheduler to help maintain desktop responsiveness.
Have a look to Github projects such as this one to learn how and what to tweak - not that you need to necessarily use this but it’s a good point to start understanding how the mojo works and what you can do even on your own with a few sysctl tweaks to get a better desktop experience while your rust code is compiling in the background. https://github.com/igo95862/cfs-zen-tweaks (in this project you’re looking at the set-cfs-zen-tweaks.sh file and what it’s tweaking in / so you can get hints on where you research goals should lead - most of these can be set with a sysctl)
There’s a lot to learn about this so I hope this gets you started down the right path on searches for more information to get the exact solution/recipe which works for you.
I’d say nice alone is a good place to start, without delving into the scheduler rabbit hole…
“they never know what you intend to do”
I feel like if Linux wants to be a serious desktop OS contender, this needs to “just work” without having to look into all these custom solutions. If there is a desktop environment with windows and such, that obviously is intended to always stay responsive. Assuming no intentions makes more sense for a server environment.
Even for a server, the UI should always get priority, because when you gotta remote in, most likely shit’s already going wrong.
Totally agree, I’ve been in the situation where a remote host is 100%-ing and when I want to ssh into it to figure out why and possibly fix it, I can’t cause ssh is unresponsive! leaving only one way out of this, hard reboot and hope I didn’t lose data.
This is a fundamental issue in Linux, it needs a scheduler from this century.
One of my biggest frustrations with Linux. You are right. If I have something that works out of the box on windows but requires hours of research on Linux to get working correctly, it’s not an incentive to learn the complexities of Linux, it’s an incentive to ditch it. I’m a hobbyist when it comes to Linux but I also have work to do. I can’t be constantly ducking around with the OS when I have things to build.
What do you even mean as serious contender? I’ve been using Linux for almost 15 years without an issue on CPU, and I’ve used it almost only on very modest machines. I feel we’re not getting your whole story here.
On the other hand whenever I had to do something IO intensive on windows it would always crawl in these machines
You are getting the whole story - not sure what it is you think is missing. But I mean a serious desktop contender has to take UX seriously and have things “just work” without any custom configuration or tweaking or hacking around. Currently when I compile on Windows my browser and other programs “just works” while on Linux, the other stuff is choppy and laggy.
The Linux kernel uses the CPU default scheduler, CFS,
Linux 6.6 (which recently landed on Debian) changed the scheduled to EEVDF, which is pretty widely criticized for poor tuning. 100% busy which means the scheduler is doing good job. If the CPU was idle and compilation was slow, than we would look into task scheduling and scheduling of blocking operations.
Responsiveness for typical everyday usage is one of the main scenarios kernels like Zen/Liquorix and their out of the box scheduler configurations are meant to improve, and in my experience they help a lot. Maybe give them a go sometime!
Edit: For added context, I remember Zen significantly improving responsiveness under heavy loads such as the one OP is experiencing back when I was experimenting with some particularly computationally intensive tasks
https://github.com/zen-kernel/zen-kernel/wiki/Detailed-Feature-List
That’s the reason I installed Zen too and use it as the default. While Zen is meant to improve responsiveness of interactive usage on the system, it comes at a price. The overall performance might decrease and it should require more power. But if someone needs to solve the problem of the OP (need to work on the computer while under heavy load), then Zen is probably the right tool. Some distributions have the Zen Kernel in their repository and the install process is straightforward.
nice +5 cargo build
nice is a program that sets priorities for the CPU scheduler. Default is 0. Goes from -19, which is max prio, to +19 which is min prio.
This way other programs will get CPU time before cargo/rustc.
So the better approach would be to spawn all desktop and base GUI things with nice -18 or something?
No. This will wreak havoc. At most at -1 but I’d advise against that. Just spawn the lesser-prioritised programs with a positive value.
It’s more of a workaround than a solution. I don’t want to have to do this for every intensive program I run. The desktop should just be responsive without any configuration.
You could give your compiler a lower priority instead of upping everything else.
I’d still need to lower the priority of my C++ compiler or whatever else intensive stuff I’d be running. I would like a general solution, not a patch just for running my Rust compiler.
The System76 scheduler helps to tune for better desktop responsiveness under high load: https://github.com/pop-os/system76-scheduler I think if you use Pop!OS this may be set up out-of-the-box.
You could try using nice to give the rust compiler less priority (higher number) for scheduling.
This seems too complicated if I need to do that for other programs as well.
