Warnings and slowness at startup due to native-compilation

What happened?

Startup time is a little bit long (1.x second) and many warnings on startup Async-native-compile-log.

What did you expect to happen?

I wanna to know whether it is normal or there is some fix I can do.

Steps to reproduce

  1. Start Emacs.

System information

Loading data dump...

And here is the dump of startup Async-native-compile-log: autoload-warnings.log (11.1 KB)

1 Like

This is normal (as of 3853dff5e116). This is Emacs natively compiling your packages as they get loaded. This will only happen the first time you load each of those packages. Packages will need to be recompiled if you up/downgrade Emacs, however (or if you delete the native bytecode in ~/.emacs.d/.local/cache/eln/).

But since native-comp appears to have stability issues on and off, from version to version of Emacs, I am rethinking the change, and/or looking for a way for users to reliably toggle it.

Having a way to toggle will be a nice. After the updates yesterday, I had emacs lockup on startup quite a few times and I wasn’t able to C-g out of it. Sending USR2 sometimes worked but only after sending it a few times but the backtrace produced contained nothing relevant, most of the time it consisted of just a substitute-command-keys("Quit"). A couple times the lokcup was accompanied by runaway memory usage but most of the times not.

I don’t have any evidence that this was related to native compilation except that emacs eventually stabilized itself and now I can open it without locking it up and I assume that this is because everything I use has been compiled.

Try adding this to $DOOMDIR/init.el:

(setq native-comp-async-jobs-number 1)

It will limit the number of jobs native-comp will do at a time; thus limiting the memory consumption and load on the system. It’s something I’m considering mainlining, given the similar complaints I’ve heard about it so far.

Thanks for your reply, it helps a lot!

p.s. The .emacs.d/.local/autoload.el is compiled everytime I launch Emacs, is that normal too? Even no any package being updated ever since?

The situation happened to me again because ran doom build -r thinking it will just build the changed packages (for me it used to build just the local packages) but it built everything. So when I restarted emacs the problem of locking up started happening again. I had to use USR2 a lot of times but I did finally manage to get a decent backtrace which lead me to believe that the problem is with doom’s incremental loading of package. The way lock up happens, just a little after the startup also makes me think that is the reason since incremental loading happens on idle timers.

Putting (setq doom-incremental-packages nil) in my config fixed the issue. I updated doom which rebuilt all my packages after this change and now emacs starts fine. I can see native compilation happening in the background and it is seamless.

The lockup were really bad. In one case the emacs consumed about 10GB of my ram really quickly before I sent SIGTERM to it, but mostly it was just emacs continuously hogging one of my computer’s cores.

While incremental loading is certainly a contributor, it is not the root of the problem. Are you on a PGTK build of Emacs, by chance? I find this to be the common denominator with folks complaining about native-comp lock ups and, if so, I’d highly recommend moving back to 28.1 stable, or at least a non-pgtk build. I suspect it will address the runaway memory usage as well.

I am indeed on a PGTK build specifically this one Show home:agraul / emacs-pgtk-nativecomp - openSUSE Build Service which is a backport to emacs 28.

PGTK is the only one that works on wayland and I am not feeling any pain after disabling incremental loading so I think I will stick to it for now. But I will keep this in mind if/when I encounter future problems, so thanks a lot.

Hi, PGTK is not an option of the emacs-28 branch, only the master branch. 28.1 and 28.2 both (with nativecomp) showed the unwanted behavior in my case. I will try building an 28.2 without nativecomp option.

This issue should be resolved as of this commit:

Update Doom and you should no longer see these lockups, freezes, or runaway memory usage, even without the fixes I’ve suggested earlier in this thread (I’d recommend doing away with all of them, in fact).

Thanks a lot! I can confirm there are no more lockups after updating to the current commit.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.