Fix for doom build `Too many open files` on native-comp

[jming422]

What happened?

During doom build, when feature NATIVE_COMP is available, the command failed with error:

    x There was an unexpected error
      Message: File error
      Error: (file-error "Creating pipe" "Too many open files")
      Backtrace:
        (make-process :name "Compiling: /Users/jming/.config/doom-emacs/.local/str...
        (comp-run-async-workers)
        (native--compile-async "/Users/jming/.config/doom-emacs/.local/straight/bu...
        (native-compile-async "/Users/jming/.config/doom-emacs/.local/straight/bui...
        (let ((inhibit-message t) (message-log-max nil)) (native-compile-async bui...
        (let ((build-dir (straight--build-dir package))) (if (and straight--build-...
        (let* ((--cl-rest-- recipe) (package (car (cdr (plist-member --cl-rest-- '...
        (progn (require 'comp) (let* ((--cl-rest-- recipe) (package (car (cdr (pli...
        (if (and straight--native-comp-available (member 'straight--build-compile ...
        (straight--build-native-compile (:type git :flavor melpa :files (:defaults...

This is not doom’s fault but macOS’s, and I wanted to share an easy way to get the build to run successfully!

Workaround to fix it

By default, macOS limits processes to only opening 256 files at a time. You can temporarily raise this limit in your current shell by running:

ulimit -n 1024

Or any other number, but 1024 was more than enough for doom build to complete for me.

Why does this happen?

This happens because of a limit imposed by macOS. I learned about this from the related discussion here:

Creating pipe: too many open files after `SPC *`

Seems that MacOs is the culprit here, I don’t remember the exact reason for this, but if you google a bit you will stumble upon reddit and stack overflow threads talking about this same exact issue, but when trying to do different things, although always trying to manipulate a certain amount of files. A common example is when trying to use magit on a large project. Anyway, I have fixed this following this gist The error still pop ups from time to time, but less than before.

But since that discussion wasn’t related to the CLI or native-comp at all, I wanted to share this workaround here for more visibility if doom build fails with this error. Also, since doom build is typically run from a shell, you can use the much simpler & more temporary ulimit solution instead of having to go through all the steps in the linked gist to change the system-wide default limit.

System information

Loading data dump...

[memory-dealer]

This worked like a charm. Thank you so much.