Doom and XDG directories

Development core
#1

The basic idea

We all love XDG, and while ~/.config/doom and ~/.config/emacs are very nice, I can’t help but feel that shoving so much under ~/.config/emacs is somewhat abusing the idea of a “configuration directory”.

I’d like to start a conversation about how things might be able to be shuffled around a bit to fit the spirit of the XDG directories more, as I think a change like this would fit well with the large shifts currently going on with Doom.

A recap of the XDG dirs

XDG_?_HOME Default Purpose
DATA $HOME/.local/share User-specific data files
CONFIG $HOME/.config User-specific configuration files
STATE $HOME/.local/state State data that should persist between (application) restarts, but that is not important or portable enough to the user that it should be stored in $XDG_DATA_HOME
CACHE $HOME/.cache User-specific non-essential data files
RUNTIME /run/user/$UID User-specific non-essential runtime files and other file objects (such as sockets, named pipes, …)
$HOME/.local/bin User-specific executable files

For windows and mac equivalents, see GitHub - OpenPeeDeeP/xdg: A cross platform package that follows the XDG Standard.

A tentative proposal

To kick things off, I’ve just taken a few of the dirs that Doom uses off the top of my head and posited where the most appropriate place for them to exist might be.

If there are any good suggestions, I’ll edit this listing to apply them.

Data

Basically the “important stuff”, so I’m thinking:

  • Packages
  • Documentation
  • Doom module definitions (maybe?)

Config

Well, we already have ~/.config/doom…

That said, profiles could go here.

State

A few things come to mind here, namely:

  • Recent files
  • Bookmarks
  • Known projects
  • Transient history
  • Scratch content
  • Forge info
  • Various other history-type files
  • Various log files

Cache

I take this as files that could be deleted where the user shouldn’t observe any different behaviour as a result.

For starters we have ~/.config/emacs/.local/cache/, and everything contained therin.

Then there’s the ELPA/MELPA/etc. archives/caches.

In addition, I think the ~/.config/emacs/.local/env file might go here, particularly if it could be automatically regenerated.

Lastly, if they’re (re)generated on-the-fly, then .elc and .eln files could potentially fit here, but nothing else comes to mind.

Runtime

The one thing that could fit here is the emacs-server files, currently in ~/.config/emacs/server/.

Binary files

Currently we have ~/.config/emacs/bin/{doom,doom.cmd,doomscript,org-capture,org-tangle}. Perhaps these could be symlinked into ~/.local/bin ?

What does that leave in ~/.config/emacs?

I’d think just enough to set up load paths and kick things off. Possibly just early-init.el?

I’d imagine that when doom install is run, it would move the main repo into say $XDG_DATA_DIR/emacs/doom, early on and then set up the rest and so allow everything other than the very first steps to assume the layout described above.

Actually implementing the restructuring

There are at least two ways we could go about this.

  1. Put everything in what we consider to be XDG-appropriate location, and have a helper script/function to do this reorganisation to the current layout
  2. Use the XDG-appropriate locations on new installs, and symlink to recreate the current layout
2 Likes
#2

I embraced doom emacs recently. And I use Guix to manage it as a system package. For simplicity, I redirected DOOMLOCALDIR to ~/.local/share/doomemacs

After this, I can do

  • doom install
  • doom sync

If we use doom run to launch a doom(emacs) instance, then .config/emacs is no longer relevant here, since doom has it’s logic to set user-home-directory. I created a wrapper called doomeamcs to doom run. So I almost can use doomemacs as an alternative to emacs.

  • doomemacs
  • doomemacs --daemon
  • doomeamcs --fg-damone
  • emacsclient

worked just fine.

Splitting DOOMLOCALDIR into xdg cache/data/state is nice to have I think.

My point here been:

  • Doom can work as a system package, no need to symlink bins into ~/.local/bin,
  • ~/.config/emacs is no long relevant is we use doom run

My Guix package definition for doomemacs

(define-public doomemacs
  (package
    (name "doomemacs")
    (version "v2.0.9")
    (source
      (origin
        (method git-fetch)
        (uri (git-reference
               (url "https://github.com/doomemacs/doomemacs")
               (commit "12bf6baa213686afb80b62c19773fdddad14e88e")))
        (file-name (git-file-name name version))
        (sha256
         (base32 "02ll6aps3ji02lipsmnq0p8cw1ggjp62nybanqbgvfs2qhqjwsfk"))
        (modules '((guix build utils)))
        (snippet '(for-each delete-file-recursively
                            '(".github"
                              "bin/doom.cmd"
                              "shell.nix")))))
    (build-system copy-build-system)
    (inputs
     (list emacs-pgtk-native-comp))
    (arguments
     (list #:install-plan #~`(("./" "."))
           #:phases
           #~(modify-phases %standard-phases
               (add-after 'install 'install-doom-run-wrapper
                 (lambda* (#:key inputs outputs #:allow-other-keys)
                   (with-output-to-file (string-append #$output "/bin/doomemacs")
                     (lambda ()
                       (display
                        (string-append
                         "#!/usr/bin/env bash\n"
                         "exec -a \"$0\" \"doom\" \"run\" \"$@\""))))
                   (chmod (string-append #$output "/bin/doomemacs") #o555)
                   (wrap-program (string-append #$output "/bin/doom")
                     '("DOOMLOCALDIR" = ("${DOOMLOCALDIR:=${XDG_DATA_HOME:=$HOME/.local/share}/doomemacs}"))
                     `("EMACS" = (,(search-input-file inputs "/bin/emacs")))))))))
    (home-page "https://doomemacs.org")
    (synopsis "An Emacs framework for the stubborn martian hacker")
    (description
     "Doom is a configuration framework for GNU Emacs tailored for Emacs
bankruptcy veterans who want less framework in their frameworks, a modicum
of stability (and reproducibility) from their package manager, and the
performance of a hand rolled config (or better). It can be a foundation
for your own config or a resource for Emacs enthusiasts
to learn more about our favorite operating system.")
    (license license:expat))) ; MIT license
#3

I would argue against splitting doom up this much. Using XDG dirs for auto generated and downloaded files is good, but i think hacking on doom would become weirder if the git repo is split up in a bunch of places. Emacs is configured using source code, I think it makes sense to keep doom config in XDG_CONFIG_HOME

#4

Hacking on doom would become weirder if the git repo is split up in a bunch of places.

In my XDG proposal, I do not see the Doom repo being split up. What I see happening is something like:

  • The Doom repo living in $XDG_DATA_DIR/doom
  • A (very) tiny helper script written to $XDG_CONFIG_DIR/emacs/early-init.el
#5

XDG_DATA_DIR must have all the modules imo, under DATA_DIR/doom/modules. That’s what packer and nvim do basically and that is what ultimately allows to have any external package managed by a 3rd party program (packer-equivalent: doom! macro, or a stow script or a manual git routine or submodules or…) so that another one (nvim-equivalent: doom-find-and-load-module-by-name function - I just made that name up) can find those to load things:

tree -L 2 .local/share/nvim/site/pack/packer/
.local/share/nvim/site/pack/packer
├── opt
│  └── nvim-lspconfig
└── start
   ├── aniseed
   ├── cmp-buffer
   ├── cmp-nvim-lsp
   ├── cmp_luasnip
   ├── Comment.nvim
   ├── conjure
...
   └── vim-startify

At least I find that system useful, because that would mean CLI module handling would really be close to «just» clone a folder or a collection of folders at the right place. It would also be possible to add extra path levels (in the case of nvim here, packer, start and opt are technically extra info), so that you could have at the same level either a collection of modules (like the 1st-class citizens Doom-maintained modules), or, in arbitrary locations, isolated modules/collections (coming from other git repositories or tarballs)

#6

I’ve planned to adhere to XDG more after the new CLI and package manager are done, with the exception of XDG_RUNTIME_HOME – where emacsclient searches for sockets/pipes is hardcoded (either to already use $XDG_RUNTIME_HOME or $EMACSDIR/server/). Changing this would be troublesome.

The new dir variables currently look like this:

I agree with the rest, but I think this belongs in state, because deleting it break things (e.g. PATH) and isn’t regenerated without user intervention (plus, $ doom env can’t be reliably automated because it’s too sensitive to environmental contamination, and there may be desktop/session envvars lost if we start with a blank slate, i.e. $ env -i $SHELL -ic "doom env").

This makes sense, but rerouting where *.elc files get written may be a challenge – not so much for straight/elpaca, but for package.el or free packages in the user’s filesystem, or packages that reference resources with a relative filepath. I’ll look into this.

I’ll put this behind a doom install flag or one of its questions (it will be an install wizard in the future), but otherwise (particularly for contributors) I think it’d be too imposing to forcibly inject itself into your $PATH, as a default.

I think that’s too extreme, and complicates the story with Doom sub-profiles (if you have multiple Doom installations; say, contributors who are testing a module). Whats more, the new Doom core will come with pre-made profiles (e.g. a safe-mode to fall back to if the primary one fails for whatever reason, a bin/doom porcelain for Windows users, and some sandbox ones), some core modules (for the elisp development tools, e.g. sandbox, ci/cd build tools, test runners, linters, documentation builders, etc), and, of course, Doom’s core files under lisp/, test/, templates/ and docs/.

Though, I could see value in sync’ing docs to {doom-data-dir}/docs/ – this gives me an opportunity to simplify the link spaghetti I’ve set up for our current docs (made more complicated by arbitrary module libraries pulled by the user in the future). This’ll have to wait until 3.1 or 3.2 though.

That’s the plan. DATA_DIR/doom/PROFILE-NAME/@/latest/modules/ to be specific (or {doom-profile-init-dir}/modules/). These will be symlinked for deduping purposes across doom sync generations.

And perhaps, at some point, deduped across multiple profiles, but for now that’s a post-3.0 problem.