Breaking change on b9933e663771

A heads up to Doom users: doom upgrade will break itself when updating Doom past b9933e6. The only way to avoid this is to update manually (with git pull && doom sync -u), but most users will likely have already run doom upgrade before they read this, so:

What happens?

You will first see this error:

Click to reveal error message
> Preparing to upgrade Doom Emacs and its packages...
- A new version of Doom Emacs is available!

    Old revision: a9866e37e4 (3 hours ago)
    New revision: 615848e3a7 (52 minutes ago)

Link to diff: https://github.com/doomemacs/doomemacs/compare/a9866e37e45b43785116ef474c8cd6aa9b5185dd...615848e3a71ca9ecd4e50cd7abc49eedc2fbea23
View the comparison diff in your browser? (y or n) n
Proceed with upgrade? (y or n) y
> Upgrading Doom Emacs...
  > Cleaning .elc files
    - No elc files to clean
  x There was an unexpected runtime error
    Message: File is missing
    Details: ("Opening input file" "No such file or directory" "/home/hlissner/emacs.d/core/packages.el")
    Backtrace:
      (insert-file-contents "/home/hlissner/emacs.d/core/packages.el")
      (progn (insert-file-contents (doom-path doom-core-dir "packages.el")) (if (re-search-forward "(package! straight" nil t) (pr...
      (unwind-protect (progn (insert-file-contents (doom-path doom-core-dir "packages.el")) (if (re-search-forward "(package! stra...
      (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents (doom-path doom-core-dir "package...
      (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn ...
      (doom-upgrade--get-straight-recipe)
      (equal straight-recipe (doom-upgrade--get-straight-recipe))
      (if (equal straight-recipe (doom-upgrade--get-straight-recipe)) nil (doom-print (doom-print--format (doom-print--class 'item...
      (let ((straight-recipe (doom-upgrade--get-straight-recipe))) (or (and (= 0 (car (doom-call-process "git" "reset" "--hard" ta...
      (let ((doom-print-indent (+ (if t doom-print-indent-increment 0) doom-print-indent))) (doom-compile-clean) (let ((straight-r...
      (if (not (or auto-accept-p (y-or-n-p "Proceed with upgrade?"))) (ignore (doom-print (doom-print--format (doom-print--class '...
      (cond ((and (null this-rev) (null new-rev)) (error "Failed to get revisions for %s" target-remote)) ((equal this-rev new-rev...
    ! Wrote extended backtrace to ~/emacs.d/.local/logs/cli.doom.220731015104.962288.error

And from that point on doom upgrade will no longer work; always throwing up Failed to fetch from upstream errors and failing to upgrade. A git pull alone may not be enough to recover from this.

The workaround

To spare you the pain of nuking your Doom install and reinstalling everything, an easy workaround is to first reset to a safe commit (like a9866e3) before upgrading manually:

$ cd ~/.emacs.d
$ git reset --hard a9866e37e45b43785116ef474c8cd6aa9b5185dd
$ git pull origin master
$ doom sync -u

Hope that helps!


:warning: Another heads up: the next few weeks will be a little unstable while the rewritten CLI trickles in. In particular, there is one more breaking change coming up sometime in August that will require a reinstall of all your packages (thanks to a rewritten package manager) and an update to your doom! block (thanks to Doom’s modules being moved out to its own repo, plus a new syntax for pulling in remote module libraries from your doom! block).

I’ll post an announcement like this one when it happens.

30 Likes

thanks for the heads-up.

Once the CLI rework will be complete and stable, will there be an upgrade path using the CLI itself or the alternate upgrade path (i.e. git pull && doom sync) will still be needed?

In other words: can users just wait a few weeks for the changes to be stable and then upgrade with doom upgrade as usual?

thanks for working on Doom! :cacopog:

1 Like

Yes, once done and stable, doom upgrade will be that upgrade path. The manual method exists only for emergencies (and may change, but will always be documented in the output of doom help upgrade). [1]

If, by then, you’ve updated beyond b9933e6 then it may work. But if you haven’t, then no. The manual update is required.

The problem right now is that I’m adding all of doom upgrade's safety and automatic migration mechanics between now and the v3.0 release, but that doesn’t help folks upgrading to v3.0 (and the changes aren’t backwards compatible), so a manual update will likely be necessary once more. I’ll go into more detail once in the announcement for that release though.


  1. There will be one more method after the v3 release, where you can pin Doom in your doom! block, and doom sync will up/downgrade itself as desired, but I’ll share more on that another time. ↩︎

3 Likes

I think those steps worked for me, but at least now I’m getting a different error, maybe that’s fine for now?

I ran what you described above and then after those steps when I run doom upgrade I get:

WARNING (:ui pretty-code) module was moved to (:ui ligatures)
> Preparing to upgrade Doom Emacs and its packages...
x There was an unexpected runtime error
  Message: Failed to fetch from upstream
  Backtrace:
    (error "Failed to fetch from upstream")
    (or (= 0 (car (setq result (sh! "git" "fetch" "--force" "--tags" doom-upgrade-remote (format "%s:%s" branch t...
    (let (result) (or (= 0 (car (sh! "git" "remote" "add" doom-upgrade-remote doom-upgrade-url))) (error "Failed ...
    (unwind-protect (let (result) (or (= 0 (car (sh! "git" "remote" "add" doom-upgrade-remote doom-upgrade-url)))...
    (let* ((branch (replace-regexp-in-string "^\\(?:[^/]+/[^/]+/\\)?\\(.+\\)\\(?:~[0-9]+\\)?$" "\\1" (cdr (sh! "g...
    (let ((default-directory doom-emacs-dir) process-file-side-effects) (doom-print (doom-print--format (doom-pri...
    (doom-cli-upgrade #s(doom-cli-context (25320 103 197247 200000) 91919 1 (117 . 83) nil nil "doom" nil nil ("d...
    (cond (packages\? (let ((recipe (doom-cli-context-get context 'straight-recipe))) (if (and recipe (not (equal...
    (let* ((force\? (progn (or (progn (and (memq (type-of context) cl-struct-doom-cli-context-tags) t)) (signal '...
    (let ((packages\? (cdr (assq 'packages\? alist))) (jobs (cdr (assq 'jobs alist))) (context (cdr (assq 'contex...
    ((closure (t) (cli alist) (let ((packages\? (cdr (assq 'packages\? alist))) (jobs (cdr (assq 'jobs alist))) (...
    (funcall (closure (t) (cli alist) (let ((packages\? (cdr (assq 'packages\? alist))) (jobs (cdr (assq 'jobs al...
  ! Wrote extended backtrace to ~/.emacs.d/.local/logs/cli.doom.220801183343.91919.error

Let me know if I can provide any more info if that helps I’m on emacs: GNU Emacs 28.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.17.6) of 2022-04-28

4 Likes

I think you have to use the workaround before going past the point of no return with doom upgrade.

If this has happened, just do rm -rf ~/.emacs.d and clone the repository again.

Hi! I’ve tried the workaround, but it hasn’t solved the issue - I’m gettingt the following when running doom sync -u:


C:\Users\Jonathan\.emacs.d>doom sync -u
> Synchronizing "default" profile...
  > Regenerating envvars file
    e[032m✓ Generated ~/.emacs.d/.local/enve[0m
  > Installing packages...
    > Building visual-fill-column...
    e[031mx There was an unexpected runtime errore[0m
      e[01mMessage:e[0m Invalid sub-list head in :files directive: :not
      e[01mBacktrace:e[0m
        (signal error ("Invalid sub-list head in :files directive: :not"))
        (error "Invalid sub-list head in :files directive: %S" :not)
        (if (stringp (car spec)) nil (error "Invalid sub-list head in :files...
        (cond ((eq spec :defaults) (setq files (append straight-default-file...
        (let ((spec (car files))) (setq files (cdr files)) (cond ((eq spec :...
        (while files (let ((spec (car files))) (setq files (cdr files)) (con...
        (let ((mappings nil) (exclusions nil)) (while files (let ((spec (car...
        (straight--expand-files-directive-internal (:defaults (:not "Cask" "...
        (let* ((default-directory src-dir) (result (straight--expand-files-d...
        (straight-expand-files-directive (:defaults (:not "Cask" "LICENSE" "...
        (let ((--dolist-tail-- (straight-expand-files-directive files (strai...
        (progn (let ((dir (straight--build-dir package))) (if (file-exists-p...
      e[033m! Wrote extended backtrace to ~/.emacs.d/.local/logs/cli.doom.221023130444.8112.errore[0m
    e[033m! Script was abruptly aborted, leaving Doom in an incomplete state!e[0m
    - Run 'doom sync' to repair it.
    e[032m✓ Finished in 1m 42se[0m

Is there any way to avoid nuking my install at this point? Thanks!

1 Like

@Jonathan_Sahar

I came across the same, at this point it has got to be less trouble to just fire in the hole

Is this still an issue or is it fixed?

1 Like

Is this still an issue? I haven’t upgraded my Doom Emacs since July since I’m afraid that it might break if I do. The OP says the a new announcement would be released within a few weeks, but it seems to have been a few months, and no new announcements have been made regarding this problem.

2 Likes

What should I do if I did not update since February 2022? If I just do doom upgrade will I encounter the bug? Should I wait some more time before upgrading?

1 Like