ESS module refactor

mklsls · May 14, 2022, 9:37pm

Hi guys!

I use R for my work, and I used RStudio Server, but for internal policies the access was closed. Trying ESS in doom the experience was a little rough. I installed spacemacs with the ESS layer and it was much smoother.

This post is to start a discussion about a refactor for the ESS module to improve the experience for the final user.

Note: I have all the motivation, but not all the lisp skills, to do this massive task. Of course, I offer my help with the refactor and co-maintain the module.

Note 2: I have started a branch in my doom’s fork to start playing around with the configuration. Please feel free to contribute as much as you want.

Initial ideas

The layout

The R editor de facto is RStudio, so I think that a good start could be to take this layout as the starting point.

I’m not attached to this, of course, but I think that this could be less intimidating to beginners that want to try Emacs as your R editor.

In my config.el I have this naive configuration for it. It needs a lot of work, though.

(set-popup-rule! “^\\*R.*\\*”  :side ‘bottom :width 0.5 :quit nil :ttl nil)
(set-popup-rule! “^\\*R Data View.*\\*” :side ‘top :slot 1 :width 0.5 :quit nil :ttl 0)
(set-popup-rule! “^\\*R dired\\*” :side ‘right :slot -1 :width 0.33 :quit nil :ttl 0)
(set-popup-rule! “^\\*help\\[R\\]”  :side ‘right :slot 1 :width 0.33 :quit nil :ttl 0))

The Buffers’ behavior

This is tricky because Emacs allows the user to have total freedom here.

In the current state of the module, if you open the *R* process and press ESC or q, it kills the buffer and, of course, the process (all the loaded data is lost).

The options here help with this problem

(set-popup-rule! “^\\*R.*\\*”  :side ‘bottom :width 0.5 :quit nil :ttl nil)

Also, I have those for the same problem.

(map! :after ess-help
      :map ess-help-mode-map
      :n “q” nil
      :n “ESC” nil)

(map! :after ess-rdired
      :map ess-rdired-mode-map
      :n “q” nil
      :n “ESC” nil)

Another idea is to use something like zoom for putting emphasis on the code, the *R* process, the environmental variables, etc.

Performance

I’m lost here, but it would be great to see if there are some configurations to do for lsp-mode and eglot.

Also, there is no configuration by default for lsp-mode over TRAMP. It’s just to put this in some place:

(lsp-register-client
  (make-lsp-client
   :new-connection
   (lsp-stdio-connection '("R" "--slave" "-e" "languageserver::run()"))
   :major-modes '(ess-r-mode inferior-ess-r-mode)
   :remote? t
   :server-id 'lsp-R-remote))

Speaking of TRAMP, I don’t know what kind of improvements we could do to reduce the latency when the user interacts with the *R* process.

More ideas

Better integration with polymode/poly-R to edit Markdown files. Also see if it’s possible to integrate it with aas y lass.
Quarto integration through quarto-emacs.
Embedded plot into a buffer. I don’t know if it is possible, but here there are some old ideas. We can explore if there is something better.

Final

This is will be my first big project for doom. All the feedback and help is very welcome.

Thanks.

fbob · May 16, 2022, 8:46am

Excellent idea and initiative, I’m not using R but ideas emerging here could also be beneficial for other scientist or data driven workflows.

I note that you mentioned quarto which I found excellent and a serious outsider for org-mode even if the latter is more powerful. Pandoc integration in quarto is really excellent.

Ideas may also be taken from vscode-r

mklsls · May 16, 2022, 1:36pm

Reviewing the vscode-r project I found some interesting ideas to explore. I’ll add them to the main post.

Thanks for the suggestion.

bryce-carson · May 17, 2022, 2:16am

Personally I dont think these should be defaults. They can be very good documentation extras, however, and we can facilitate a level of “rstudio-like-emulation” with a new package.

mklsls · June 3, 2022, 3:43pm

That sounds awesome. Could you please expand a little further how to implement this idea?

My first thought is this rstudio-mode package should be a configuration meta-package of other ones, or am I wrong?

mklsls · June 3, 2022, 3:48pm

I created a new doom’s fork to start the contributions there.

tecosaur · January 17, 2023, 2:26pm

@mklsls I see you’re still working on this, which is great to see . Henrik has had some awful luck recently with a prolonged internet outage in the middle of a big rework, then moving country. However, in a few months I think a PR could be well received.

Regarding the rstudio like layout being optional, we can just put that behind a module flag (e.g. ess +panelayout or similar).

mklsls · January 17, 2023, 3:12pm

Thanks @tecosaur for the update. Yes, I’m aware about the Henrik’s journey, I’m sure he will rise victorious. For me, there isn’t any rush about it.

With respect to the RStudio layout, your flag idea sounds awesome.

For the moment, I haven’t figured out (specifically, I don’t know) how to arrange properly the windows in Emacs. I did a couple of tweaks using set-popup-rule! for some buffers, but this is all. For the moment, I ran out of ideas.

I recall a System Crafters video about how to handle windows and buffers. I’ll rewatch to see how to start this part.

Of course, any suggestion is always well received.

Best.

kai · February 17, 2023, 3:03pm

@mklsls Thank you for putting effort into this! A feature I have been missing has been code folding. RStudio provides folding not only for functions, but also for section headers - ie

# My section ------
my_fn("x")

# Folds to next section
# |
# |
# v

# My section 2 --------

Unfortunately I have no idea how this would be replicated using ts-fold, but something tells me it must be possible.

Really great work, thank you again!

mklsls · February 25, 2023, 12:24am

Hi @kai. This weekend I’ll have to rebase the ess-refactor because Henrik did some bumps in :lang ess.

I don’t have any experience with ts-fold (or Elisp in general) but I can try to see what happens.

Best.

mklsls · February 28, 2023, 3:48pm

Reviewing a little, I found that the structure for these blocks are defined here. Basically you need at least four dashes (-), equal sign (=) or pound signs (#) after your comment.

Something like this

# Heading ---------------------------------
# Heading =================================
# Heading #############################

In ts-fold, the fold is only defined for braces (see ts-fold/ts-fold-parsers.el at 9d9e0c5cf7b5397e06571bb6bf497598dc8796a8 · emacs-tree-sitter/ts-fold · GitHub)

And tree-sitter only has defined nodes for regular comments (lines starting with #).

I was reading this section in ts-fold (GitHub - emacs-tree-sitter/ts-fold: Code-folding using tree-sitter), but I still don’t know how to add the new blocks. I need to play a little more with tree-sitter to see how it works.

Any idea to implement this is welcome.

Best.

bryce-carson · July 1, 2023, 8:09pm

@mklsls You may be interested to know that Atomic Windows (and atomic window layouts) may provide the necessary facility to emulate the columns of RStudio.

mklsls · August 7, 2023, 5:30pm

Hi @bryce-carson. I did a draft of the layout function in this discord message (Discord)

Basically, the layout is more or less like this

The WIP function is

(defun my-rstudio-layout2 () ""
       (interactive)
       (let ((ess-startup-directory 'default-directory)
             (ess-ask-for-ess-directory nil))
         (ess-switch-to-ESS t)
         (hide-mode-line-mode)
         (ess-eval-linewise "dummy<<-NULL\n")
         (let ((browse-url-browser-function 'xwidget-webkit-browse-url))
           (ess-eval-linewise (concat "source(\"" doom-emacs-dir "modules/lang/ess/helpers.R\")\n"))
           (ess-eval-linewise ".start_hgd()" t t t t 0.1 0.1)
           (xwidget-webkit-browse-url (replace-regexp-in-string "\"" "" (ess-string-command "dput(.server_hgd)\n")))
           (hide-mode-line-mode))
         (ess-switch-to-ESS t)
         (ess-rdired)
         (hide-mode-line-mode)
         (ess-help "help")
         (hide-mode-line-mode)
         (let* ((source-buf (car (match-buffers '(and (derived-mode . ess-mode) "\\.[Rr]$"))))
                (plot-buf (car (match-buffers '(and (derived-mode . xwidget-webkit-mode) "R\\ Plot\\*$"))))
                (repl-buf (ess-get-current-process-buffer))
                (rdired-buf (get-buffer ess-rdired-buffer))
                (help-buf (car (ess-help-get-local-help-buffers))))
           (if source-buf
               (progn
                 (pop-to-buffer source-buf)
                 (condition-case nil (delete-other-windows) (error t))
                 (display-buffer-in-atom-window
                  repl-buf
                  `((window        . ,(get-buffer-window source-buf))
                    (side          . below)
                    (window-height . 0.5)
                    ;; (window-width  . 0.5)
                    ;; (window-height . ,#'fit-window-to-buffer)
                    (dedicated     . t)))
                 (display-buffer-in-atom-window
                  rdired-buf
                  `((window        . ,(get-buffer-window source-buf))
                    (side          . right)
                    (window-width  . 0.5)
                    (window-height . 0.5)
                    ;; (window-width  . ,(lambda (win)
                    ;;                     (fit-window-to-buffer win
                    ;;                                           nil nil
                    ;;                                           (floor (frame-width) 3)
                    ;;                                           40)))
                    (dedicated     . t)))
                 (display-buffer-in-atom-window
                  plot-buf
                  `((window        . ,(get-buffer-window repl-buf))
                    (side          . right)
                    ;; (window-width  . ,(lambda (win)
                    ;;                     (fit-window-to-buffer win
                    ;;                                           nil nil
                    ;;                                           (floor (frame-width) 3)
                    ;;                                           40))))
                    ))

                 nil)))))

I plan to install this function into the feature branch and see if there are any comments or improvements.