Remote Intelephense with Emacs LSP

I have come to rely on Intelephense for any kind of PHP hackery, and this includes when I’m quickly editing a file on my own remote server, or in a vagrant box or what have you. Remote editing is one of the big strengths for me when it comes to using Emacs, and TRAMP has been serving me very well for it, by default so LSP servers don’t “just work”(TM) when accessing a remote file as the local LSP server won’t have access to the files. Luckily lsp-mode already has provisions for it so and will recognize that a file is remote, and startup an LSP server on the target machine if available, but it requires some configuration. For Intelephense this means making sure it is installed on the remote, easily done via NPM, and setting up a remote client configuration somewhere, for me best done as part of initializing lsp-mode. 

(use-package lsp-php
  :config
  ;; register remote intelephense
  (lsp-register-client
   (make-lsp-client :new-connection 
                    (lsp-tramp-connection lsp-intelephense-server-command)
                    :activation-fn (lsp-activate-on "php")
                    :priority -1
                    :notification-handlers 
                    (ht ("indexingStarted" #'ignore)
                        ("indexingEnded" #'ignore))
                    :initialization-options 
                    (lambda ()
                      (list :storagePath lsp-intelephense-storage-path
                            :globalStoragePath 
                            lsp-intelephense-global-storage-path
                            :licenceKey lsp-intelephense-licence-key
                            :clearCache lsp-intelephense-clear-cache))
                    :multi-root lsp-intelephense-multi-root
                    :completion-in-comments? t
                    :remote? t
                    :server-id 'iph-remote
                    :synchronize-sections '("intelephense"))))

I am reusing the lsp-intelephense-server-command and most of the other lsp-intelephense variables as they contain the right configuration, and I haven’t had an issue, even so lsp-intelephense-storage-path might need configuration depending on the value and setup, but it has worked for me. Importantly server-id and remote? need to be set for lsp-mode to handle the remote server correctly, but this has worked like a charm for me. Having the same tooling both locally and remote sure is nice!

MariaDB Kubernetes Setup

Based on the tutorial Laravel in Kubernetes I have been setting up a Databased in Kubernetes, since my preferred choice is MariaDB, and it tends to largely work the same as MySQL, used in this tutorial I decided to switch it out, as I am running locally and not on a cloud provider a couple changes are required.

Providing storage

As my local install of Kubernetes doesn’t use a cloud provider for storage but is self contained on my desktop, I am using local-path-provisioner to provide the backend for the persistent volume claims, installing worked straight out of the box, no issues.

Provisioning MariaDB

MariaDB is providing images to use on hub.docker.com so no issues, here and since MariaDB works mostly like MySQL the configuration options are very similar, except switching the names, which is actually optional. The big issue I ran into is that in the original tutorial a livenessProbe and a readinessProbe are setup to restart the container if required. 

livenessProbe:
  exec:
    command:
    - bash
    - -c
    - mysqladmin -u ${MARIADB_USER} -p${MARIADB_PASSWORD} ping
  initialDelaySeconds: 10
  periodSeconds: 5
  timeoutSeconds: 5
readinessProbe:
    exec:
      command:
      - bash
      - -c
      - mysql -u ${MARIADB_USER} -p${MARIADB_PASSWORD} -e "SELECT 1"
    initialDelaySeconds: 5
    periodSeconds: 2
    timeoutSeconds: 1

A readinessProbe does not however delay the livenessProbe also the time given, 10sec, seems to be to short, at least in my setup so my databased continued to be restarted during initialization leaving it not working since the MariaDB startup script will only attempt the database setup if the data directory is empty, which it will only be on the first start. This leaves the database in a state without users, or properly set passwords. The fix, and likely “better” way of handling this for me was replacing the readinessProbe with a startupProbe. 

livenessProbe:
  exec:
    command:
    - bash
    - -c
    - mysqladmin -u ${MARIADB_USER} -p${MARIADB_PASSWORD} ping
  periodSeconds: 10
startupProbe:
  exec:
    command:
    - bash
    - -c
    - mysql -u ${MARIADB_USER} -p${MARIADB_PASSWORD} -e "SELECT 1"
  periodSeconds: 10
  failureThreshold: 60 # total 60x10s = 600s = 10min to initialize

The final setup that works for me I uploaded in a Gist seperated into files for each part. With all the files present and local-path-provisioner installed kubectl apply -f . should work, make sure to replace the secrets in secrets.yml with base64encoded proper values.

New Scala 3 syntax in Emacs

Scala 3 has a new “quiet” syntax, which means it uses whitespace and indention to delineate blocks of code, instead of the parentheses. Currently emacs-scala-mode does not support this yet fully, and while it is in the works the quick solution is to turn of automatic indention for scala 3 projects. To achieve this two things are required, it needs to be detected that a scala file is part of a scala 3 project, and second the automatic indention and reformatting needs to be turned off.

A quick, and somewhat hacky, way to detect scala 3 projects is to check the build.sbt file for the version, using projectile to find the root of the project and searching for a scalaVersion to start with 3.

(defun is-scala3-project ()
  "Check if the current project is using scala3.

Loads the build.sbt file for the project and serach for the scalaVersion."
  (projectile-with-default-dir (projectile-project-root)
    (when (file-exists-p "build.sbt")
      (with-temp-buffer
        (insert-file-contents "build.sbt")
        (search-forward "scalaVersion := \"3" nil t)))))

Now using emacs advice-add the function adding the hooks to indent and format can be modified to no longer no longer insert the hooks for automatic formatting

(defun with-disable-for-scala3 (orig-scala-mode-map:add-self-insert-hooks &rest arguments)
    "When using scala3 skip adding indention hooks."
    (unless (is-scala3-project)
      (apply orig-scala-mode-map:add-self-insert-hooks arguments)))

(advice-add #'scala-mode-map:add-self-insert-hooks :around #'with-disable-for-scala3)

To still have some indention but not use the scala syntax specific one, the indent-line-function needs to be replaced with one purely using the previous line for reference ignoring syntax.

(defun disable-scala-indent ()
  "In scala 3 indent line does not work as expected due to whitespace grammar."
  (when (is-scala3-project)
    (setq indent-line-function 'indent-relative-maybe)))

(add-hook 'scala-mode-hook #'disable-scala-indent)

While it is more convenient to have indention supported, while the issue in emacs-scala-mode is being worked on this provides a quick fix to still have scala 3 code formatted. Also with metals and lsp supporting scala 3, the buffer can be reformatted in case formatting is an issue.

Attaching Metals LSP debugger to existing process in Emacs

Metals supports the DAP Protocol which allows to debug Scala code in Emacs. By default all the setup is done to use code lenses to start an instance of the program to debug it, set breakpoints inspect variables, all the good stuff to expect from a debugger. It is not as obvious how to debug a remote Scala process however as this requires a bit of setup.

Setting up the Scala process

Scala, as running on the JVM, allows to be run with the well know flags to be debugged namely

-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005

When using sbt this can be simplified to running as

sbt -jvm-debug 5005

which launches an sbt shell exposing a debug port 5005 on localhost. It is good practice to always use localhost and if truly remote to use SSH port forwarding, as the debugger has no authentication or any security in place.

Attaching the debugger

In dap-mode in Emacs uses debug-templates to determine what to run, in the case of scala and metals we need to provide the connection information as well as the lsp-project to use as the source for the code to set breakpoints and in general browse the code. The project name to use can normally be found in the build.sbt file this means for a project like the one associated with the FP Tower Foundations Course the setup looks like 

(dap-register-debug-template
  "Scala Attach Foundations (localhost:5005)"
  '(:type "scala"
    :request "attach"
    :name "Scala Attach Foundations (localhost:5005)"
    :hostName "localhost"
    :port 5005
    :targets [(:uri "file:///Users/pfehre/source/foundations?id=foundation")]))

important is the targets uri in this case as the default template for scala attach sets the project name to root resulting in the invalid target file:///Users/pfehre/source/foundations?id=root. With this setup and evaluated the commad dap-debug will now contain a Scala Attach Foundations (localhost:5005) target to use to attach to the process.

As this is project specific setup it makes sense to add it to the .dir-locals.el file so it automatically gets registered when visiting the project folder.

((nil . ((eval . (dap-register-debug-template
                  "Scala Attach Foundations (localhost:5005)"
                  '(:type "scala"
                    :request "attach"
                    :name "Scala Attach Foundations (localhost:5005)"
                    :hostName "localhost"
                    :port 5005
                    :targets [(:uri "file:///Users/pfehre/source/foundations?id=foundation")]))))))

would make this work.

Opening Sourcegraph from Emacs

Sourcegraph is super useful when browsing through code and dependencies, there is currently no plugin for emacs, but it is pretty easy to configure git-link to do the trick as @sqs pointed out to me on Twitter.

This is all the configuration to make git-link work as expected, and it should be pretty generic.

view raw git-link-sourcegraph.el hosted with ❤ by GitHub

Bloop integration in Emacs

Some quick and hacky integration of Scala bloop with Emacs, using ammonite as the console and projectile for compilation and testing.

;;; bloop --- bloop minor mode
;; Author: Philipp Fehre <philipp@fehre.co.uk>
;; Keywords: scala, bloop, tools, convenience
;;; Commentary:
;; Helpers to integrate better with bloop, inspired by emacs-bloop
;; https://github.com/tues/emacs-bloop/blob/master/bloop.el
;; C-c M-j jack-in a bloop project running a new Ammonite REPL buffer
;; C-c M-z switch to an active Ammonite REPL
;; C-c b c Compile a bloop project backed by projectile-compile-project
;; C-c b t Test a bloop project backed by projectile-test-project
;; C-c b r Run a bloop project
;; Changelog:
;; - 25/8/2020 - Added run command mapping bloop-run (C-c b r)
;; - 1/5/2020 - initial working version
;;; Code:
(require 'projectile)
(require 'scala-mode)
(require 'ammonite-term-repl)
(require 's)
(defgroup bloop nil
"Bloop integration for emacs"
:group 'tools
:group 'convenience
:link '(url-link :tag "Gist" "https://gist.github.com/sideshowcoder/a9b2ceaca38cdf0ea95f29bf0130b171&quot;))
(defcustom bloop-program-name "bloop"
"Program used to run bloop commands, default to whatever is in the path."
:type 'string
:group 'bloop)
(defcustom bloop-reporter "scalac"
"Either bloop or scalac.
The main difference is that bloop shows errors in reverse order.
Emacs generally assumes the first error in the output is the most
relevant so the scalac reporter will most likely be preferred.
This is used for test and compile."
:type 'string
:group 'bloop)
(defun bloop--command (&rest args)
"Build a bloop command for ARGS."
(s-join " " (cons bloop-program-name args)))
(defun bloop--available-projects ()
"Get a list of currently available projects from bloop."
(projectile-with-default-dir (projectile-project-root)
(let ((projects-string (shell-command-to-string (bloop--command "projects"))))
(split-string projects-string))))
(defun bloop-switch-to-ammonite ()
"Switch to the running Ammonite REPL."
(interactive)
(if-let ((ammonite-buffer (get-buffer ammonite-term-repl-buffer-name)))
(switch-to-buffer ammonite-buffer)
(message "Ammonite is not running try C-c M-j to start an Ammonite REPL for bloop.")))
(defun bloop-run-ammonite (project)
"Run Ammonite for a bloop PROJECT."
(interactive (list (completing-read "Run Ammonite REPL for project: " (bloop--available-projects))))
(projectile-with-default-dir (projectile-project-root)
(let ((ammonite-term-repl-program bloop-program-name)
(ammonite-term-repl-program-args (list "console" project)))
(run-ammonite))))
(defun bloop-compile (project)
"Compile a bloop PROJECT."
(interactive (list (completing-read "Compile bloop project: " (bloop--available-projects))))
(let ((command (bloop--command "compile" project)))
(projectile--run-project-cmd command projectile-compilation-cmd-map
:show-prompt 't
:prompt-prefix "Compile command: "
:save-buffers t)))
(defun bloop-test (project)
"Test a bloop PROJECT."
(interactive (list (completing-read "Test bloop project: " (bloop--available-projects))))
(let ((test-command (bloop--command "test" "--reporter" bloop-reporter project)))
(projectile--run-project-cmd test-command projectile-test-cmd-map
:show-prompt 't
:prompt-prefix "Test command: "
:save-buffers t)))
(defun bloop-run (project)
"Run a bloop PROJECT."
(interactive (list (completing-read "Run bloop project: " (bloop--available-projects))))
(let ((run-command (bloop--command "run" project)))
(projectile--run-project-cmd run-command projectile-run-cmd-map
:show-prompt 't
:prompt-prefix "Run command: "
:save-buffers t)))
;;;###autoload
(define-minor-mode bloop-mode
"Bloop integration for emacs."
:lighter " bloop"
:keymap (let ((map (make-sparse-keymap)))
(define-key map (kbd "C-c M-j") #'bloop-run-ammonite)
(define-key map (kbd "C-c M-z") #'bloop-switch-to-ammonite)
(define-key map (kbd "C-c b c") #'bloop-compile)
(define-key map (kbd "C-c b t") #'bloop-test)
(define-key map (kbd "C-c b r") #'bloop-run)
map))
;;;###autoload
(add-hook 'scala-mode-hook 'bloop-mode)
(provide 'bloop)
;;; bloop.el ends here
view raw bloop.el hosted with ❤ by GitHub

Emacs Markdown list DWIM

Markdown has multiple types of lists, and in my mind checking items from those list means different things depending on the type.

- [ ] this item is open
- [x] this item is close
- this item is also open
- ~~this item is also close~~

For checklists I want the box to be checked when I toggle the item for, others I want the item to be crossed out via strike-through. All of those actions already exist in Emacs Markdown-Mode, but they are different depending on the list, I want Emacs to do what I mean no matter the list so I added this

(defun coder--markdown-toogle-list-item-dwim ()
    "Toogle the current list item depending on the type do the right thing.

1. When it is not a markdown list, ignore
2. When the list is a checklist indicated by [ ] check the checkbox
3. When the list is a normal list, strike-through the current item
4. When the item already has strike-through applied, un-strike it"
    (interactive)
    (save-match-data
      (save-excursion
        (when-let ((bounds (markdown-cur-list-item-bounds)))
          ;; move to the beginning of the item after the list marker
          (goto-char (cl-first bounds))
          (forward-char (cl-fourth bounds))

          (cond ((looking-at "\\[[\s-xX]\\]") (markdown-toggle-gfm-checkbox))
                ((thing-at-point-looking-at markdown-regex-strike-through) (markdown-insert-strike-through))
                (t (progn
                     (set-mark (point))
                     (end-of-line)
                     (activate-mark)
                     ;; remove trailing whitespace from line first, this
                     ;; otherwise breaks strikethrough rendering
                     (delete-trailing-whitespace)
                     (markdown-insert-strike-through))))))))

Happy markdown editing!

Lauching Emacsclient via Spotlight

When installing Emacs via 

$ brew cask install emacs

it automatically installs emacsclient, but this needs to be launched via the terminal. When running the Emacs.app from Applications it will launch a new instance of emacs every time. Most of the time when I want to quickly edit something I tend to prefer to launch emacsclient to create a new window or frame in the existing running instance. Since I launch almost all my applications via Spotlight the simple way to achieve this is to create a script with the .command and use this instead. 

$ cat ec.command 
#!/bin/bash

EMACSCLIENT=/usr/local/bin/emacsclient
$EMACSCLIENT -n -c -a '' $*

With this in place configure the script to be run via the terminal of your choice, in my case iTerm, by right clicking on the file and Get Info.

ec.command GetInfo

Thats it now Spotlight will execute the script when typing ec and hitting Enter on the top hit, assuming it is indexed.

My experience in getting my AWS Solutions Architect Professional certification

Disclaimer: While studying for the exam I was working Amazon as a Software Engineer, not part of AWS but Prime Video. Given that I largely didn’t have any special AWS resources or knowledge. I did however work work with AWS technologies for quite some time especially CloudFormation, Dynamo and EC2.

Here we go, a couple of weeks ago I finally got around to taking my AWS Certified Solutions Architect – Professional exam and therefor I am now certified.

giphy.gif

I decided to take the exam mostly because I was looking for a reason to invest the time to really get to know what services are available and get a good overview of AWS in general (at least at this given point in time since AWS changes very quickly after all). Given this background I have to say the certification is well worth it, it is a great motivation to get studying and is a good way to guide the learning towards the more relevant (less buzzwords more value) parts of AWS.

My most reliable resource has been A Cloud Guru, which provide a quite comprehensive course with exercises, which make studying less dry but also are actually valuable in getting to know AWS better. I don’t think they are required to pass the exam, but given my motivation of getting to know AWS better they most certainly were helpful! Besides A Cloud Guru, I mostly ended up reading white-papers, and watching reinvent videos. Personally I found the white-papers mostly useful to pickup on the terms used, I don’t think the contain much new knowledge outside of that. I think as with most AWS exams the main enemy is time, with each question allowing for on average 2 minutes. It comes down to knowing what you know well and plow through those question quickly to have time for the ones which are less clear and require more analysis. While taking the exam I flagged multiple questions I was unsure about, but didn’t have time to go back, it was a close one for me.

Overall would I take it again: Yes, Is it worth it? Depending on the motivation for me it was. Thanks to A Cloud Guru for putting together a worth while course.

Grep in sections of a file

Processing files which are obviously organized in sections, chunks whatever you want to call it, happens and trying to find elements in it in an AND relationship is my most common use case. Sadly grep does not seem to have a nice way of processing based on a separator, but purely goes by line, so the easy way around this is to join all the lines in a chunk and grep in the result. Think input like this

SECTION
foo=bar
xyz=111
baz=blob
SECTION
foo=1
bat=man
baz=3

Some AWK magic does the joining trick,

awk '/SECTION/ {printf "\n%s\n",$0;next} {printf "%s ",$0} END {print "\n"}' INPUT_FILE

And now

awk '/SECTION/ {printf "\n%s\n",$0;next} {printf "%s ",$0} END {print "\n"}' INPUT_FILE | \
grep 'foo=bar' | \
grep 'baz=blob'

Gives the matching section.