Modularizing Start Emacs

24 Feb, 2025 by Graham Marlow in emacs

Some folks don't want their entire Emacs configuration to live in a single, thousand-line file. Instead, they break their config into separate modules that each describe a small slice of functionality. Here's how you can achieve this with Start Emacs.

Step one: load your custom lisp directory

Emacs searches for Emacs Lisp code in the Emacs load path. By default, Emacs only looks in two places:

• /path/to/emacs/<version>/lisp/, which contains the standard modules that ship with Emacs
• ~/.emacs.d/elpa/, which contains packages installed via package-install

Neither of these places are suitable for your custom lisp code.

I prefer to have my custom lisp code live within ~/.emacs.d/, since I version control my entire Emacs configuration as a single repository. Start Emacs adds ~/.emacs.d/lisp/ to the load path with this line in init.el (the Init File):

(add-to-list 'load-path (expand-file-name "lisp" user-emacs-directory))

Where user-emacs-directory points to ~/.emacs.d/, or wherever it may live on your machine.

The rest of this guide assumes your load path accepts ~/.emacs.d/lisp/, but feel free to swap out this path for your preferred location.

Step two: write your module

Next we'll create a module file that adds evil-mode with a few configurations and extensions.

Create the file evil-module.el in your ~/.emacs.d/lisp/ directory. Open it up in Emacs and use M-x auto-insert to fill a bunch of boilerplate Emacs Lisp content. You can either quickly RET through the prompts or fill them out. Note: to end the "Keywords" prompt you need to use M-RET instead to signal the end of a multiple-selection.

Your evil-module.el file should now look something like this:

;;; evil-module.el ---      -*- lexical-binding: t; -*-

;; Copyright (C) 2025

;; Author:
;; Keywords:

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;;

;;; Code:

(provide 'evil-module)
;;; evil-module.el ends here

Most of these comments aren't relevant for your custom lisp module but they're good to have in case you ever want to share your code as an Emacs Lisp package. The single line of Emacs Lisp code, (provide 'evil-module), is the most important part of the template. It denotes 'evil-module as a named feature, allowing us to import it into our Init File.

Since we're building an evil-mode module, I'll add my preferred Evil defaults to the file:

;;; Commentary:

;; Extensions for evil-mode

;;; Code:

(use-package evil
  :ensure t
  :init
  (setq evil-want-integration t)
  (setq evil-want-keybinding nil)
  :config
  (evil-mode))

(use-package evil-collection
  :ensure t
  :after evil
  :config
  (evil-collection-init))

(use-package evil-escape
  :ensure t
  :after evil
  :config
  (setq evil-escape-key-sequence "jj")
  (setq evil-escape-delay 0.2)
  ;; Prevent "jj" from escaping any mode other than insert-mode.
  (setq evil-escape-inhibit-functions
        (list (lambda () (not (evil-insert-state-p)))))
  (evil-escape-mode))

(provide 'evil-module)
;;; evil-module.el ends here

Step three: require your module

Back in our Init File, we need to signal for Emacs to load our new module automatically. After the spot where we amended the Emacs load path, go ahead and require 'evil-module:

;; init.el
;; ...
(add-to-list 'load-path (expand-file-name "lisp" user-emacs-directory))

(require 'evil-module)

Reboot Emacs and your module is ready to go!

Async IO in Emacs

16 Feb, 2025 by Graham Marlow in til

Stumbled on the emacs-aio library today and it's introduction post. What a great exploration into how async/await works under the hood! I'm not sure I totally grok the details, but I'm excited to dive more into Emacs generators and different concurrent programming techniques.

The article brings to mind Wiegley's async library, which is probably the more canonical library for handling async in Emacs. From a brief look at the README, async looks like it actually spawns independent processes, whereas emacs-aio is really just a construct for handling non-blocking I/O more conveniently.

Karthink on reddit comments on the usability of generators in Emacs:

I've written small-medium sized packages -- 400 to 2400 lines of elisp -- that use generators and emacs-aio (async/await library built on generator.el) for their async capabilities. I've regretted it each time: generators in their current form in elisp are obfuscated, opaque and not introspectable -- you can't debug/edebug generator calls. Backtraces are impossible to read because of the continuation-passing macro code. Their memory overhead is large compared to using simple callbacks. I'm not sure about the CPU overhead.

That said, the simplicity of emacs-aio promises is very appealing:

(defun aio-promise ()
  "Create a new promise object."
  (record 'aio-promise nil ()))

(defsubst aio-promise-p (object)
  (and (eq 'aio-promise (type-of object))
       (= 3 (length object))))

(defsubst aio-result (promise)
  (aref promise 1))

Pulling Puzzles from Lichess

03 Feb, 2025 by Graham Marlow in til

Lichess is an awesome website, made even more awesome by the fact that it is free and open source. Perhaps lesser known is that the entire Lichess puzzle database is available for free download under the Creative Commons CC0 license. Every puzzle that you normally find under lichess.org/training is available for your perusal.

This is a quick guide for pulling that CSV and seeding a SQLite database so you can do something cool with it. You will need zstd.

First, wget the file from Lichess.org open database and save it into a temporary directory. Run zstd to uncompress it into a CSV that we can read via Ruby.

wget https://database.lichess.org/lichess_db_puzzle.csv.zst -P tmp/
zstd -d tmp/lichess_db_puzzle.csv.zst

CSV pulled down and uncompressed, it's time to read it into the application. I'm using Ruby on Rails, so I generate a database model like so:

bin/rails g model Puzzle \
  puzzle_id:string fen:string moves:string rating:integer \
  rating_deviation:integer popularity:integer nb_plays:integer \
  themes:string game_url:string opening_tags:string

Which creates the following migration:

class CreatePuzzles < ActiveRecord::Migration
  def change
    create_table :puzzles do |t|
      t.string :puzzle_id
      t.string :fen
      t.string :moves
      t.integer :rating
      t.integer :rating_deviation
      t.integer :popularity
      t.integer :nb_plays
      t.string :themes
      t.string :game_url
      t.string :opening_tags

      t.timestamps
    end
  end
end

A separate seed script pulls items from the CSV and bulk-inserts them into SQLite. I have the following in my db/seeds.rb, with a few omitted additions that check whether or not the puzzles have already been migrated.

csv_path = Rails.root.join("tmp", "lichess_db_puzzle.csv")
raise "CSV not found" unless File.exist?(csv_path)

buffer = []
buffer_size = 500
flush = ->() do
  Puzzle.insert_all(buffer)
  buffer.clear
end

CSV.foreach(csv_path, headers: true) do |row|
  buffer << {
    puzzle_id: row["PuzzleId"],
    fen: row["FEN"],
    moves: row["Moves"],
    rating: row["Rating"],
    rating_deviation: row["RatingDeviation"],
    popularity: row["Popularity"],
    nb_plays: row["NbPlays"],
    themes: row["Themes"],
    game_url: row["GameUrl"],
    opening_tags: row["OpeningTags"]
  }

  if buffer.count >= buffer_size
    flush.()
  end
end

flush.()

And with that you have the entire Lichess puzzle database available at your fingertips. The whole process takes less than a minute.

Puzzle.where("rating < 1700").count
# => 3035233

Logseq Has Perfected Note Organization

01 Feb, 2025 by Graham Marlow

A little while ago Apple Notes left me with quite the scare. I booted up the app to jot down an idea and found my entire collection of notes erased. I re-synced iCloud, nothing. Just the blank welcome screen.

Luckily my notes were still backed up to iCloud, even though they weren't displaying in the app (I checked via the web interface). After 40 minutes of debugging and toggling a series of obtuse settings, my notes were back on my phone. Yet the burn remained.

Since then I've been looking at alternatives for my long-term document/note storage. Apple Notes was never meant to be a formal archive of my written work, it just came out that way due to laziness in moving my notes somewhere permanent. I investigated the usual suspects: Notion, Obsidian, Bear, Org mode, good ol' git and markdown. Nothing stuck. Then I found Logseq and was immediately smitten.

The truth is, I don't actually use Logseq. I use Obsidian. You see, Logseq is a outliner. Every piece of text is attached to some kind of bulleted list, whether you're writing a code sample or attaching an image. Bulleted lists are great for notes, but not so great for blog posts or longform writing. I need a tool that can easily handle standard markdown for this blog, for example.

But despite not actually using Logseq, I've structured my Obsidian identically to Logseq. The Logseq method of organization is just so good. Everything boils down to three folders:

• journal/: the place for daily notes.
• pages/: high-level concepts that link between other pages or entries from the journal.
• assets/: storage for images pasted from clipboard.

That's it! Just three folders, each containing a ton of flat files. All of my actual writing happens in journal pages, titled with the current day in YYYY-MM-DD format. I never need to think about file organization, nor do I struggle to find information.

Looking at a long list of YYYY-MM-DD files sounds difficult to navigate, but the key is that they're tagged with links to relevant pages (like [[disco-elysium]]) that attach the journal entry to a concept. When I want to view my notes on a concept, I navigate to the concept page (disco-elysium) and read through the linked mentions. I don't need to worry about placing a particular thought in a particular place because the link doesn't care.

I got hooked on this workflow because Logseq is incredible at linked mentions. Just take a look at this example page:

All of the linked mentions (journal entries containing the tag [[disco-elysium]]) are directly embedded into the concept page. Logseq will even embed images, code samples, to-do items, you name it. It works incredibly well.

The Obsidian equivalent isn't quite as nice, but it gets the job done. Obsidian mentions are briefer, lack context, and stripped of formatting:

The flip-side is that I don't need to write notes in an outline form and can more easily handle moving my Obsidian notes into plain markdown files for my blog.

If you're like me and you want to use Logseq-style features in Obsidian there are a few configuration settings that are worth knowing about:

• In your Core plugins/Daily notes settings, set the New file location to journal/ and turn on "Open daily note on startup".
• In Core plugins/Backlinks, toggle "Show backlinks at the bottom of notes".
• In Files and links, set the "Default location for new attachments" path to assets/.

These three settings changes will get you most of the way there. That said, before messing with those settings I encourage you to give Logseq a try. It's free and open source, it's built in Clojure, and it has an excellent community forum. Although I don't use it for my longform/personal writing, I use it at work where outlining fits my workflow better.

Paper Puzzle Remixes

11 Jan, 2025 by Graham Marlow in puzzles, games

The holidays are always a great time for puzzles. My parents still receive print newspapers, offering an ideal opportunity to catch up on crosswords. This year I also picked up NYT's Puzzle Mania, a treasure-trove of paper puzzle goodness. Just a few days ago my partner and I finished the whopping 50x50 crossword puzzle. That's over 1000 clues!

What struck me as especially interesting with Puzzle Mania were the paper remixes of the popular "-dles":^[1] Wordle, Spelling Bee, and Connections. Each remix tweaks the digital puzzle form so that it suits a printed medium, changing a few mechanics but keeping the puzzle evocative of its original design. Puzzmo did something similar with their Crossword Vol. 1, offering print versions of Really Bad Chess and Flipart.

In fact, when Puzzmo soft-launched they sent out beta invites via physical postcards to your address. Solve the puzzle on the postcard to unlock your way into the app.

The first and third pictured are remixes of Zach Gage games: Typeshift and Really Bad Chess. Typeshift is the more interesting of the two, since the digital version relies on a clever sliding interface to differentiate the game from a simple wordsearch. Adapting the game to print means the player can no longer find words by randomly moving the slider up and down.^[2] It also means lowering the number of possible words to simplify the search.

I think the popularity of "-dle" puzzle games, the kind of daily games that one finds on NYT and Puzzmo, have to do with their resemblance to newspaper puzzles. They're short and snackable, perfect while waiting for coffee to brew. They're also crunchy enough that the player makes observable improvements over a long period of time, often in the form of a solving streak.

However, despite that resemblance there's a design tension that arises when adapting a digital puzzle into a print puzzle. What kinds of mechanics are translatable and why? How do the designers behind games like Flipart approach print adaptation of their digital games?

Zach Gage (creator of Flipart) gives some insight into the process in the Crossword Vol. 1 collection:

When we first started thinking about what kinds of puzzles we could make in print, we felt like Flipart was one Puzzmo game that truly could not work on paper. It was friend and fellow game designer JW Nijman who suggested a grid with embedded shapes that players would have to draw corresponding shapes on top of. [...] I didn't want players to have to do shape rotation in their heads (this is tough for many people!), so I brought JW's idea to Jack...

I recommend playing through a game of Flipart to get a sense of the difficulties Zach alludes to in this quote. A game of Flipart only takes tens of seconds. It's borderline instinctual; the ocular faculties take control as shapes rotate to avoid overlapping.

In contrast, the print version of the game is slow and methodical. Rotation is removed in favor of drawing the shape as-is. The fundamental constraint is drawing the shape in the grid such that the drawn shape contains the square that originally depicted it. Shapes cannot rotate and drawings cannot overlap. Print Flipart is much more of a logic puzzle.^[3]

Both the digital and print forms of Flipart play to the strengths of their medium. The digital form takes advantage of the fact that the computer can trivially render shapes in different rotations, something that's incredibly difficult for the human mind (and tedious to draw). The print form remains evocative of the digital, but ditches rotation in favor of something easier to both conceptualize and draw.

Converting a digital puzzle to a print puzzle is an interesting exercise. What can we learn from the process? A few rules come to mind:

• Keep state simple. Unlike their digital counterparts, print puzzles cannot represent game state that often changes or changes in unintuitive ways (like rotations in Flipart). The best print puzzles have the player fill in the game state as they progress, e.g. letters in crosswords, numbers in sudoku, and shapes in print Flipart.

• Complicated rule evaluations are a better fit for digital puzzles. Chess puzzles often feel more like an academic exercise than a casual puzzle, as the player must not only think about their own optimal move, but also the optimal response from their opponent. A puzzle that requires multiple back-and-forth turns quickly balloons into an overwhelming number of possibilities.

• Rethink UI affordances. On the web, Typeshift uses a vertical slider to add extra flavor to the puzzle-solving experience. On paper, implementing a vertical slider is impossible. To compensate, the overall complexity of the puzzle is reduced.

• Grids make for great playgrounds. I don't think it's a coincidence that crosswords and sudokus are confined to a grid. The grid is satisfying to fill and clearly denotes progress. It also provides a natural place to store game state.

I also want to shout-out a fantastic game that released last year: LOK Digital. It's relevant to this whole conversation because it actually goes in the reverse direction, adapting a print puzzle into a digital form. Because the rules of LOK are heavily reliant on rules evaluation, I personally think the digital adaptation is the way to go. It makes the overall experience quite a bit more enjoyable.