Grant Custer

Feed  Index  Twitter

Writing
Latest posts
2020.07.30 Automadraw: release notes
2020.07.28 Bushido Blade 2: a design appreciation
2020.07.12 Swapping color schemes across all terminals and Vim with Pywal and Base16
2020.06.25 Vimlike
***
2020.07.30

Automadraw: release notes

Automadraw is a new experimental app I made for my Constraint Systems project. It lets you draw and evolve your drawing using cellular automata using two keyboard controlled cursors.

What is it for

I think there are two main uses for Automadraw:

  1. Get more familiar with the cellular automata (Conway’s Game of Life and Langton’s Ant) that it runs. You can quickly experiment with lots of different patterns.
  2. Draw something collaboratively with the automata. The interaction design aims to make working with the automata intuitive. These design techniques (two cursors, keyboard controls) could be applied to a wide range of creative apps.

Two cursors

I had originally planned to use just one cursor, and have it shift between draw mode and “act” (run automata) mode. As I experimented I found that usually for act mode I wanted to cover a large area and draw mode a smaller one. Having to resize when switching between modes ruined the flow, so I split the cursors up.

Splitting them up opened up some new possibilities. I realized I could set it up so that I could use each cursor’s actions (draw or act, respectively) regardless of which one was in focus. This set up a couple of interactions I really liked:

Sweeping: draw some lines then use a long, narrow act cursor to sweep over the lines, running Game of Life over each sweep step. This usually produces intricate symmetrical designs that really feel like they're evolving through each sweep.

Active environment: resize the act cursor over a large area, use the draw cursor and have it move in and out of the act area as the automata is run. The act area becomes an environment where different rules apply. It feels like a physics or chemistry simulation.

This set-up is uniquely suited to keyboard cursor controls, where each cursor’s position is fully visible and fully predictable (versus a touch interface where you would have to use multiple fingers and the fingers themselves would obscure your view of the changes taking place). I use Vim-like keyboard controls because I honestly prefer them. My suspicion is that they may enable modes of interaction other methods do not. I was happy to find an interaction that fit them so well. I’m looking forward to seeing how even more multiple cursors feel in future experiments.

Stamp is a different example of the possibilities of multiple cursors: two cursors across two canvases.

Keyboard events

Part of the reason the two cursor interaction is interesting is because of an accident of keyboard event handling. A lot of the Constraint System experiments let you hold down multiple keys. This is tricky to handle in Javascript for everything except modifier keys. The main issue is that if you’re holding down one key, and start holding an additional one, the new one will take over the keyDown event. The solution is to make a keymap object, store each key on keyDown and remove it on on keyUp. You then use the keymap object for the source of truth about what is pressed on each keyDown event.

This technique mostly just works, but there turns out to be an issue, arguably a bug, that makes things like the “sweep” technique I discussed above possible. If you are pressing one key, add another key, then let up on the second key, keyDown events stop firing. For Automadraw, this behavior enables this interaction:

  • Hold down ‘a’ to run automata, press a direction key to move the act cursor. When you let up on the direction key the automata will pause running… until you press a direction key again. Using this technique you can run “sweeps”, moving the act cursor across a set of pixels, automatically running the automata once each step.

This interaction was a happy accident, and I’m looking forward to thinking about how to expand and support it more in future experiments.

Limitations and future possiblities

I had been wanting to experiment with cellular automata and a drawing app for a long time. For this experiment, I needed to really scope things down in order to get started. I restricted the drawing app colors to 1-bit (on or off). This usefully limited the number of cellular automata I could use and the number of interactions I needed to support. I also made the app ‘pixels’ large, at 16 actual pixels. This makes drawing quick and the automata actions more legible, but also restricts the fidelity of the final image. Someday I would like to build a cellular automata app more focused on image editing, where you could evolve parts of an image at a higher fidelity. That would also involve using automata that use color information, there are some interesting examples of those in this CA Lab demo video.

Code

The code for Automadraw is avaliable on github.

Slowly recreating React

I built the early Constraint Systems experiments using React, but have moved off of it to vanilla Javascript for the most recent ones. I do find myself recreating a lot of the set-up of React. I’ve found out firsthand that a lot of the React boilerplate I questioned is in there to work around the constraints of Javascript itself. I may switch back to React sometime, but right now I’m still enjoying experimenting on my own. It is also true that a lot of the benefits of React don’t mesh well with HTML canvas, which is where most of the action for this app takes place.

ES6 modules

This was the first project where I used ES6 modules. It was nice to be able to organize the code into sections like keyboard and state. I’ll continue to use them and refine my organization going forward. Maybe someday I’ll have a true base starter kit I can reuse across projects.

Canvas compositing

One switch I’ve made that I’ve been very happy with, is moving from rendering multiple canvas DOM elements on top of eachother, to placing only one canvas on the dom and compositing the different layers (in this case: cursor, grid, art) on to the DOM layer for each render. My rendering code is a little knotty, but it still feels a lot cleaner than stacking the canvases in the DOM.

Permalink
2020.07.28

Bushido Blade 2: a design appreciation

Bushido Blade 2 was a Playstation game I played a lot in high school. It was a fighting game with swords, and its main hook was that instead of health bars, damage was based on where you struck your opponent. You could injure limbs or finish the an opponent with one strike if you hit the right spot.

Design-wise, Bushido Blade rethought the premise of a fighting game from first principles. I love what this approach allowed them to do in terms of immersion: during a fight, nothing is visible on the screen except the two characters.

This is what I want to do when I design something: communicate everything through the core action. Design things so well that you don’t need to bring in health bars and labels.

Bushido Blade 2, like most of the games I played, was well-reviewed but never really that popular. I remember it having a pretty good story-mode with fun voice acting.

Permalink
2020.07.12

Swapping color schemes across all terminals and Vim with Pywal and Base16

Switching between light and dark colorschemes in all terminals using a hotkey.

I recently got instant light and dark color scheme toggle working for all open terminals, including those running Vim. I used a combination of techniques from Pywal and Base16 shell, and learned some things about scripting in Linux and escape sequences along the way.

Pywal

Pywal is a package for switching color schemes system wide. Mostly it is known for generating those color schemes from images, but it also comes bundled with a bunch of predefined themes. I wanted to use it to switch between gruvbox light and dark themes.

Pywal can change the color schemes for all open terminals automatically. It can also switch colors for several other Linux applications.

How Pywal works

This is what the gruvbox dark theme looks like in Pywal’s colorschemes directory:

# Pywal gruvbox colorscheme
{
  "special": {
    "background": "#282828",
    "foreground": "#a89984",
    "cursor": "#ebdbb2"
  },
  "colors": {
    "color0": "#282828",
    "color1": "#cc241d",
    "color2": "#d79921",
    "color3": "#b58900",
    "color4": "#458588",
    "color5": "#b16286",
    "color6": "#689d6a",
    "color7": "#a89984",
    "color8": "#928374",
    "color9": "#cc241d",
    "color10": "#d79921",
    "color11": "#b58900",
    "color12": "#458588",
    "color13": "#b16286",
    "color14": "#689d6a",
    "color15": "#a89984"
  }
}

A JSON file declaring each color. OK, but how do those colors get communicated to the applications? The customization instructions mention ~/.cache/wal a lot, so let’s see what’s in there:

# ls ~/.cache/wal
colors                      colors-putty.reg         colors-tty.sh        colors.Xresources
colors.css                  colors-rofi-dark.rasi    colors-wal-dmenu.h   colors.yml
colors.hs                   colors-rofi-light.rasi   colors-wal-dwm.h     sequences
colors.json                 colors.scss              colors-wal-st.h      wal
colors-kitty.conf           colors.sh                colors-wal-tabbed.h
colors-konsole.colorscheme  colors-speedcrunch.json  colors-wal.vim
colors-oomox                colors-sway              colors-waybar.css

Ah! It’s using the JSON color schemes to generate application specific color scheme files. This is a great example of figuring out which level of abstraction to intervene at: Pywal defines a standard color scheme spec and uses application specific templates to generate files from it. If anyone wants to add a new color scheme or application template the procedure for doing so is clear and self-contained.

Live reload and escape sequences

To change color schemes in most applications, Pywal builds the color config file and sends a message to the application to reload. For terminals, it does something different. It uses ANSI escape codes, invisible character sequences that give a terminal color and formatting instructions, to instantly swap out the colors.

You can see how this works in Pywal’s sequences.py. The conversion from the JSON hex color to the terminal readable escape sequence is here:

# from pywal/sequences.py
def set_color(index, color):
    """Convert a hex color to a text color sequence."""
    if OS == "Darwin" and index < 20:
        return "\033]P%1x%s\033\\" % (index, color.strip("#"))

    return "\033]4;%s;%s\033\\" % (index, color)

Escape sequences, which I’ve only seen otherwise in terminal prompt customizations, are not easy to parse or write for a human, but as part of a script they’re a powerful way to achieve instant terminal color palette swaps. I don’t think anyone would design an API featuring anything like escape sequences today, but in this case they make for a much smoother experience than a “change config and reload” cycle.

Now let’s look at how the escape sequences get sent to the terminal:

# from pywal/sequences.py
def send(colors, cache_dir=CACHE_DIR, to_send=True, vte_fix=False):
    """Send colors to all open terminals."""
    if OS == "Darwin":
        tty_pattern = "/dev/ttys00[0-9]*"

    else:
        tty_pattern = "/dev/pts/[0-9]*"

    sequences = create_sequences(colors, vte_fix)

    # Writing to "/dev/pts/[0-9] lets you send data to open terminals.
    if to_send:
        for term in glob.glob(tty_pattern):
        util.save_file(sequences, term)

    util.save_file(sequences, os.path.join(cache_dir, "sequences"))
    logging.info("Set terminal colors.")

This shows the power of Unix’s “everything is a file” approach. The script locates the file for each open terminal and writes the sequences directly to it (same as you would write to a text file). And it just works.

Vim issues

Pywal worked beautifully for me except for Vim. It may not be an issue depending on how your Vim and terminal color schemes are configured, but in my case to get the proper color scheme I needed to not only swap the terminal colors but also toggle the background setting in Vim between light and dark. I eventually got this working using xdotool to trigger a toggle hotkey in Vim, but it was not nearly as clean a process as the main write directly to terminal Pywal approach. So I went hunting for other solutions.

Base16

Base16 is a standardized format for creating 16-color terminal color schemes. Those color schemes can then be combined with templates to produce color configurations for a wide range of applications. Base16 shell is a set of scripts that converts those color schemes into escape sequences to be applied to terminals.

The main draw for me for Base16 was that their Vim package lets you set a base Vim color scheme that works wonderfully with any Base16 terminal color scheme, no background setting change needed. (Pywal does also have a version of this, but I was much less impressed with the base Pywal Vim color scheme.)

Applying Base16 to all open terminals

Base16 shell, unlike Pywal, only applies the new color scheme to your current terminal. This set-up has its own interesting possibilities (different color schemes for terminals where you sshed; random color scheme for each new terminal) but I wanted the color scheme to be applied globally. So I frankensteined a bit of Pywal into the Base16 shell script:

# Modified Base16 shell script
...
terms=`ls /dev/pts/[0-9]*`
terms="${terms} $PWD/.cache/base16/sequences"
for term in $terms
do
  # 16 color space
  put_template 0  $color00
  ...
done
...

I converted the Pywal send function into Bash, and wrapped the part of the shell script that sent the escape sequences. I also set it to save the sequences to a cache, to be run for each new terminal. This got me the exact terminal and Vim color swap I wanted. I set up a toggle script and assigned a hotkey using my window mananger i3wm. If I want to swap color palettes on other applications, I can add the necessary steps into the toggle script. I like knowing exactly what the toggle script is doing, vs. Pywal’s “we’ll try and take care of everything we can”.

The final result.

I just modified the shell scripts for the specific gruvbox color schemes I wanted, but the cleaner way to do it would be to modify the shell template and regenerate them all. For now, I’m happy I got everything working and learned more about escape sequences and the structure of the Linux file system in the process.

Lessons learned

Part of why I’m exploring Linux and scripting is to get a feel for how software could be more customizeable. A few things were especially interesting to me here:

  1. Writing to all open terminals is a great example of the power of “everything is a file”. Being able to locate all the open terminals and send the escape sequences to them through the file system interface shifted my mental model about scripting possibilities. I usually think of applications and files as very separate, and this blurred that a bit. I’d seen people talk about the power of the file concept before but this is one of the first times its been useful for something I was trying to do. I will spend some time thinking about how the file system concept could be applied to the software I make.
  2. Escape sequences. I’m trying to think if you would ever want to include them (or a concept like them) in an application created from scratch. I don’t think so. They’re useful when you want to do formatting and the only interface you have with the program is that you can write text characters to it. The style is embedded in the text, but because the embedding is invisible it’s going to be pretty unpredictable if you try and move it between programs.
  3. The power of plain text and the being able to manipulate plain text. Lots of the config files for Linux applications are in a simple, plain text format. Coming from Javascript, I’m more used to intaking data as JSON and doing the manipulation in Javascript. In Linux you’re more likely to manipulate the text directly, and there’s a bunch of tools to help you do this. I’m sure that in some respects this leads to more formatting edge-case errors, but there’s also a beauty to the simplicity. You can see this in how Pywal handles changing color config for a lot of applications: generate a color config in the proper format, then just include that in the larger appplication configuration.
Permalink
2020.06.25

Vimlike

This thread, by Zach Gage, on how genre conventions serve as interaction shortcuts, got me thinking about how I use Vim conventions in my creative tools at Constraint Systems.

https://twitter.com/helvetica/status/1274450330726645762:

7/ A big part of making games involves working with genre literacy. In game design a key concept is the idea of weight: Every rule you add has a cognitive load on the player, and you must balance the weight of your rules against how meaningful they are to the play experience.

8/ An idea might be great, but if it makes the game unwieldy, ditch it. But genre-conventions are different – they’re weightless. They allow for an increased complexity and nuance in games, because they let designers include a huge number of rules without adding any weight.

Almost all of the experiments on Constraint Systems use Vim conventions: at least the hjkl characters for movement. One of the big reasons I started the experiments was my fascination with how I felt using Vim in the terminal. The combination of a strict character grid and keyboard controls provide a feeling of stability, and through that calm, that I don’t feel in other programs, or using a computer in general.

This was especially in contrast to how I’ve felt when making gestural interface, or ones that simulate physics. Building those often felt like piling on edge-case handler on top of edge-case handler. If you did it well you could make a pleasing user experience, as long as they stuck to the path you had prepared. If they wanted to go a different direction, or you wanted to take the program in a new direction, you had to deal with that unwieldy tower, either by rearchitecting it or by adding even more code for handling the new edge cases.

I wanted to strip things down, and see if I could start from a more stable foundations, and I turned to Vim conventions to do that. It was a natural choice because I was chasing that feeling from Vim. Choosing Vim also gave me the interaction bootstrapping effect that Zach is talking about. Rather than asking the user to start from interaction scratch, I had the Vim foundation. That’s not directly relevant for the majority of people, Vim is only used a subset of programmers, so it doesn’t solve everything, but it is a place to start.

Even for users not familiar with Vim conventions, I think there’s a benefit to starting the experiments there, rather than trying to introduce a new paradigm. Vim has proven itself to at least be useful to many people (and inspired a lot of loyalty). So there’s an implicit promise that even if this looks weird, you know it can be learned and at least some people have found it useful.

There’s a whole series of Vim-like programs, mostly terminal-based, that use similar key combinations. There’s also a number of browser extensions that let you use Vim keybindings in the browser. Tiling window managers (I use i3wm) also share a lot of conventions. Putting these all together, you can put together a system for daily use that is keyboard-focused and mostly Vim-based. I’ve started referring to the Constraint Systems experiments as “alternative” interfaces. Vimlike interfaces are arguably the longest running, most fully fleshed out alternative interface for computers. I want to add to and learn from that system, and keep it alive in the face of the conventions (often imported from mobile/touchscreen design) that are dominating today.

Permalink
***
About

Grant Custer is a designer-programmer interested in alternative interfaces.

You can see work and inspiration in progress on my Feed and my alternative interface experiments on Constraint Systems. I also design and build prototypes for Cloudera Fast Forward. I’m happy to talk on Twitter, email: grantcuster at gmail dot com, or Mastodon. You can see a full list of projects on my Index.