Lang Jam 1 Retrospective

A few weeks ago, Rizato and I participated in the first Lang Jam. The goal of the jam was to create a prototype language in 48 hours according to a theme, for the first jam the theme was “first-class comments”. Overall this was a great experience, and it was cool to produce a language that, while maybe not real-world useful, was a fun take on a very interesting theme. We were also fortunate enough to be featured in the winners video, along with several other amazing entries!

Our Entry

Our entry was a language called rundown. Our take on “first-class comments” borrows from literate programming, where a rundown program is meant to be primarily human-readable start to end, with code snippets embedded between text that define the logic of the program.

The original concept was a language for creating choose your own adventure-style interactive fiction by interleaving code between textual sections inside a markdown file. Basic arithmetic and conditional logic was implemented, along with a goto statement for jumping around between sections defined by headers in the markdown. When run, a rundown program starts at the beginning of the file, outputting markdown and evaluating rundown code sections, which can jump to any other section in the file.

Here’s an example run of the rundown README:

What went well

• We were able to finish while accomplishing basically everything we wanted in our language (minus some stretch features). Due to timezones and day-jobs, the vast majority of implementation happened on Saturday, with Friday being mostly brainstorming and Sunday morning being cleanup.
• We were able to demonstrate the language with several example programs, including one longer interactive fiction example that Rizato was kind enough to take point on.
• One of the things we definitely wanted to do was have our README be a valid rundown program, which I think is a neat tour of the language and helped us find some bugs.
• Using Rust to implement rundown ended up being a great choice, in particular the excellent library ecosystem made getting the markdown and parsing portions working a breeze, and language features such as pattern matching made the interpreter implementation very clean. Rust was a very popular choice in general for this jam, based on some statistics calculated on the Discord, 28/89 (31%) of submissions also used Rust.

What could have gone better

• We used the pest.rs library for the non-markdown parsing portion of our entry, overall this was a good choice with good getting started documentation. However, we ran in to a few issues:

  • The examples from the book were excellent resources, however it would have been great if some more complex examples were available. The “Calculator” and “Awk clone” examples are incomplete, and would have presumably been very helpful in creating the grammar for a programming language.
  • The online editor was very useful for debugging grammars, however several times it crashed/hung and we ended up losing edits that we did solely in the web editor. In a jam situation (time-constrained) this was a bit frustrating, but ultimately solveable by saving often and testing locally vs. in the editor.
  • Some issues in the grammar were hard to debug, and it wasn’t very intuitive at times what the grammar was doing or why certain constructs weren’t valid. This is a problem with developing grammars in general though in my experience, so hard to fault pest on this.
  • Figuring out operator precedence was hard, so we basically punted on it. Pest documentation indicates that a PrecClimber should be used for this, but there isn’t a good example of how to actually do this that we could find.

  Next time, we might experiment with another parsing library to gain some more experience / be able to compare and contrast.

• Rust’s excellent library ecosystem made the parsing and rendering of markdown pretty painless, and it was one of the first things we implemented. However, our final implementation ended up using two different markdown parsers, one for our internal parsing, and one included by the library that we used to pretty-print markdown strings to the terminal. In a non-jam scenario, it would have made sense to land on one implementation of markdown parsing, to ensure we don’t encounter weird bugs where the two different libraries disagreed.

• In an ideal world, with a team of two we would have been able to design out the language and agree on an intermediate representation so we could simultaneously work on the parser and the interpreter. This didn’t really happen, probably because a lot of the details of the language we created through discovery and were implemented ad-hoc rather than designed up-front. It also took a team effort to debug some of the parsing issues (which was good!), whereas the interpreter was comparatively straightforward.

Closing thoughts

This was a really fun weekend project that encouraged creativity due to the theme and stretched some muscles that aren’t often exercised (how often day-to-day does one implement a new language?) Thanks a lot to JT for all the work setting this up!

Definitely looking forward to the next one, if you are interested check out the repo at https://github.com/langjam/langjam for more details.