CrystalMeet

This is the second library from Matz Told Me Why, after macroaggression. It won’t get a repository link, because the code doesn’t deserve one.

What it was¶

CrystalMeet was a Rust command-line tool. The name means Crystal meets Ruby, though CrystalMeth is the cooler reading. I wrote it to learn Clap, the argument parser most Rust CLIs start with. These days I use jdx’s usage instead.

It read Ruby and wrote Crystal. Crystal borrows Ruby’s syntax and infers static types on its own, so a lot of Ruby is almost Crystal already. CrystalMeet handled the almost with a pile of regular expressions.

It half worked. Plenty of scripts came out as working Crystal, in ugly code.

The runtime is the wall¶

The other half ran into Ruby’s runtime. A Ruby program can create classes while it runs, and a regex only ever sees the source.

closure_tree, a gem I co-maintain, shows it. When a model calls has_closure_tree, the gem builds a second model for the hierarchy table, on the fly:

def hierarchy_class_for_model
  parent_class = model_class.module_parent
  hierarchy_class = parent_class.const_set(short_hierarchy_class_name, Class.new(abstract_base_class))
  model_class_name = model_class.to_s
  hierarchy_class.class_eval do
    # ...
    belongs_to :ancestor, class_name: model_class_name
    belongs_to :descendant, class_name: model_class_name
    # ...
  end
  # ...
end

For a Tag model the class is TagHierarchy, a name the gem builds from a string. No source file defines it, and finding it means running the gem’s Ruby code. Crystal could build a name like that with a macro at compile time. A regex over Ruby source cannot. ActiveRecord goes further and defines each model’s attribute methods from the database schema, which no Ruby file contains.

People asked for method_missing. I said it could not be done, and they were upset that I would not promise it anyway. Ruby calls method_missing at run time, when a lookup fails, and hands it the missing name as a symbol. That name can come from send on a string the program read a second ago. Crystal has a method_missing macro, but it expands at compile time, once for each call written in the source, and its body is macro code that works on the call’s syntax. A Ruby method_missing body is ordinary code that works on a symbol. No regex turns one into the other.

A proper parser would not have saved CrystalMeet either. spinel, matz’s ahead-of-time Ruby compiler, reads Ruby through Prism, the parser CRuby ships, and its README says plainly: “No dynamic metaprogramming or eval.” I gave it the pattern in a few lines:

class Tag
end
name = "#{Tag}Hierarchy"
klass = Object.const_set(name, Class.new)
klass.class_eval do
  def depth = 1
end
p Object.const_get(name).new.depth

Ruby prints 1. spinel stops at the line that looks the class up by name:

const_get with a name known only at run time is not supported: constants are resolved at compile time (use a literal name, or a Hash from names to classes)

The same program with class TagHierarchy written out compiles and prints 1. One of the tests I added to spinel lives in test/reject/, where every file is a program spinel has to refuse. It adds a method to Class inside a method body, and that method only exists once the body runs.

spinel gives the same answer on method_missing that I gave. Every call site compiles to a direct C call, so a failed lookup has no hook to fall back to. A class that defines method_missing gets a compile-time warning, and the method runs only when something calls it by name.

The tests were the good part¶

What CrystalMeet did have was tests, about 200 scenarios. Two years later they are the only part worth keeping, because they record what Ruby does at its edges. spinel translates Ruby too, into C, and needs exactly that list.

Where they went¶

Since September 24 I have been porting those scenarios into spinel. As of September 29:

Matz merged every pull request of mine I checked.

The titles read like scenarios. One says “An empty **h passes no argument: a keyword list of only spreads rides a *rest or Proc#call only when it has entries.” Another says “A new local starts with every flag cleared, so the same program compiles to the same C every run.” More than half of those commits add or change a test. Most of the rest fold duplicated code in the compiler into shared helpers.

The regexes are gone and nobody will miss them.