Building a DSL in Ruby - Part 2

In the previous blog post, we implemented the first version of an example of a DSL library in ruby, a simple FactoryBot clone. In this second part, we’ll extend and improve the library so that we can also build associations between objects.

The final syntax of our DSL will look like this:

ConstructionGirl.register do
  structure :owner do
    name "Owner"
    age 10
    with :product do
      product_type "Type 2"
      name "Product 3"
      price 0
    end
  end

  structure :product do
    product_type "Package"
    name "Product 2"
    price 11.20
    owner
  end
end

ConstructionGirl.create(:product) # Create a product with an implicit owner!
ConstructionGirl.create(:owner) # Create a owner with a product!

In contrast to the previous implementation, we don’t create the records as soon as we call some method ( create_structure in the previous version), instead, we have to explicitly state what we want to be able to create later on, as well as their relationships.

I call this post “Part 2” but, technically, it is a complete re-write of our previous implementation. The idea, however, remains the same: creating a domain language powerful enough to express relationships between objects and their attributes. Ok, enough talking, show me the code!

The register method

The method register is the entry point of our library. It allows us to register specific structures that we can ,later on, create with the create method. This method should accept a block and run everything inside it in the context of itself. This context should have an implementation of the method structure . Remember the instance_eval from the previous post? We’ll use it now!

class ConstructionGirl
  def self.register(&block)
    DSL.run(&block)
  end
end

class DSL
  def self.run(&block)
    new.instance_eval(&block)
  end
end

I decided to leave the responsibility of handling the DSL syntax to a proper class DSL . Now, every method called inside the block passed to register will run in the context of a newDSL object.

The structure method

We need to implement a method called structure in the DSL class:

class DSL
  def self.run(&block)
    new.instance_eval(&block)
  end
  
  def structure(name, &block)
    raise "Structure #{name} already registered" if ConstructionGirl.registered? name
    @local_construction = Construction.new(name)
    @local_construction.instance_eval(&block) if block_given?
    ConstructionGirl.register_structure(@local_construction)
  end
end

Woah! That’s a whole lot of new code. Let’s go through the structure method line by line. As we can see in the desired syntax (in the top of the post), structure should accept a name, which defines the name of the structure, and a block. First, we check if we have already registered any structure with the name provided. If so, we raise an error message. This prevents the user from doing something like this:

ConstructionGirl.register do
  structure :owner { ... }
  structure :owner { ... }
end

This is important! This tells us that we need to keep track of which structures were already registered in memory.

We’ll see how this is achieved later on. The next important thing is the Construction class. This is the core class of our library. A Construction represents a thing that we register and that we may want to create later. In the syntax that we want, we can pass a block to a construction. Just like we did in the first implementation, the block that we pass to thestructure method should be executed in the context of an object that we’ll create to represent a structure : A Construction . It should respond to methods that correspond to the attributes of the object along with their values (static or dynamic, like before). Finally, we register the new structure. We’ll see later on how we do this final step too.

The construction object

class Construction
  attr_reader :supposed_table_name

  def initialize(supposed_table_name)
    @supposed_table_name = supposed_table_name
    raise "Table #{supposed_table_name} does not exist" unless table_exists?
   end
    
   private
    
   def table_exists?
     ActiveRecord::Base.connection.table_exists? @supposed_table_name.to_s.downcase.pluralize
   end
end

Just like before, we’re writing this library assuming that we want to create active record objects and for that, we use ActiveRecord for it (the ending of this post justifies this choice, so keep reading!).

There’s a difference between this version of the DSL and the previous one: In the first version, we would run the methods inside the passed block in the context of the Model class being created Owner , Product , etc. We won’t do that in this version. But, if we’re not doing that, how can we know that the object responds to the methods name , age , etc? Here’s the fun part: we don’t. We’ll let the user create the object as he pleases, with whatever relationships and properties, and we assume that the user knows that those properties / relationships will map to valid columns/relations in the underlying database.

This raises another, important thing: a Construction object will have to keep track of all the relationships and properties specified by the user. However, we don’t know apriori which properties the user will need, and of course, we can’t define methods for all possible properties that an object can have. So we have to, again, use metaprogramming!

All objects in Ruby implement a method call method_missing which is called whenever we call a method that an object doesn’t respond to:

class A
  def method_1
    puts "Hi!"
  end
end

A.new.method_1
# "Hi!"

A.new.method_2
# NoMethodError: undefined method `method_2' for #<A:0x000000017a93c8>

class A
  def method_missing(name, *args, &block)
    puts "Tried to call #{name} with #{args}"
    super
  end
end

A.new.method_2(1, 2)
# Tried to call method_2 with [1, 2]
# NoMethodError: undefined method `method_2' for #<A:0x000000034968a0>

In this example, we intercept the method method_missing to log a message when we try to call a method that doesn’t exist in the A class.

Nothing is stopping us from overriding that method and use it to our advantage here. Let’s see how we can leverage this powerful yet dangerous feature in our library:

class Construction
  attr_reader :supposed_table_name, :associations

  def initialize(supposed_table_name)
    @supposed_table_name = supposed_table_name
    raise "Table #{supposed_table_name} does not exist" unless table_exists?
    @attributes = []
  end

  def method_missing(name, *args, &block)
    add_attribute(name, *args, &block)
  end
  
  private

  def add_attribute(name, value = nil, &block)
    raise "Can not given value and block" if value && block_given?
    to_pass = value || block
    add_attribute_value(name, to_pass)
  end
  
  def add_attribute_value(name, value)
    @attributes << { name: name, value: value }
  end
end

Whenever we call a method in the context of a Construction , we assume that that method refers to a property of the object being created. There’s no special reason for this: it was just the way I chose to do it. We keep the attributes of the construction in memory in an array. Just like we did before, we accept either a static value for the property (for example, name "My Name") or a dynamic value (for example, name { "My Name Version #{Time.now.to_i}"}.

After all the code passed to the block of the new structure object is executed, we finally register the new structure in memory using the register_structure method:

class DSL
  # ...
  def structure(name, &block)
    raise "Structure #{name} already registered" if ConstructionGirl.registered? name
    @local_construction = Construction.new(name)
    @local_construction.instance_eval(&block) if block_given?
    ConstructionGirl.register_structure(@local_construction)
  end
end

To keep track of the structures registered, we use a simple array stored as a class variable of the ConstructionGirl class:


class ConstructionGirl
  @@constructions = []
  def self.registered_structures    
    @@constructions
  end

  def self.registered?(name)
    registered_structures.any? { |c| c.supposed_table_name === name }
  end

  def self.structure_by_name(name)
    registered_structures.find { |c| c.supposed_table_name === name }
  end

  def self.register(&block)
    DSL.run(&block)
  end

  def self.register_structure(structure)
    @@constructions << structure
  end
  
  # ...
end

Storing associations

The last thing that we need is a way to store associations. As we’ve seen in the syntax of our DSL, there are two ways to register associations for our objects: we can either call the method with in the context of a new structure:


structure :owner do
  # ...
  with :product do
    product_type "Type 2"
    name "Product 3"
    price 0
  end
end

or we can simply call the name of an already registered structure when creating a new structure (implicit association):

structure :product do
  product_type "Package"
  name "Product 2"
  price 11.20
  owner
end

The with method

To store associations with the with method, we need to add that method to the Construction class so that it can be recognized as a method inside the block passed to the structure method:

class Construction
  attr_reader :supposed_table_name, :attributes, :associations

  def initialize(supposed_table_name)
    # ...
    @associations = []
  end
  
  # ...
  def with(assoc_name, &block)
    raise "Must pass a block!" unless block_given?
    assoc = Construction.new(assoc_name)
    assoc.instance_eval(&block)
    @associations << assoc
  end
  # ...
end

The with method is straightforward: it requires a block and a name, creates a new construction with the name given and executes whatever code we passed inside the block in the context of the construction (just like we did before: creating properties and, of course, other associations) and storing the new object in the associations array.

Implicit association

Creating implicit associations requires us to change the add_attribute method in the Construction class. We’ll assume that if the user called a method but with no arguments or block, then he’s adding a new association (for example, when simply calling owner in the example above):


class Construction
  # ...
  def add_attribute(name, value = nil, &block)
    raise "Can not given value and block" if value && block_given?
    to_pass = value || block
    if to_pass.nil? && ConstructionGirl.registered?(name)
      @associations << ConstructionGirl.structure_by_name(name)
    else
      add_attribute_value(name, to_pass)
    end
  end
end

We check if the user didn’t pass any block or value. In that case, we’ll check if there’s already any structure registered in memory with the name given and if so, we add that structure to the associations array. Note that this assumes that the construction must be already registered, which is not the same as the with method since we can create a new structure on the fly with that approach.

The create method

Last step! Now we need to be able to take whatever is in memory (in the constructions class variable of the ConstructionGirl class) and actually create them. In this post I’ve been assuming that we’re using Rails, so creating records is easy, but the implementation of the create method may change depending on what you’re circumstance.

class ConstructionGirl
  # ...
  def self.create(name)
    construction = @@constructions.find { |c| c.supposed_table_name == name }
    raise "No structure found with name #{name}" unless construction
    construction.create
    end
end

If the construction doesn’t exist (it wasn’t registered), we throw an error, otherwise, we call the create method on the construction with the name passed as the argument:

class Construction
  # ...
  def create
    model = supposed_table_name.to_s.classify.constantize
    inst = model.new

    set_attributes(inst)
    inst.save
    set_associations(inst)

    inst.save unless inst.persisted?
    inst.reload
  end
end

First, we initialize the model. Then we set the attributes of the model using whatever we have stored in the attributes array:

def set_attributes(inst)
  @attributes.each do |attr|
    column = attr[:name]
    raise unless inst.attributes.keys.include? column.to_s
    value = attr[:value]
    if value.respond_to? :call
      inst[column] = value.call
    else
      inst[column] = value
    end
  end
end

There’s an important thing in this method: the difference between dynamic and static properties. Note that if the value stored in the attribute hash responds to call , then it means that it is a block (this is not 100% safe, as we can implement any object that implements a method call , but for this example, is enough) and so the value of that property will be the result calling that block, otherwise, we just store whatever the value is. Note that we’re using the Rails way to access properties of a model: model_instace['some_property'].

After setting the properties of the new instance, we need to save the record. This is important for the set_associations method:

def set_associations(inst)
  @associations.each do |assoc|
    relation_plural = assoc.supposed_table_name.to_s.pluralize
    relation_singular = assoc.supposed_table_name.to_s
    if inst.respond_to? relation_plural
      inst.send(relation_plural).create(**assoc.attributes_to_columns)
    elsif inst.respond_to? relation_singular
      inst.send("#{relation_singular}=", assoc.create)
    else
      raise "#{model} doesnt have the relation #{relation}!"
    end
  end
end

For each association, we check if it is either a plural association (like has_many products or a singular association (like belongs_to owner ). This is a naive implementation on top of Rails but, again, it’s just for exemplification here.

If it is a plural association, then we create another association for that relation with the new object (in this case, we’re calling create on an active record relation), otherwise, we just set the singular association. Note that for these two methods to work, the receiver record inst must be already persisted, and that’s why we called inst.save before.

The last two lines of the create method will try to save the object again and return the instance reloaded from the db (important if new associations were created).

Conclusion

In this two-part blog post we created a simple and naive clone of the FactoryBot library and, hopefully, along the way, we learned a couple of things:

  • What a DSL is
  • What instance_eval is
  • What method_missing is and how to leverage it to create a DSL
  • How to create a simple DSL using these two metaprogramming features of Ruby

This post came from my desire to learn a bit more about FactoryBot and the internals of it, and I hope you’ve learned something with this too!

Thanks for reading. 😃