Implementing the "expand" API parameter in Rails

In our previous blog post, we motivated the existence and utility of an expand parameter on REST APIs. As a reminder, the expand parameter provides a reasonable tradeoff for API callers, sending minimal data while still allowing them to request more information as necessary.

At Pocketflows, we’re building embedded APIs to help VSaaS companies build high-quality marketing tools faster and better. Our API is built with the endlessly useful Ruby on Rails, and we’d like to share parts of our implementation! This walkthrough does use Ruby syntax and Rails idioms, but the concepts still generalize! We hope this will enable other companies to more easily incorporate the expand parameter into their APIs.

The Resource Class

At its core, an API serves response objects or resources to the API caller. In our codebase, we represent these as Resources

# app/models/resource.rb

class Resource < ApplicationRecord
  # note: necessary so that this doesn't try to read the `resources` SQL table
  self.abstract_class = true

  def json(expand = [])
    raise NotImplementedError
  end
end

Each subclass of Resource is expected to implement the json method which will optionally receive an array of expand parameters.

Taking an example from our own API, our Member resource (which represents a user belonging to a rewards program) would be defined as

# app/models/member.rb

class Member < Resource
  belongs_to :rewards_program, autosave: false, optional: true, validate: false
  belongs_to :user, autosave: false, optional: true, validate: false

  def json(expand = [])
    # parsing out the expand parameters that correspond to "user"
    # and "rewards_program"
    user_expand = expand.filter { |x| x.length > 0 && x[0] == :user }
    rewards_program_expand =
      expand.filter { |x| x.length > 0 && x[0] == :rewards_program }

    {
      id: public_id,
      user: # deciding to return the id or expand the user resource
        (
          if user_expand.present?
            user.json(user_expand.map { |x| x[1...] })
          else
            user.public_id
          end
        ),
      rewards_program: # deciding to return the id or expand the rewards program
        (
          if rewards_program_expand.present?
            rewards_program.json(rewards_program_expand.map { |x| x[1...] })
          else
            rewards_program.public_id
          end
        )
    }
  end
end

Here we see that we’re filtering the expand variable, ensuring that we’re pulling out keys that correspond to :user and :rewards_program. If we find any, we forward those along the next layer of serialization. This is pretty wordy! Also note that we’re not doing any sort of error checking. Before we unpack that further, let’s examine how we preprocess the expand parameter.

**Pre-processing **expand

Recall that when API callers provide the expand parameter in their HTTP request, they do so either via GET with URL params

https://api.pocketflows.com/?expand[]=member.user&expand[]=location

or POST with body params

{
  "expand": ["member.user", "location"]
}

Rails already parses these equivalently, leading to the params variable containing

params[:expand] == ["member.user", "location"]

So that the Resource class is not responsible for parsing and splitting these inputs, we can normalize these into nested arrays of symbols so the above becomes

expand == [[:member, :user], [:location]]

We’ll pass this normalized form to the .json method of each of our Resources.

We can accomplish this in Rails by defining a before_action in ApplicationController. Simultaneously, we can also add simple error-checking that prevents arbitrarily-deep expansion.

# app/controllers/application_controller.rb

class ApplicationController < ActionController::Base
  before_action :normalize_expand

  attr_accessor :expand

  def normalize_expand
    @expand = []

    if params[:expand].present?
      if params[:expand].is_a?(Array) &&
           params[:expand].all? { |x|
             x.is_a?(String) && x.split(".").length <= 3 # prevent deep expansion
           }
        @expand = params[:expand].map { |x| x.split(".") }.map(&:to_sym)
      else
        return head :bad_request
      end
    end
  end
end

Improving Our Implementation

In our first outline of the Member class above, there was a lot of repetitive code, and we also did not take care of error handling. We can handle both by defining some utilities on the base Resource class.

Our goal is to parse out the keys that are actually valid for expansion and raise errors if any invalid keys are present. We also want to return a hash that provides simple access to the parsed keys for forwarding. The following function does the trick:

# app/models/resource.rb

class Resource < ApplicationRecord
  # ... previous stuff

  class << self
    # allowing the class to define which keys are expandable
    attr_accessor :expandable_keys
  end

  # by default, nothing is expandable
  self.expandable_keys = [].freeze

  # defining a custom error for expansion
  class ExpandError < StandardError
  end

  def parse_expand!(expand)
    # if any of the expand parameters start w/ an unexpandable key or other
    # typo, raise an error
    if expand.any? { |x|
         x.length > 0 && !self.class.expandable_keys.include?(x[0])
       }
      raise ExpandError.new
    end

    # for each of the possible keys, return the forwardable expansion params
    # to the next layer. If no expansion was requested, pass `nil` to the
    # next layer. For an Event being passed ["member.user"], would
    # return { member: [[:user]] }
    self
      .class
      .expandable_keys
      .each_with_object({}) do |key, hash|
        result =
          expand.filter { |x| x.length > 0 && x[0] == key }.map { |x| x[1...] }
        hash[key] = result.present? ? result : nil
        hash
      end
  end
end

This utility can be used in the Member class as follows:

# app/models/member.rb

class Member < Resource
  # defining which keys can be expanded!
  self.expandable_keys = %i[user rewards_program].freeze

  belongs_to :rewards_program, autosave: false, optional: true, validate: false
  belongs_to :user, autosave: false, optional: true, validate: false

  def json(expand = [])
    # note this early return helps simplify ternaries in nested resources
    return public_id if expand.nil?

    # using our expand
    parsed_expand = parse_expand!(expand)

    {
      id: public_id,
      user: user.json(parsed_expand[:user]), # forwarding things along
      rewards_program: rewards_program.json(parsed_expand[:rewards_program]) # forwarding things along
    }
  end
end

Defining new Resources is now simplified to just defining self.expandable_keys and passing the parsed expand parameters to the relevant keys. There are still some opportunities to potentially simplify redundancy in this code — for example, we still manually have to call .json and index parsed_expand — but the tradeoffs and implementation are left as exercises to the reader .

Complete Implementation

The complete code for Resource, Member, and ApplicationController is reproduced below

# app/controllers/application_controller.rb

class ApplicationController < ActionController::Base
  before_action :normalize_expand

  attr_accessor :expand

  def normalize_expand
    @expand = []

    if params[:expand].present?
      if params[:expand].is_a?(Array) &&
           params[:expand].all? { |x|
             x.is_a?(String) && x.split(".").length <= 3 # prevent deep expansion
           }
        @expand = params[:expand].map { |x| x.split(".") }.map(&:to_sym)
      else
        return head :bad_request
      end
    end
  end
end

# app/models/resource.rb

class Resource < ApplicationRecord
  # note: necessary so that this doesn't try to read the `resources` SQL table
  self.abstract_class = true

  class << self
    # allowing the class to define which keys are expandable
    attr_accessor :expandable_keys
  end

  # by default, nothing is expandable
  self.expandable_keys = [].freeze

  # defining a custom error for expansion
  class ExpandError < StandardError
  end

  def parse_expand!(expand)
    # if any of the expand parameters start w/ an unexpandable key or other
    # typo, raise an error
    if expand.any? { |x|
         x.length > 0 && !self.class.expandable_keys.include?(x[0])
       }
      raise ExpandError.new
    end

    # for each of the possible keys, return the forwardable expansion params
    # to the next layer. If no expansion was requested, pass `nil` to the
    # next layer. For an Event being passed ["member.user"], would
    # return { member: [[:user]] }
    self
      .class
      .expandable_keys
      .each_with_object({}) do |key, hash|
        result =
          expand.filter { |x| x.length > 0 && x[0] == key }.map { |x| x[1...] }
        hash[key] = result.present? ? result : nil
        hash
      end
  end
end

# app/models/member.rb

class Member < Resource
  # defining which keys can be expanded!
  self.expandable_keys = %i[user rewards_program].freeze

  belongs_to :rewards_program, autosave: false, optional: true, validate: false
  belongs_to :user, autosave: false, optional: true, validate: false

  def json(expand = [])
    # note this early return helps simplify ternaries in nested resources
    return public_id if expand.nil?

    # using our expand
    parsed_expand = parse_expand!(expand)

    {
      id: public_id,
      user: user.json(parsed_expand[:user]), # forwarding things along
      rewards_program: rewards_program.json(parsed_expand[:rewards_program]) # forwarding things along
    }
  end
end

Conclusion

We hope this guided implementation of the expand parameter in Ruby on Rails is helpful! It’s our hope that more API providers start to offer this feature and improve developer ergonomics across the board.