Photo by Paweł Czerwiński on Unsplash

In all the time that I've been building and maintaining Rails apps, I've somehow never noticed that ActiveRecord attributes backed by columns with default values are nil on create.

I'm going to demonstrate a number of ways to avoid this bit of confusion, but first let's start with an example.

Magic Links and Missing Tokens

This is a real example from a recent Rails side project. I added a MagicLink model and magic_links table for implementing magic sign in links for authentication. A magic link is a short-lived link that gets emailed to a user which when clicked will sign the user into the app.

Here is the schema description for the magic_links table in PostgreSQL.

\d magic_links
                                        Table "public.magic_links"
   Column    |              Type              | Collation | Nullable |              Default
-------------+--------------------------------+-----------+----------+------------------------------------
 id          | uuid                           |           | not null | gen_random_uuid()
 user_id     | uuid                           |           |          |
 token       | uuid                           |           | not null | gen_random_uuid()
 expires_at  | timestamp without time zone    |           | not null |
 created_at  | timestamp(6) without time zone |           | not null |
 updated_at  | timestamp(6) without time zone |           | not null |

If you scroll to the right on that code block, you'll see that both id and token have default values of gen_random_uuid(). This is a function in Postgres that can produce a random (v4) UUID -- as a random, unique identifier it makes a great id and a great magic sign in token.

Here is the accompanying ActiveRecord model, though not a lot to see.

class MagicLink < ApplicationRecord
  belongs_to :user
end

And lastly, here is what I get when I create a new MagicLink record.

> magic_link = MagicLink.create(expires_at: 5.minutes.from_now, user: User.last)
  User Load (0.3ms)  SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1  [["LIMIT", 1]]
  TRANSACTION (0.1ms)  BEGIN
  MagicLink Create (0.4ms)  INSERT INTO "magic_links" ("user_id", "expires_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id"  [["user_id", "f683477e-c284-4677-8804-3852fa82b799"], ["expires_at", "2021-01-19 00:54:01.490924"], ["created_at", "2021-01-19 00:49:01.492309"], ["updated_at", "2021-01-19 00:49:01.492309"]]
  TRANSACTION (0.2ms)  COMMIT
=> #<MagicLink:0x00007fcd0691eff8
 id: "9fe2c468-fa11-4df3-9683-d11eb45aff06",
 user_id: "f683477e-c284-4677-8804-3852fa82b799",
 token: nil,
 expires_at: Tue, 19 Jan 2021 00:54:01.490924000 UTC +00:00,
 created_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
 updated_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
 used_at: nil>

The #create call inserts a new record into the magic_links table and produces an in-memory representation of that record which is an instance of MagicLink. There are two things I want to note about the in-memory (ActiveRecord) object.

1. The id has been set to "9fe2c468-fa11-4df3-9683-d11eb45aff06". Notice in the insert statement right about it that no value was sent to the Postgres server for id. Because of that, Postgres uses the default value. It invokes gen_random_uuid() to produce the record's id.

2. The token value appears to be nil. No value was passed for token as part of the insert statement. Because of that, I can assure you that Postgres invoked gen_random_uuid() for this field as well.

So, why is token showing up as nil in that MagicLink object?

To answer that, let's look a little closer at the insert statement that was produced by the #create call.

INSERT INTO "magic_links"
  ("user_id", "expires_at", "created_at", "updated_at")
  VALUES ($1, $2, $3, $4)
  RETURNING "id"
  [...]

That statement has a returning clause which specifies exactly "id" and nothing else. The result of ActiveRecord running that query will be:

1. Inserting a new record into magic_links with the provided values and any necessary defaults.
2. Returning just the newly produced id value for that record and nothing else.

When ActiveRecord gets back the response from the Postgres server for that insert, it will take the new information (just the id) and combine that with the info already embedded in the MagicLink object. The token value was not returned by the insert statement, so ActiveRecord doesn't know what it is. That's why token is nil.

So, what can we do about this?

A Variety of Solutions

There are several ways of getting at these default values. They all have tradeoffs, so you'll want to weigh those against your particular scenario.

1. Reload

Perhaps the first and most straightforward solution is to reload the record.

> magic_link.reload
  MagicLink Load (0.4ms)  SELECT "magic_links".* FROM "magic_links" WHERE "magic_links"."id" = $1 LIMIT $2  [["id", "9fe2c468-fa11-4df3-9683-d11eb45aff06"], ["LIMIT", 1]]
=> #<MagicLink:0x00007fcd0691eff8
 id: "9fe2c468-fa11-4df3-9683-d11eb45aff06",
 user_id: "f683477e-c284-4677-8804-3852fa82b799",
 token: "b90427cb-ab54-4e08-97dd-f8e02aa2820c",
 expires_at: Tue, 19 Jan 2021 00:54:01.490924000 UTC +00:00,
 created_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
 updated_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
 used_at: nil>

This will make a (wasteful?) roundtrip to the database, in addition to the initial insert to fetch all the up-to-date version of this object, all defaults included.

2. Produce the Default Value in Rails-Land

Most default values can be produced on the Rails side of things at the time of the create call.

> magic_link = MagicLink.create(
    expires_at: 5.minutes.from_now,
    user: User.last,
    token: SecureRandom.uuid
  )
  User Load (0.3ms)  SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1  [["LIMIT", 1]]
  TRANSACTION (0.1ms)  BEGIN
  MagicLink Create (0.4ms)  INSERT INTO "magic_links" ("user_id", "token", "expires_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"  [["user_id", "f683477e-c284-4677-8804-3852fa82b799"], ["token", "33fc275e-4235-4c59-9e8c-69c5ba622a3c"], ["expires_at", "2021-01-19 01:16:09.026531"], ["created_at", "2021-01-19 01:11:09.029372"], ["updated_at", "2021-01-19 01:11:09.029372"]]
  TRANSACTION (0.3ms)  COMMIT
=> #<MagicLink:0x00007fcd069aeba8
 id: "43c4956c-87f0-40f3-933c-9a382057d464",
 user_id: "f683477e-c284-4677-8804-3852fa82b799",
 token: "33fc275e-4235-4c59-9e8c-69c5ba622a3c",
 expires_at: Tue, 19 Jan 2021 01:16:09.026531000 UTC +00:00,
 created_at: Tue, 19 Jan 2021 01:11:09.029372000 UTC +00:00,
 updated_at: Tue, 19 Jan 2021 01:11:09.029372000 UTC +00:00,
 used_at: nil>

This approach gets you back to a single round-trip with the database.

I find this solution irksome because I want to be able to rely on the database for things. Postgres provides many conveniences and affordances including the assurance of default values. I want to avoid duplicating that logic in my Rails code as much as possible.

That said, based on this particular magic link use case and my code maintenance preferences, this is my preferred solution.

3. Insert with overridden returning

ActiveRecord's #insert accepts an optional returning argument that allows you to override the default value of the returning clause. Setting it as returning: '*' adjust the insert SQL statement to return everything in the result, not just the id.

> result = MagicLink.insert(
  {
    expires_at: 5.minutes.from_now,
    user_id: User.last.id,
    created_at: Time.zone.now,
    updated_at: Time.zone.now
  }, returning: "*")
  User Load (0.3ms)  SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1  [["LIMIT", 1]]
  MagicLink Insert (0.8ms)  INSERT INTO "magic_links" ("expires_at","user_id","created_at","updated_at") VALUES ('2021-01-19 01:20:00.912387', 'f683477e-c284-4677-8804-3852fa82b799', '2021-01-19 01:15:00.913344', '2021-01-19 01:15:00.913351') ON CONFLICT  DO NOTHING RETURNING *
=> #<ActiveRecord::Result:0x00007fcd1722dbe8 ... >

> result.rows.first[result.columns.index("token")]
=> "92f64a62-8136-4243-9da9-f91c5786255f"

This is another single roundtrip solution. The drawbacks are:

1. The result of the call is an ActiveRecord::Result object instead of a MagicLink object.

2. The #insert call skips a bunch of the utilities of the #create call including validations and callbacks. Notice also that I had to manually specify the created_at and updated_at values.

4. Improved insert with overridden returning

The first drawback of the previous approach was that you get back an ActiveRecord::Result object instead of a MagicLink object. This can be solved by splatting and wrapping the whole thing in a MagicLink.new call.

> MagicLink.new(
  **MagicLink.insert(
    {
      expires_at: 5.minutes.from_now,
      user_id: User.last.id,
      created_at: Time.zone.now,
      updated_at: Time.zone.now
    }, returning: "*"
  ).first)
  User Load (0.3ms)  SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1  [["LIMIT", 1]]
  MagicLink Insert (0.9ms)  INSERT INTO "magic_links" ("expires_at","user_id","created_at","updated_at") VALUES ('2021-01-19 01:33:27.064213', 'f683477e-c284-4677-8804-3852fa82b799', '2021-01-19 01:28:27.065227', '2021-01-19 01:28:27.065234') ON CONFLICT  DO NOTHING RETURNING *
=> #<MagicLink:0x00007fcd06a25dc0
 id: "bb17ad85-7e19-4cbf-9821-21f9c631243b",
 user_id: "f683477e-c284-4677-8804-3852fa82b799",
 token: "4e99b551-60a2-4956-aba9-394ac2a1c9f0",
 expires_at: Tue, 19 Jan 2021 01:33:27.064213000 UTC +00:00,
 created_at: Tue, 19 Jan 2021 01:28:27.065227000 UTC +00:00,
 updated_at: Tue, 19 Jan 2021 01:28:27.065234000 UTC +00:00,
 used_at: nil>

This still suffers from missing out on callbacks and validations and having to specify timestamps.

5. Improved+ insert with overridden returning

At this point I feel like we are starting to jump the shark. This solution adds a custom #create_returning method that simulates a #create call that gets to include a returning override. It create a new object of the model's type with the given attributes, tries the #insert call from previous solutions if the attributes are valid, and then returns that in-memory representation.

class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true
  def self.create_returning(attrs = {}, returning: "*")
    record = new(**attrs)
    record.attributes = insert(attrs, returning: returning).first if record.valid?
    record
  end
end

This approach gains back the validation check, but still leaves us without most of our callbacks (though I'd happily discourage the use of AR callbacks). I struggle to see myself using this approach because it requires abandoning standard ActiveRecord methods. That is hard enough as an active maintainer, and presents a nasty surprise for anyone new to the codebase.

Conclusion

I'd like to see Rails add an optional returning argument to the #create method. Until then, I'll probably opt for solution 2 or 1. If 2 isn't feasible for some reason and the second roundtrip required by 1 is untenable, I'd probably opt for some version of 3 with the details buried in a domain object or service object.

If you enjoyed this post, join my newsletter to get a direct line on more stuff like this.

Huge shoutout to Dillon Hafer who had a long conversation with me that led to this post and was the inspiration behind many of these solutions.

References:

• Columns With Default Values Are Nil On Create
• Postgres UUID default value nil in created model instance
• Postgres default function values do not get loaded on create