Learn about Ractors and build a mini sidekiq
Alexandre Ignjatovic
Posted on March 10, 2021
In this article, you'll learn more about Ractor, and how you can use them to build your own clone of sidekiq (a background processing framework for Ruby).
What is Ractor?
Ruby 3.0 introduced the Ractor class. This is Ruby's Actor-like concurrent abstraction, and its goal is to provide a parallel execution feature of Ruby without thread-safety concerns.
According to wikipedia:
The actor model in computer science is a mathematical model of concurrent computation that treats actor as the universal primitive of concurrent computation.
Actors are able to:
- Create more actors
- Receive messages
- Send messages
- Take local decisions
Be aware that the Ractor implementation is not stable yet, do NOT use them for production code. See the warning displayed when you use them if you still need to be convinced to not use it (yet) in production:
warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues.
Creating a ractor
Creating a ractor is as simple as using Ractor.new
:
ractor = Ractor.new { puts 'Hello Ractor!' }
Receiving a message
There are two ways to receive a message, depending if you have a reference on the ractor sending it.
Use Ractor.receive
if you don't know who is sending the message:
message = Ractor.receive
And use Ractor#take
if you have a reference on the ractor sending the message:
message = ractor.take
As those method calls are blocking until you receive a message, avoid expecting a message from a ractor that will never send one, or your program is going to be stuck forever.
Also, please be aware that the objects you are sending must be shareable.
Sending a message
If you know your recipient:
ractor.send(message)
# or
ractor << message # `<<` is an alias to `send`
And if you don't:
Ractor.yield(message)
Ractor.yield
is blocking as well until some ractor receive your message.
Taking local decision
You can do pretty much anything you want in the block given to Ractor.new
. Unless accessing shared objects. An exception will be raised if you try to use a variable defined outside of your block. In order to share a variable between several ractors, you can, for example, use the Ractor::TVar gem.
Another interesting method I'm going to use later on is the Ractor.select(*actors)
method. It takes several ractors as an input, and returns the first ractor to send something, and its output:
slow_ractor = Ractor.new { sleep 2; Ractor.yield(:too_late) }
fast_ractor = Ractor.new { Ractor.yield(:fast) }
ractor, output = Ractor.select(slow_ractor, fast_ractor)
# output == :fast && ractor == fast_ractor
Let's build a mini sidekiq!
Please be aware that this crude POC will only allow you to use a pool of 10 ractors to achieve parallel execution of jobs. It won't handle error flow control, statistics, queueing and all the other features that make sidekiq a super useful project.
We'll build a simple design with:
- A
WorkerPool
, taking care of our pool of ractors - A
Job
base class, that all our dedicated jobs will inherit from.
The goal being to allow people to easily implement their own jobs without having to handle all the pool logic.
WorkerPool
class WorkerPool
attr_reader :ractors
def initialize
@ractors = 10.times.map { spawn_worker }
end
def spawn_worker
Ractor.new do
Ractor.yield(:ready)
loop { Ractor.yield Job.run(Ractor.receive) }
end
end
def self.run(parameters)
ractor, _ignored_result =
Ractor.select(*(@instance ||= new).ractors)
ractor << parameters
end
end
There is a trick here. When using Ractor.yield(:ready)
, we are just making sure that the ractors of the pool have something to send for the initial Ractor.select
to work (remember, it's blocking).
Job base class
class Job
def self.process(*args)
WorkerPool.run({ class: self, args: args })
end
def self.run(hash)
case hash
in { class: klass, args: args }
klass.new.process(*args)
end
end
end
Be aware that anything you will provide as argument must be shareable.
Implementing a specific job
Let's say that we'd like to create an asynchronous job that prints something:
class PrintJob < Job
def process(message)
puts message
end
end
Using it asynchronously is now as simple as:
PrintJob.process('Hello World!')
Conclusion
Ractor introduces a new and interesting model for parallel execution in Ruby.
If you found the ractor topic interesting, I suggest you check out these interesting resources:
- https://github.com/ruby/ruby/blob/master/doc/ractor.md
- https://docs.ruby-lang.org/en/3.0.0/Ractor.html
And if you liked this post, check out our awesome weekly tech newsletter. We are regularly sharing top ruby & javascript content!
Photo by Mark Thompson
Posted on March 10, 2021
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
November 20, 2024