Between monoliths and microservices, by Vladimir Dementyev
Abstract
Rails applications tend to turn into giant monoliths. Keeping the codebase maintainable usually requires architectural changes. Microservices? A distributed monolith is still a monolith. What about breaking the code into pieces and rebuild a monolithic puzzle out of them?
In Rails, we have the right tool for the job: engines. With engines, you can break your application into components—the same way as Rails combines all its parts, which are engines, too.
This path is full of snares and pitfalls. Let me be your guide in this journey and share the story of a monolith engine-ification.
Details
Intro (5min)
Rails is well-known for its monolith-first design. Some monoliths are successful (you know, the one that starts with the letter "B"), others suffer from maintainability problems. From my experience, the majority of medium-to-large Rails applications is from the second group (here I want to demonstrate a few examples of the particular problems).
A quick overview of how younger frameworks (Hanami, Elixir Phoenix) and giant companies (Shopify) solve this problem (spoiler: they encourage/use component-based architecture).
How can we do something like this in Rails? Here come Rails Engines!
Engines overview (5-7min)
This part answers the question "What is an engine?" and "How to use engines?" using the example from OSS (likely, Devise and Action Cable or Active Storage).
The purpose of this part is to make sure that everyone is on the same page, and the word "engine" is no longer abracadabra.
That could be the end of the talk if Rails had 1) A complete guide on how to build applications from engines; 2) Rails and popular libraries/tools had proper engines support. None of these two holds, so we can continue :)
The story (3-5min)
A bit about a particular project which was engine-ified by my team: what challenges did we have (from both technical and product side), what intermediate steps were made.
This part demonstrates the gradual approach to migration: first, separation by namespaces, then extracting the core, and only after that moving to engines.
Taming engines (15-18min)
As I already mentioned, building applications with engines turned out to be not so simple.
In order to make everything go smooth, and to improve maintainability and productivity, we had to solve a bunch of problems(mostly related to development tools):
- Sharing and keeping dependencies in sync (or why eval_gemfile is awesome!)
- Managing migrations and DB seeds
- Re-using test factories and utilities
- Testing engines themselves (also, running only affected engines' tests on CI)
- Separation of engine-root concerns (or what should go to engines and what should be implemented in the main app?)
- Extending entities (models, controllers, etc.)
- Communication between engines (yes, we need event sourcing!)
- … and more
In this part, I also would like to introduce the idea of shared local gems: plain Ruby gems used by the application (and stored in the same monorepo at the beginning and the private registry in the future). We usually have the following gems: common-rubocop, common-testing, common-graphql, etc.
The goal of this part is to demonstrate the potential caveats one may encounter when deciding to move to engines, and, of course, useful techniques to deal with them. In other words, everything I wish I knew before going this way :)
Outro (2-3min)
Engines helped us to improve our codebase:
- Code (and tests) isolation makes it harder to write high coupled code
- Faster tests and CI runs
- Ability to migrate (some) engines to services later
- Upgrading dependencies with ease (consider, for example, Rails upgrades: they could be done engine by engine)
- Ability to re-use code in other applications (that was, by the way, one of the reasons we chose this architecture—we had to build a semi-clone of an existing application).
It might seem from the talk that engines are not there yet (about half of the talk was devoted to the problems ). That's true unless you have a useful guide and a set of tools to make painless. And I am going to share it at the end of the talk!
This talk is for everyone who wants to learn about alternative architectural approaches in building Rails apps.
Pitch
The talk is based on the personal experience of using engines as building blocks for a large Rails application.
It turned out that Rails Engines are not so easy to use out-of-the-box: many popular gems (including Rails frameworks) are not engine-ready; gluing engines together is something we should take care of by ourselves.
While working on this project, we've built many tools and pushed a bunch of patches to the existing projects, now using engines is much easier than the day we started to apply this technique.
Along the way, we also cover some hidden gems of Rails (ActiveSupport hooks and notifications, custom generators) and other useful tools (e.g., event sourcing with RailsEventStore).
I want to show that Engines are ready to rock!