Weaving and seaming mocks, by Vladimir Dementyev
Abstract
To mock or not mock is an important question, but let's leave it apart and admit that we, Rubyists, use mocks in our tests.
Mocking is a powerful technique, but even when used responsibly, it could lead to false positives in our tests (thus, bugs leaking to production): fake objects could diverge from their real counterparts.
In this talk, I'd like to discuss various approaches to keeping mocks in line with the actual implementation and present a brand new idea based on mock fixtures and contracts.
Details
This talk is an attempt to sway the ever-hot discussion on whether to use mocks or not in a different direction: how to mock with confidence?
I see this talk structured as follows.
Intro (5-7mins). A bit of theory, terminology, and examples (to ensure everyone is on the same page). A false positive demonstration (a real-life one, which caused a downtime 🙃).
A perfect (but theoretical) solution (5-7mins). Sociable and solitary tests, their test graphs, and what the problem under consideration has in common with minimum spanning trees. (For me, an ability to add some Math to a Ruby presentation is like adding bacon to a PB&J sandwich—a perfect addition)
Existing solutions overview (~10min). We're taking a closer look at RSpec verified doubles and their weak parts (they only verify syntax, not constraints). Then, we can talk a bit about Ruby types (RBS or Sorbet) and how they could increase the mocking confidence. Finally, I want to share some gems providing contract-based mocking capacities (e.g.,
compact).Introducing yet another solution (~10min). Here I'd like to share my attempt to turn the theoretical solution into reality. The keywords: YAML fixtures for mocks/stubs, automatic verification via Ruby metaprogramming tools.
Conclusion (2-3min). Cycling back to the importance of understanding the trade-offs of mocking and a discussion on non-runtime/non-code solutions (like linters and core review rules).
This talk is for all Rubyists writing tests, i.e., for all Rubyists. It could help newcomers better understand the potential pitfalls of using the mockist style and brings some new ideas for seasoned developers.
Pitch
Like many other Rubyists, I wrote about twice more test code than production code. And I've seen dozens of test suites of different code styles, but all of them used at least a bit of mocking.
Mocking in Ruby is simple; major testing frameworks provide outstanding support for "fake it 'till you make it (green)". So, we tend to use mocking for good; but sometimes it could lead to a situation when your test suite is all green, but the production server is down—some of your fake objects went out of sync, and no real test has covered you. That happened to almost every project I worked on.
Years ago, I started researching this problem, analyzing the existing solutions and their limitation, searching for a perfect one. I can't say I found one, but I've definitely reached the point when I'm ready to share my finding with the community—that's why I'm trying to do this by proposing this talk.