Outside in Test Driven Development

In a talk I gave at The Big Elixir 2022 in New Orleans I made a comment about how you can use browser testing in order to practice outside in Test Driven Development (TDD) in Elixir. In my experience that's often an easier way to learn and apply test driven development as a more junior engineer, or when you're exploring a new domain.

After seeing the talk two early career engineers approached me and asked if I'd be willing to give a talk on the idea of outside in TDD, as they'd heard about TDD and wanted to give it a shot but were struggling in figuring out where to start. This is a common thing I hear, even among more senior engineers.

While this isn't a fully baked talk yet, this may be the start of one going forward.

If you're not senior it can be especially hard to practice TDD, because it's harder to immediately visualize or intuit the data structure and functions that you'll need to create, and what they ought to look like in the end state.

In fact, I've found this even blocks many senior folks from practicing TDD when they're exploring a new problem space.

I've found that outside in TDD alleviates much of this pain, and makes it easier for a less experienced engineer to discover what they need to create, guided by tests.

I'm going to start by giving a bit of a trivialized example, but we'll walk towards more involved examples and how the testing can guide the architecture.

Outside In vs Inside Out

Before we dive in, I want to give an example of what I mean by outside in, and inside out.

Inside Out Classic MVC example

Most TDD examples I see are what I call inside out.

They assume you can visualize the data structures that you'll need, what functions you'll need to create in order to create them, how those turn into controller actions, and how those controller actions get translated into views.

So if you're building a Trello clone, you would need to be able to visualize the following:

1. The task data structure and the fields that it needs
2. The functions that are needed to create tasks
3. The controller actions that implement the functions that create the tasks
4. The views and templates for the tasks

This would likely result in you creating a schema that looks something like this, with the appropriate migrations.

Board
  has_many    :stages
  
  id          :binary,    pkey: true
  name        :string,    required: true
  created_at  :datetime,  required: true
  updated_at  :datetime

Stage
  has_many    :tasks
  
  id          :binary,    pkey: true
  order       :integer,   required: true
  name        :string,    required: true
  created_at  :datetime,  required: true
  updated_at  :datetime

Task
  belongs_to  :stage,     required: true
  has_many    :assignees
  
  id          :binary,    pkey: true
  title       :string,    required: true
  description :string
  due_date    :datetime
  created_at  :datetime,  required: true
  updated_at  :datetime

Which would lead to the creation of tests hat looks something like:

task = Task.create(params)
assert task.errors.title == :required

As an experienced engineer it's easy enough to recall and intuit all of these. As an early career engineer however, this may not be as obvious because they haven't done this hundreds of times.

As you can also see, this isn't something where your tests are guiding your design. Rather it's your prior thoughts/intuition guiding it, with tests being put in post hoc in order to ensure future confidence. This is good, not great.

Outside In Classic MVC example

Outside in contrast, would start with the UI or API interactions. So in the Trello clone example, we'd start with the fact that we have columns representing stages, that we need to be able to move cards, representing tasks in the browser.

That immediately guides us to a testable user story that looks something like the following from the browser layer:

Browser
  .visit("/board/1")
  .click("New Task")
  .fill_in("title").with("Allow users to create new tasks")
  .click("Submit")

assert Task.count == 1

Browser
  .vist("/board/1")
  .drag("#card-1")
  .to("Column 2")
  
assert Task.where(stage == "Column 2") == 1

Now, obviously this is pseudocode, but it gives you the idea of what you're looking to build from the outside, at the browser layer.

This then tells you need a controller, and some tests for it:

describe "BoardController.show" do
  test "should render a board" do
    ...
  end  
end
  
describe "TaskController.new" do
  test "should render a  new task creation form" do
    ...
  end
end
  
describe "TaskController.create" do
  test "should create a new task and render it on the board" do
    ...
  end
end

Which from here, guides you to the point where you'd realize you need to create a Boards context, and possibly a Task context, depending on how you want to architect your bounded context, which would result in the following tests:

describe "Boards.get/1" do
  test "should retreive a board from the database" do
    ...
  end  
end
  
describe "Boards.create_task(board, params)" do
  test "should create a new task when given a board and valid params" do
    ...
  end
  
  test "should return an error tuple, with a changeset showing the errors when given improper params" do
    ...
  end
end

This would drive us down to understanding what schema and database migrations need to be created, along with what the changeset validation on create would look like.

Depending on how rigorously you like to test, you may implement that changeset validation as a test, or at this point you may begin the process of rebounding back up the tower of tests, beginning by writing the necessary tests to first make each of the Schema or Context tests go green.

From there, you'd implement the code to pass the controller tests, and then the browser tests.

Inside Out - Interactive video player example

Now you may say that's all well and good for a trivial example like this, but this isn't so complicated.

What about if you're doing something you've never done before, and aren't sure where to start?

That's a great question.

Begin with the end in mind

Well, let's say you're developing an interactive multiple video player that needs to be kept in sync, with a single timeline that can control all of them.

You may say you don't know where to begin, but you've actually defined it well enough right there that you can start, and that drives the architecture both of the implementation, and testing.

Just by saying you've got multiple players that need to be kept in sync and manipulated via a single shared element, you've already told us that you're looking at a more JavaScript heavy application interface.

Now you could just lean on browser testing, using a Chromedriver, or other webdriver type approach, but those tests are both expensive and slow compared to lower level tests, and can fail for a multitude of reasons.

This cost is likely worthwhile, but you'll also want faster feedback while iterating on the development.

And that tells you that in order to ensure maximum testability you probably ought to invest in a bit of testing using a JavaScript test runner and some of that ecosystem.

You'd still likely start with the Browser tests, as above, but at the level where you start building into the Controller in the traditional MVC model, you'd bifurcate, into both JavaScript/Jest tests on one side, and Controller tests on the other.

This assumes you're not actually going straight into a single page application. If you were going into a single page application then you'd need the API endpoint tests on the other side.

So you can start your tests with essentially pseudocode, that you then flesh out as actual implementation code, filling in things as you go.

Here we start with some pseudocode for a browser test:

describe "multiple video players synced via a single timeline" do
  test "should all video players to the same point in time when a time is selected on the timeline" do
    refute video_player1.time == halfway_point_in_timeline
    refute video_player2.time == halfway_point_in_timeline
    Browser.click(halfway_point_in_timeline)
    assert video_player1.time == halfway_point_in_timeline
    assert video_player2.time == halfway_point_in_timeline
  end
  
    test "should pause all video players when the global pause element is clicked" do
    refute video_player.paused?
    refute video_player.paused?
    Browser.click(“#global_pause_element”)
    assert video_player1.paused?
    assert video_player2.paused?
  end
end

These tests don't have to be set in stone. The exact browser API or button names don't have to stay the same from the time you write the first test to the final commit. They don’t even have to be actual browser APIs.

You can then lookup (or if you knew it off the top of your head) implement the actual syntax.

You can, and should modify them as you learn more. But the test allows you to directionally define a direction, and learn more as you go. Obviously the closer you can define to the final state initially the better.

By starting with these tests, it also ensures the solution you develop isn't untestable (or incredibly difficult to test) in the end. So you never end up with a solution that is completely unmaintainable, and needs rewritten.

Give it a shot

By now you should have an idea of how you could give outside in TDD a shot. If not, reach out to me at brittonbroderick@gmail.com.

I'd love to see how I could help you practice this, especially if you're at an earlier stage in the Elixir ecosystem.

If you'd like to learn more about these concepts, you'd probably also want to check out Behavior Driven Development (BDD) or Acceptance Test Driven Development (ATDD).