Starting Browser Testing for Phoenix LiveView with Wallaby

Browser testing can be a powerful way to build confidence in your codebase, and it's easy to get started in a Phoenix app with Wallaby.

I gave a conference talk about this at The Big Elixir 2022 with a few more examples. If you just want to dive into the code, it can be found here.

What is Wallaby?

Wallaby is a browser testing tool written in Elixir that allows you to drive a browser, and write browser-based tests all without leaving the Elixir ecosystem. So you have access to all your favorite tools like ExUnit.

Setting Up

Install Wallaby

Add the dependency to your mix.exs:

defp deps do
  ...
  {:wallaby, "~> 0.29.0", runtime: false, only: :test}
end

mix.exs

Install Chromedriver

How you install the chromedriver is going to depend on your preferences. On MacOS you can install it with brew but the latest security updates make that a little more painful and less cross-platform friendly in the event that cross-platform matters to you.

brew install --cask chromedriver && \ xattr -d com.apple.quarantine /usr/local/Caskroom/chromedriver/$VERSION/chromedriver

Installing chromedriver with brew

You can also install the binary directly from https://sites.google.com/chromium.org/driver/.

Finally, the easiest cross-platform option that I've found is just installing it via npm.

npm install chromedriver --save-dev --prefix assets

Installing chromedriver with npm

Update the endpoint

You need to update the endpoint.ex to leverage Ecto's SQL sandbox, which allows your concurrent/async tests to be isolated at the database layer.

You also need to modify the socket connect_info to include :user_agent this means that the user agent information is passed along to the socket. The user agent is how Wallaby links up which isolated sandbox database process should be connected with the particular browser test session.

defmodule MyAppWeb.Endpoint do
  use Phoenix.Endpoint, otp_app: :my_app
  
  if Application.get_env(:my_app, :sql_sandbox) do
    plug(Phoenix.Ecto.SQL.Sandbox)
  end
  
  ...
  
  socket("/live", Phoenix.LiveView.Socket,
    websocket: [connect_info: [:user_agent, session: @session_options]]
  )

  ...
end

lib/my_app_web/endpoint.ex

Update the test helper

We need to ensure Wallaby is started as part of running your tests:

{:ok, _} = Application.ensure_all_started(:wallaby)
...

test/test_helper.exs

Update the test config

You now want to configure your application in test to use the Ecto SQL sandbox, and then you need to configure Wallaby.

import Config
...

config :my_app, sql_sandbox: true

config :wallaby,
  base_url: MyAppWeb.Endpoint.url(),
  otp_app: :my_app,
  screenshot_on_failure: true,
  chromedriver: [
    path: "assets/node_modules/chromedriver/bin/chromedriver", # point to your chromedriver path
    headless: true # change to false if you want to see the browser in action
  ]

config/test.exs

A few of the options that you may want to consider modifying include:

screenshot_on_failure - this option ensures that the . You can also configure the path to where the screenshots ought to be dumped.
chromedriver - includes the path to find the chromedriver binary, as well as headless mode, which determines if you'll see the actual browser bootup and take the actions you've given it, or if it will run in the background.

Add the on_mount hook

If you're using Phoenix LiveView you'll also need to add an on_mount hook, which leverages the mount lifecycle hook to ensure that on each LiveView mount the browser and test instance are correctly wired up together.

Note that this assumes Phoenix LiveView version 0.17.7. Things may have changed for you.

Below is an example hook implementation:

defmodule MyAppWeb.Hooks.AllowEctoSandbox do
  import Phoenix.LiveView

  def on_mount(:default, _params, _session, socket) do
    allow_ecto_sandbox(socket)
    {:cont, socket}
  end

  defp allow_ecto_sandbox(socket) do
    if Application.get_env(:testing_live_view_wallaby, :sql_sandbox) do
      %{assigns: %{phoenix_ecto_sandbox: metadata}} =
        assign_new(socket, :phoenix_ecto_sandbox, fn ->
          if connected?(socket), do: get_connect_info(socket, :user_agent)
        end)

      Phoenix.Ecto.SQL.Sandbox.allow(metadata, Ecto.Adapters.SQL.Sandbox)
    end
  end
end

lib/my_app_web/live/hooks/allow_ecto_sandbox.ex

You'll then need to update the router to use this on_mount hook for each of your LiveView sessions with the live_session helper.

defmodule MyAppWeb.Router do
  ...
  live_session :default, on_mount: TestingLiveViewWallabyWeb.Hooks.AllowEctoSandbox do
    ...
  end
end

lib/my_app_web/router.ex

Setup a FeatureCase

This one is optional, but I've found it useful for developer ergonomics when you're building with Wallaby. It sets up an ExUnit case similar to the generated ConnCase, but geared specifically for Wallaby.

defmodule MyAppWeb.FeatureCase do
  @moduledoc """
  This module defines the test case to be used by
  tests that require setting up a full browser.

  If the test case interacts with the database,
  we enable the SQL sandbox, so changes done to the database
  are reverted at the end of every test. If you are using
  PostgreSQL, you can even run database tests asynchronously
  by setting `use MyAppWeb.FeatureCase, async: true`, although
  this option is not recommended for other databases.
  """

  use ExUnit.CaseTemplate

  using do
    quote do
      use Wallaby.Feature

      import MyAppWeb.FeatureCase
      import Wallaby.Query

      alias MyAppWeb.Router.Helpers, as: Routes

      @moduletag :e2e

      @endpoint MyAppWeb.Endpoint

      setup _ do
        on_exit(fn -> Application.put_env(:wallaby, :js_logger, :stdio) end)
      end
    end
  end

  def enable_latency_sim(session, latency) do
    Application.put_env(:wallaby, :js_logger, nil)
    Wallaby.Browser.execute_script(session, "liveSocket.enableLatencySim(#{latency})")
  end

  def disable_latency_sim(session) do
    Wallaby.Browser.execute_script(session, "liveSocket.disableLatencySim()")
  end
end

test/support/feature_case.ex

Testing a simple form interaction

You can see the full example on GitHub to enable this test to go green. But here is a snippet:

describe "live index when creating a question" do
  feature "users should be able to submit the form and create a question", %{session: session} do
    question_text = "How do I test simple things with Wallaby?"

    session
    |> visit("/questions")
    |> click(link("New Question"))
    |> fill_in(text_field("Text"), with: question_text)
    |> click(button("Save"))
    |> assert_has(css(".alert", text: "Question created successfully"))
    |> assert_has(css("#questions > tr > td", text: question_text))
  end
end

Example simple Wallaby feature test

Simulate Latency

Finally, thanks to JavaScript helpers provided by LiveView, we can also simulate high levels of latency in the user's network:

describe "simulating latency" do
  feature "should ensure the saving is shown", %{session: session} do
    html =
      session
      |> visit("/questions")
      |> click(link("New Question"))
      |> fill_in(text_field("Text"), with: "Latency isn't fun, but should be accounted for")
      |> enable_latency_sim(2000)
      |> click(button("Save"))
      |> find(css("#question-form > div > button"))
      |> Wallaby.Element.attr("innerHTML")

    assert html == "Saving..."
  end
end

Example Wallaby test with 2 seconds of latency

Conclusion

Now you should be able to get started testing your Phoenix application using Wallaby.