Deploying Elixir Applications with Duct Tape and Bubblegum

Eric Oestrich -

There are a lot of examples on the internet (including some I've written) of how to deploy your Elixir applications, and a lot of them are pretty complicated. I want to show you a simple but reliable deployment example that's a simpler place for you to get started if you're new to deploying Elixir.

I do want to point out that this is not what we do at SmartLogic; but some of my side project deployments are fairly similar. This guide will be skipping everything but base Elixir releases and a bash script that runs a few ssh/scp commands for you. No Ansible, no Kubernetes, no automated deployments. The one thing it does reach for is Docker, to generate the release in the same environment as your server.

This guide is geared towards a fairly standard Phoenix application, but can be adapted for use with a plain Elixir app by removing the Phoenix-specific steps. This guide also assumes you're looking to deploy on a cheap VPS on something like Linode or DigitalOcean.

Release Configuration

We will be configuring the generated release with the file config/releases.exs. See below for my minimal runtime configuration. We'll fill in SECRET_KEY_BASE and DATABASE_URL in the systemd service file.

import Config

config :my_app, Web.Endpoint,
  http: [:inet6, port: 4000],
  url: [host: "myapp.example.com", port: 443, scheme: "https"],
  cache_static_manifest: "priv/static/cache_manifest.json"

config :my_app, Web.Endpoint, secret_key_base: System.get_env("SECRET_KEY_BASE")

config :my_app, MyApp.Repo,
  url: System.get_env("DATABASE_URL"),
  pool_size: 15

config :phoenix, :serve_endpoints, true

config :logger, level: :info

Migrations

We'll also need to set up a migration task. This is mostly copy-pasted from distillery. Include this as lib/my_app/release_tasks.ex and update for your application.

# From https://github.com/bitwalker/distillery/blob/master/docs/guides/running_migrations.md
defmodule MyApp.ReleaseTasks do
  @moduledoc false

  @start_apps [
    :crypto,
    :ssl,
    :postgrex,
    :ecto,
    :ecto_sql
  ]

  @apps [
    :my_app
  ]

  @repos [
    MyApp.Repo
  ]

  def migrate() do
    startup()

    # Run migrations
    Enum.each(@apps, &run_migrations_for/1)

    # Signal shutdown
    IO.puts("Success!")
  end

  defp startup() do
    IO.puts("Loading my_app...")

    # Load the code for my_app, but don't start it
    Application.load(:my_app)

    IO.puts("Starting dependencies..")
    # Start apps necessary for executing migrations
    Enum.each(@start_apps, &Application.ensure_all_started/1)

    # Start the Repo(s) for my_app
    IO.puts("Starting repos..")
    Enum.each(@repos, & &1.start_link(pool_size: 2))
  end

  def priv_dir(app), do: "#{:code.priv_dir(app)}"

  defp run_migrations_for(app) do
    IO.puts("Running migrations for #{app}")
    Ecto.Migrator.run(MyApp.Repo, migrations_path(app), :up, all: true)
  end

  defp migrations_path(app), do: Path.join([priv_dir(app), "repo", "migrations"])
end

Setting up your Dockerfile

First let's configure a multi-stage Dockerfile. This will generate in stages your final release. First it compiles only your dependencies, which will cache as long as your mix.exs and mix.lock files don't change.

Next up is your front-end assets. If these package.json and yarn.lock don't change they are cached as well. You can remove this step if you don't have front-end assets. Finally, it generates your release and makes a tarball in the /opt directory for you to copy out later.

Save this file as Dockerfile.releaser since this is specific to generating a release and copying it out. If you eventually move to something like Kubernetes, this will be a good start.

FROM elixir:1.9 as builder
RUN mix local.rebar --force && \
    mix local.hex --force
WORKDIR /app
ENV MIX_ENV=prod
COPY mix.* /app/
RUN mix deps.get --only prod
RUN mix deps.compile

FROM node:10.9 as frontend
WORKDIR /app
COPY assets/package.json assets/yarn.lock /app/
COPY --from=builder /app/deps/phoenix /deps/phoenix
COPY --from=builder /app/deps/phoenix_html /deps/phoenix_html
RUN npm install -g yarn && yarn install
COPY assets /app
RUN yarn run deploy

FROM builder as releaser
ENV MIX_ENV=prod
COPY --from=frontend /priv/static /app/priv/static
COPY . /app/
RUN mix phx.digest
RUN mix release && \
  cd _build/prod/rel/my_app/ && \
  tar czf /opt/my_app.tar.gz .

Release Generation

Next we have a simple bash script to generate the release. This builds the docker image with the releaser-specific file. Once the generation is complete, the tarball is copied out into a local tmp folder.

Copy this into release.sh 

  
#!/bin/bash
set -e

mkdir -p tmp/

docker build -f Dockerfile.releaser -t my_app:releaser .

DOCKER_UUID=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1)

docker run -ti --name my_app_releaser_${DOCKER_UUID} my_app:releaser /bin/true
docker cp my_app_releaser_${DOCKER_UUID}:/opt/my_app.tar.gz tmp/
docker rm my_app_releaser_${DOCKER_UUID}

And run chmod +x release.sh to make it executable.

Setting up the Server

We're going to set up the server with a fairly straightforward set of commands, directly on the server. This guide assumes you're using Ubuntu 18.04 LTS.

Once you're done with these commands, you should make sure the user you signed into can only be accessed by SSH keys. If you signed in via root with a password you should disable that and sign in with deploy going forward.

# Make sure you're up to date
sudo apt update
sudo apt upgrade
# This should already be installed, but to be safe
sudo apt install unattended-upgrades

# Reboot for the upgrades
sudo systemctl reboot

# Install postgres
sudo bash -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

sudo apt update
sudo apt install postgresql-12

# Create deploy user and database in PostgreSQL
# save the password for later
sudo -u postgres createuser deploy -P
sudo -u postgres createdb my_app -O deploy

# Create a deploy user and grant sudo access
sudo useradd -m -s /bin/bash deploy
sudo bash -c 'echo "deploy ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/deploy'
sudo -u deploy bash -c 'echo -e "export DATABASE_URL=postgresql://deploy:PASSWORD@localhost/my_app\n$(cat /home/deploy/.bashrc)" > /home/deploy/.bashrc'

# Create folder to untar into
sudo -u deploy mkdir /home/deploy/my_app

# Setup a systemd file for your application
# Edit and copy in the file below
sudo vim /etc/systemd/system/my_app.service
sudo systemctl daemon-reload
sudo systemctl enable my_app

# Copy in your SSH keys to the deploy user
sudo -u deploy mkdir /home/deploy/.ssh/
sudo -u deploy touch /home/deploy/.ssh/authorized_keys
sudo -u deploy chmod 0600 /home/deploy/.ssh/authorized_keys
# add your ssh keys to this file
sudo -u deploy vim /home/deploy/.ssh/authorized_keys

# Setup nginx and certbot
sudo apt install nginx certbot python-certbot-nginx -y
sudo rm /etc/nginx/sites-enabled/default
# Copy in the nginx config below and edit for your domain/app
sudo vim /etc/nginx/sites-enabled/my_app
# Verify the configuration
sudo nginx -t
sudo systemctl reload nginx
# Follow certbot to configure HTTPS, this assumes your DNS is configured
sudo certbot --nginx
# After certbot, you may want to edit /etc/nginx/sites-enabled/myapp to verify and update any config it generated

# Enable the firewall
sudo ufw allow ssh
sudo ufw allow https
sudo ufw allow http
sudo ufw enable

systemd file

The systemd configuration file mentioned in the script above.

[Unit]
Description=Runner for MyApp
After=network.target

[Service]
User=deploy
Group=deploy
WorkingDirectory=/home/deploy/my_app
Environment=LANG=en_US.UTF-8
Environment=SECRET_KEY_BASE="run `mix phx.gen.secret` to generate one"
Environment=DATABASE_URL="postgresql://deploy:PASSWORD@localhost/my_app"
ExecStart=/home/deploy/my_app/bin/my_app start
SyslogIdentifier=my_app
RemainAfterExit=no
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

nginx config

The nginx configuration file mentioned in the script above.

upstream my_app {
  server localhost:4000;
}

server {
  server_name my-app.example.com;

  gzip on;

  client_max_body_size 2M;

  error_page 502 /site-down.html;
  location = /site-down.html {
    root /usr/share/nginx/html;
    internal;
  }

  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;

    proxy_pass http://my_app;
  }
  
  # CSS and Javascript
  location ~* \.(?:css|js)$ {
    expires 1w;
    access_log off;
    add_header Cache-Control "public";
    proxy_pass http://my_app;
  }

  # Images
  location ~* \.(jpe?g|png|gif|ico)$ {
    expires 1w;
    access_log off;
    add_header Cache-Control "public";
    proxy_pass http://my_app;
  }
  
  listen 80 default;
  listen [::]:80;
}

Deploying the Release

Once we have a local release and the server prepped, we can finally do the deploy!

Save the following as deploy.sh and update with your applications information. Run chmod +x deploy.sh after saving so it can execute.

set -e

host=my-app.example.com

./release.sh
scp tmp/my_app.tar.gz deploy@${host}:

ssh deploy@${host} 'sudo systemctl stop my_app'
ssh deploy@${host} 'tar xzf my_app.tar.gz  -C my_app'
ssh deploy@${host} './my_app/bin/my_app eval "MyApp.ReleaseTasks.migrate()"'
ssh deploy@${host} 'sudo systemctl start my_app'

With that in place, you can now perform a deploy! ./deploy.sh to get your application on the remote server. You should be able to see your application running after this succeeds.

A Simple but Solid Foundation

There are a few drawbacks with this approach. Your application will be down when deploying. This is likely an OK thing depending on where you are in your application's lifecycle. If no one is using the application because you're still creating it, will anyone notice the 3-5 seconds of downtime?

This guide is here to get you going with a solid and simple foundation for deploying your Elixir application. Once you get further along with your application and more comfortable with releases and deploying, you can start looking at more complex options. But a single Linode/DigitalOcean instance will go a long way with Elixir, so that day may be far away.