Jack Develops
Projects About Source Code

Overview

This post should be quick as I try to keep this setup easy and cheap. I'll start by going over the infrastructure and then finish it up by going through how I deploy sites.

Infrastructure

This site and all other projects you see hosted are running on a DigitalOcean droplet. The droplet SKU is the cheapest they have, the $4 droplet.

This VM comes with:

  • 1 vCPU
  • 512 MBs of RAM
  • 10 GBs of storage
  • 500 GBs of bandwidth

I do run into issues with the limited amount of RAM, but all those issues are related to deploying code and not the code that runs on them. I'll touch on that point later.

On this VM I run Caddy. It's a web server that I use to reverse proxy the sites. I would say it's slightly more easier to use than nginx.

The final component is Cloudflare, for better or for worse they have become the dominate tooling in this space. I use them for DNS routing because it is easy and free.

Below is how I configure the VM

VM setup process

Steps for setting up the server at https://jack-develops.com

Initial Environment Setup

Let's set up the environment first

mkdir sites/
apt update
apt upgrade --yes
apt install unzip git curl --yes
echo "export RELEASE_COOKIE=secret" >> /etc/profile
echo "export SECRET_KEY_BASE=secret" >> /etc/profile

Swap File

Due to the virtual machines limited amount of RAM, it will run out of memory when it comes to compiling things like erlang or even Elixir projects using Phoenix. By adding a swap file it can get through the compilation on these things. So what I do is have a swap file setup and turn it on when needed

fallocate -l 1G /swapfile
chmod 600 /swapfile
mkswap /swapfile

This will create a file allocated for 1 gigabyte of storing, which on this VM means 9 GBs is left for everything else storage related. Then to turn it on and allow for the swaps to happen when under memory pressure

swapon /swapfile

This swap is now enabled until the machine reboots.

Install asdf

adsf is a package manager for installing many things. I use it for managing installed Elixir versions.

git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.14.1
echo ". $HOME/.asdf/asdf.sh" >> ~/.bashrc
source ~/.bashrc
asdf plugin add erlang https://github.com/asdf-vm/asdf-erlang.git
asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git
asdf plugin-add rebar https://github.com/Stratus3D/asdf-rebar.git

Install Erlang

Installing using adsf

apt -y install build-essential autoconf m4 libncurses-dev libwxgtk3.2-dev libwxgtk-webview3.2-dev libgl1-mesa-dev libglu1-mesa-dev libpng-dev libssh-dev unixodbc-dev xsltproc fop libxml2-utils openjdk-17-jdk
export KERL_CONFIGURE_OPTIONS="--disable-debug --without-javac"
asdf install erlang 26.2.5.3
asdf install rebar 3.24.0
asdf global rebar 3.24.0

Using asdf to install Erlang will fail due to error 137, running out of memory. The virtual machine has 500 megabytes of RAM which is not enough to build from source without using a swap. It's also worth noting that this will take a long time (15 minutes) to compile given the CPU speeds.

Thus installing from a debian package also works, here's how

apt install libncurses5 libsctp1 --yes
wget http://security.debian.org/debian-security/pool/updates/main/o/openssl/libssl1.1_1.1.1n-0+deb11u5_amd64.deb
dpkg -i libssl1.1_1.1.1n-0+deb11u5_amd64.deb
wget https://binaries2.erlang-solutions.com/debian/pool/contrib/e/esl-erlang/esl-erlang_26.2.3-1~debian~buster_amd64.deb
dpkg -i esl-erlang_26.2.3-1~debian~buster_amd64.deb

Install Elixir

Installing using adsf. Currently using a few versions

asdf local elixir 1.15
asdf local elixir 1.16

Caddy

Install Caddy, a web server that will serve our websites

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

Now we want to create the configuration file, Caddyfile

touch ~/Caddyfile

Then add the following content

jack-develops.com:80, www.jack-develops.com:80 {
  root * /var/lib/caddy/sour_shark/
  file_server
  encode gzip
  try_files {path}.html
  header ?Cache-Control "public, max-age=3600"
}

blue-mint.jack-develops.com:80 {
  reverse_proxy localhost:4001
}

wavy-bird.jack-develops.com:80 {
  respond "Not deployed"
}

icky-venus.jack-develops.com:80 {
  reverse_proxy localhost:4000
}

Finally reload the Caddyfile after making changes

caddy reload

Deploying Sites

I went through many ways to deploy these sites. My first route was to build a docker image and have the VM run those. Which I then quickly ran into the problem of different CPU architectures as I'm using an ARM (Apple silicon) CPU and the VM on DigitalOcean uses an Intel x86. My next strategy was to build the image on the VM instead, but this made for slow deployments, and overall, I found this to be overkill.

My next idea was to build the code locally and tarball it, then SCP it to the VM. But before I could do that I needed the VM setup to have the run times needed to run the code. This brought me to using adsf to install the run times needed. Tried using other tools but kept running out of memory during Erlang installs due to having to build from source.

But then I ran into the issue of the code built locally won't run on the VM due to CPU architecture differences. So I then went the route of having the code built on the VM.

Here's an example deploy_vm.sh script for an Elixir application

#!/bin/bash

rm /tmp/icky_venus.tar.gz
tar --exclude-from='deployment/exclude.txt' --no-xattrs -czvf /tmp/icky_venus.tar.gz . && \
scp /tmp/icky_venus.tar.gz root@digital-ocean:/tmp && \
ssh digital-ocean << EOF
rm -rf ~/sites/icky_venus/
mkdir -p ~/sites/icky_venus/
tar -xzvf /tmp/icky_venus.tar.gz -C ~/sites/icky_venus && \
rm /tmp/icky_venus.tar.gz && \
cd ~/sites/icky_venus/ && \
asdf local elixir 1.16 && \
asdf local erlang 26.2.5.3 && \
mix deps.get && \
MIX_ENV=prod mix release --overwrite && \
~/sites/icky_venus/_build/prod/rel/icky_venus/bin/icky_venus daemon && \
~/sites/icky_venus/_build/prod/rel/icky_venus/bin/icky_venus restart
EOF

Phoenix sites are a bit different

deployment/deploy_vm.sh 
#!/bin/bash

rm /tmp/blue_mint.tar.gz
tar --exclude-from='deployment/exclude.txt' --no-xattrs -czvf /tmp/blue_mint.tar.gz . && \
scp /tmp/blue_mint.tar.gz root@digital-ocean:/tmp && \
ssh digital-ocean << EOF
rm -rf ~/sites/blue_mint/
mkdir -p ~/sites/blue_mint/
tar -xzvf /tmp/blue_mint.tar.gz -C ~/sites/blue_mint && \
rm /tmp/blue_mint.tar.gz && \
cd ~/sites/blue_mint/ && \
asdf local elixir 1.16 && \
asdf local erlang 26.2.5.3 && \
MIX_ENV=prod mix setup && \
MIX_ENV=prod mix assets.deploy && \
MIX_ENV=prod mix phx.gen.release && \
MIX_ENV=prod mix release && \
PHX_SERVER=true ~/sites/blue_mint/_build/prod/rel/blue_mint/bin/blue_mint daemon
EOF

Finally, if I need to IEx into a site and to tweak or inspect things I can do so by doing the following:

ssh digital-ocean
~/sites/icky_venus/_build/prod/rel/icky_venus/bin/icky_venus remote