Building a Blog

I'll go through what it took to build the site you are viewing this on.

I started off knowing that I wanted to create a site that can organize my side projects and be an outlet for my thoughts. I started off by looking at off the shelf static site generators. There are many to choose from and I saw a few that source their content from markdown files. Seeing that give me the idea to make my own site from scratch and use a markdown to HTML tool.

Lately when writing code in my free time I lean towards going to Elixir, so naturally I started my search for a tool there. There I discovered nimble_publisher which is intended for what I am wanting to do.

Markdown to HTML

While reading through nimble_publisher documentation there are links that guide you through a generic blog site. These were really helpful and got me going in the right direction.

I then started by creating the project under the code name sour_shark

mkdir ~/source/sour_shark
cd $_
mix new . --sup

I started creating files with heavy influence from the guides listed above.

I added my dependencies and then ran mix deps.get

# mix.exs
defp deps do
  [
    {:nimble_publisher, "~> 1.1"},
    {:bandit, "~> 1.0"}
  ]
end

Then created the modules that power nimble_publisher

lib/sour_shark/blog.ex
defmodule SourShark.Blog do
  alias SourShark.Blog.Post

  use NimblePublisher,
    build: Post,
    from: "priv/posts/**/*.md",
    as: :posts

  @posts Enum.sort_by(@posts, & &1.date, {:desc, Date})

  def all_posts, do: @posts

  def all_development_posts,
    do: Enum.filter(@posts, &(&1.category == :development))
end
lib/sour_shark/post.ex
defmodule SourShark.Blog.Post do
  @enforce_keys [:id, :title, :category, :body, :description, :tags, :date, :path]
  defstruct [:id, :title, :category, :body, :description, :tags, :date, :path]

  def build(filename, attrs, body) do
    path = "#{String.replace(filename, "priv/", "")}.html"
    [year, month_day_id] = filename |> Path.rootname() |> Path.split() |> Enum.take(-2)
    [month, day, id] = String.split(month_day_id, "-", parts: 3)
    date = Date.from_iso8601!("#{year}-#{month}-#{day}")

    struct!(
      __MODULE__,
      [id: id, date: date, body: body, path: path] ++ Map.to_list(attrs)
    )
  end
end

Now I wanted to test the markdown to HTML process. So I started by creating this post

touch priv/posts/2024/09-08-building-a-blog-part-1.md

Afterwards I put in some markdown to validate the HTML looked correct. I did so by opening the file in my browser. As expected the HTML looked right, but just ugly. The next step is making it look better

Styling and Scaffolding

With the core component validated making HTML, the next step is improving the appearance using CSS and designing pages for better navigation. I aim to keep the tooling simple, it only needs to build static assets for web server deployment.

Tools

  • tailwindcss: A simple, community-supported styling library with a CLI that avoids the need for node and NPM management.
  • highlight-js: Chosen for its ease of setup and extensive language support, as opposed to makeup which I was not able to get the styling I wanted after tinkering.
  • EEx: Used for generating HTML files. Considering Phoenix for hot reloading and templating APIs in the future. For now will just use EEx with Bandit/Plug will serve locally.

Building the Site

I need to add an easy to invoke entry point in order to invoke the process of converting markdown files to HTML and to build any assets needed for browser rendering. This can be done by creating our own mix task.

Our building of the web site consists of the following components:

  • Retrieving highlight.js JS and CSS files
  • Running HTML files through EEx
  • Converting markdown into HTML
  • Bundling up all of the assets (images/js/css)

Our initial build task looks as follows:

lib/mix/tasks/build.ex
defmodule Mix.Tasks.Build do
  use Mix.Task
  @impl Mix.Task

  @output_dir "./output"

  @shortdoc "Creates all the static files for the site"
  def run(_args) do
    {micro, _} =
      :timer.tc(fn ->
        clean_output_dir()
        download_highlight_js_assets()
        build_pages()
        build_blog()
        build_assets()
      end)

    ms = micro / 1000
    IO.puts("Finished building site in #{ms}ms")
  end

  defp clean_output_dir() do
    IO.puts("Cleaning output directory")
    {_, exit_status} = System.cmd("rm", ["-rf", @output_dir])
    if exit_status != 0, do: raise("Failed to clean output directory")
  end

  defp download_highlight_js_assets() do
  end

  defp build_pages() do
  end

  defp build_blog() do
  end

  defp build_assets() do
  end
end

This gives us the ability to create all the needed site files by running

mix build

Below is the breakdown of each component that composes the build

Highlight JS

lib/mix/tasks/build.ex
@highlight_js_version "11.9.0"
@highlight_js_cdn "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/#{@highlight_js_version}"
@highlight_js_languages ["elixir", "bash", "yaml", "json", "csharp"]

defp download_highlight_js_assets() do
  IO.puts("Downloading highlight.js assets")
  {_, exit_status} = System.cmd("mkdir", ["-p", "#{@output_dir}/js/languages"])
  if exit_status != 0, do: raise("Failed to create directories for highlight.js assets")

  {highlight_js_output, highlight_js_exit_status} =
    System.cmd("wget", ["-P", "#{@output_dir}/js", "#{@highlight_js_cdn}/highlight.min.js"])

  if highlight_js_exit_status != 0 do
    raise("Failed to download highlight.js: #{highlight_js_output}")
  end

  Enum.each(@highlight_js_languages, fn lang ->
    {lang_output, lang_exit_status} =
      System.cmd("wget", [
        "-P",
        "#{@output_dir}/js/languages",
        "#{@highlight_js_cdn}/languages/#{lang}.min.js"
      ])

    if lang_exit_status != 0 do
      raise("Failed to download highlight.js language #{lang}: #{lang_output}")
    end
  end)
end

This just uses wget to get the specified version for highlight.js and each language I care about. These get downloaded into the output directory which will then be the staging area for what gets deployed to the web server.

HTML EEx Templates

lib/mix/tasks/build.ex
defp build_pages() do
  for source <- Path.wildcard("lib/sour_shark/templates/pages/*.html.eex") do
    target = source |> Path.basename() |> String.replace(".eex", "")
    IO.puts("Building #{source} -> #{target}")

    File.write!(
      "#{@output_dir}/#{target}",
      eval_file("lib/sour_shark/templates/root.html.eex",
        assigns: [extra_head: nil, content: eval_file(source, assigns: [])]
      )
    )
  end
end
lib/sour_shark/templates/root.html.eex
<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title><%= Application.get_env(:sour_shark, :product_name) %></title>
  <link href="/css/app.css" rel="stylesheet">
  <%= @extra_head %>
</head>

<body class="bg-white antialiased dark:bg-slate-800">
  <header>
    <div class="flex items-center justify-between border-b border-secondary-800 p-3 dark:border-secondary-800 dark:bg-gray-800">
      <div class="flex items-center gap-4">
        <a href="/" class="rounded-full bg-primary/5 px-2 font-medium leading-6 text-brand">
          <%= Application.get_env(:sour_shark, :product_name) %>
        </a>
      </div>
      <div class="flex items-center gap-4 text-[0.7525rem] font-semibold leading-6 text-secondary-800 active:text-secondary-800/70">
        <a href="/projects">
          Projects
        </a>
        <a
          href="https://github.com/jflowaa/sour_shark"
          class="rounded-lg bg-primary-100 px-2 py-1 text-[0.8125rem] hover:bg-secondary-200/80"
        >
          Source Code <span aria-hidden="true">&rarr;</span>
        </a>
      </div>
    </div>
  </header>
  <main class="px-4 py-5 sm:px-6 lg:px-8">
    <div class="mx-auto w-fit">
      <%= @content %>
    </div>
  </main>
</body>
</html>

There isn't much going on here. This is only taking all the *.html.eex found and inflating them inside a root.html.eex. Where the root.html.eex contains things like the navigation bar and headers (title, stylesheets, scripts).

Building the Blog Pages

lib/mix/tasks/build.ex
defp build_blog() do
  posts = SourShark.Blog.all_posts()

  extra_head =
    eval_file("lib/sour_shark/templates/blog_head.html.eex",
      assigns: [languages: @highlight_js_languages]
    )

  for post <- posts do
    if Path.dirname(post.path) != ".",
      do: File.mkdir_p!(Path.join([@output_dir, Path.dirname(post.path)]))

    IO.puts("Building #{post.path}")

    File.write!(
      "#{@output_dir}/#{post.path}",
      eval_file("lib/sour_shark/templates/root.html.eex",
        assigns: [
          extra_head: extra_head,
          content: "<article class=\"prose dark:prose-invert\">#{post.body}</article>"
        ]
      )
    )
  end
end
lib/sour_shark/templates/blog_head.html.eex
<script src="/js/highlight.min.js"></script>
<%= for language <- @languages do %>
  <script src="/js/languages/<%= language %>.min.js"></script>
<% end %>
<script>hljs.highlightAll();</script>

Slightly more complicated than the building of pages. Instead of getting all the *.html.eex files it is getting all the markdown files. Additionally these pages will have an extra header template to load in. This extra headers are just the highlight.js script files as this library is only being used on the blog pages.

Building Assets

lib/mix/tasks/build.ex
defp build_assets() do
  IO.puts("Building assets")
  {_, exit_status} = System.cmd("mkdir", ["-p", "#{@output_dir}/css"])
  if exit_status != 0, do: IO.puts("Failed to create CSS directory")

  for asset <- Path.wildcard("priv/static/**/*.{js,txt,ico}") do
    IO.puts("Copying #{asset}")
    File.cp!(asset, "#{@output_dir}/#{String.replace(asset, "priv/static/", "")}")
  end

  IO.puts("Building Tailwind CSS")

  {output, exit_status} =
    System.cmd("sh", ["-c", "source ./tailwind/build_css.sh"], stderr_to_stdout: false)

  if exit_status != 0 do
    raise("Failed to build Tailwind CSS: #{output}")
  end
end
tailwind/build_css.sh
#! /bin/bash

./tailwind/tailwindcss -c ./tailwind/tailwind.config.js -i ./priv/static/css/app.css -o ./output/css/app.css --minify
tailwind.config.js
/** @type {import('tailwindcss').Config} */

const colors = require('tailwindcss/colors')

module.exports = {
  content: [
    "output/**/*.html"
  ],
  darkMode: 'class', // remove once dark mode styles are better
  theme: {
    extend: {
      colors: {
        brand: "#800080",
        primary: colors.purple,
        secondary: colors.emerald,
        neutral: colors.gray
      }
    },
  },
  plugins: [
    require('@tailwindcss/typography'),
  ],
}

This copies over the static assets we already have and then finally uses Tailwind to inspect HTML files and generates the CSS file needed

Serving the Site Locally

To get a more proper experience on serving the site locally I wanted to spin up a web server. This is done by using Plug and Bandit for handling the connections. The code to do is simple:

lib/sour_shark/application.ex
defmodule SourShark.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      {Bandit, scheme: :http, plug: SourShark.Router, port: 8080}
    ]

    opts = [strategy: :one_for_one, name: SourShark.Supervisor]
    Supervisor.start_link(children, opts)
  end
end
lib/sour_shark/router.ex
defmodule SourShark.Router do
  use Plug.Router
  use Plug.ErrorHandler

  alias SourShark.Plug.ServeFiles

  plug(Plug.Logger)
  plug(:match)
  plug(:dispatch)

  get _ do
    ServeFiles.call(conn, %{})
  end

  @impl Plug.ErrorHandler
  def handle_errors(conn, %{kind: kind, reason: reason, stack: stack}) do
    IO.inspect(kind, label: :kind)
    IO.inspect(reason, label: :reason)
    IO.inspect(stack, label: :stack)
    send_resp(conn, conn.status, "Encountered an error that cannot be handled")
  end
end
lib/sour_shark/plug/serve_files.ex
defmodule SourShark.Plug.ServeFiles do
  @output_dir "./output"

  def init(options), do: options

  def call(%Plug.Conn{request_path: path} = conn, _opts) do
    if path == "/",
      do: serve_file(conn, "/index"),
      else: serve_file(conn, path)
  end

  defp serve_file(conn, path) do
    desired_path =
      if path |> Path.extname() == "",
        do: "#{@output_dir}#{path}.html",
        else: "#{@output_dir}#{path}"

    case File.read(desired_path) do
      {:ok, content} ->
        conn
        |> Plug.Conn.put_resp_content_type(MIME.from_path(desired_path))
        |> Plug.Conn.send_resp(200, content)

      {:error, :enoent} ->
        conn
        |> Plug.Conn.send_resp(404, "Not found")
    end
  end
end
mix/tasks/serve.ex
defmodule Mix.Tasks.Serve do
  use Mix.Task
  @impl Mix.Task

  @output_dir "./output"
  File.mkdir_p!(@output_dir)

  def run(_args) do
    Application.start(:inets)

    # TODO: doesn't work as desired, use `mix run --no-halt`
    :inets.start(:httpd,
      port: 8000,
      server_root: ~c".",
      document_root: @output_dir,
      server_name: ~c"localhost",
      mime_types: [
        {~c"html", ~c"text/html"},
        {~c"htm", ~c"text/html"},
        {~c"js", ~c"text/javascript"},
        {~c"css", ~c"text/css"},
        {~c"gif", ~c"image/gif"},
        {~c"jpg", ~c"image/jpeg"},
        {~c"jpeg", ~c"image/jpeg"},
        {~c"png", ~c"image/png"}
      ]
    )
  end
end

Now after running mix build the command mix serve can be ran and the site will now be hosted on http://localhost:8080

HTML Detail Tag Support

The HTML generated from nimble_publisher (which uses earmark) does not support doing a <details> tag. The detail block could be opened and collapsed but code blocks within would not be converted into <code>.

So I did a test for using a different markdown to HTML library, I went with the first one I found after doing a search which was MDEx. In order to set it I added an HTML converter like so:

lib/sour_shark/markdown_converter.ex
defmodule SourShark.MarkdownConverter do
  def convert(filepath, body, _attrs, _opts) do
    if Path.extname(filepath) in [".md", ".markdown"] do
      body |> MDEx.to_html!(render: [unsafe_: true])
    end
  end
end
lib/sour_shark/blog.ex
defmodule SourShark.Blog do
  alias SourShark.Blog.Post
  alias SourShark.MarkdownConverter

  use NimblePublisher,
    build: Post,
    from: "priv/posts/**/*.md",
    html_converter: MarkdownConverter,
    as: :posts
  ...

This library was able to produce details blocks with code blocks within. I also didn't notice any other side effects, so I'll continue to stick with this library.

Deploying

Now that it is working as expected, all that is left to do is deploy it. I created a deployment script

#!/bin/bash

rm /tmp/sour_shark.tar.gz
mix build && \
tar --no-xattrs -czvf /tmp/sour_shark.tar.gz -C output . && \
scp /tmp/sour_shark.tar.gz root@digital-ocean:/tmp && \
ssh digital-ocean << EOF
rm -rf /var/lib/caddy/sour_shark/
mkdir -p /var/lib/caddy/sour_shark/
tar -xzvf /tmp/sour_shark.tar.gz -C /var/lib/caddy/sour_shark/ && \
rm /tmp/sour_shark.tar.gz
EOF

Then updated Caddy for a new entry to serve for

jack-develops.com:80, www.jack-develops.com:80 {
	root * /var/lib/caddy/sour_shark/
	file_server
	encode gzip
	try_files {path}.html
	header ?Cache-Control "public, max-age=3600"
}

To deploy I ran:

./deployment/deploy_vm.sh

More on how this works under the hood can be found at Maintaining and Deploying to this VM