Building a Blog
I'll go through what it took to build the site you are viewing this on.
I started off knowing that I wanted to create a site that can organize my side projects and be an outlet for my thoughts. I started off by looking at off the shelf static site generators. There are many to choose from and I saw a few that source their content from markdown files. Seeing that give me the idea to make my own site from scratch and use a markdown to HTML tool.
Lately when writing code in my free time I lean towards going to Elixir, so naturally I started my search for a tool there. There I discovered nimble_publisher which is intended for what I am wanting to do.
Markdown to HTML
While reading through nimble_publisher documentation there are links that guide you through a generic blog site. These were really helpful and got me going in the right direction.
I then started by creating the project under the code name sour_shark
mkdir ~/source/sour_shark
cd $_
mix new . --sup
I started creating files with heavy influence from the guides listed above.
I added my dependencies and then ran mix deps.get
# mix.exs
defp deps do
[
{:nimble_publisher, "~> 1.1"},
{:bandit, "~> 1.0"}
]
end
Then created the modules that power nimble_publisher
lib/sour_shark/blog.ex
defmodule SourShark.Blog do
alias SourShark.Blog.Post
use NimblePublisher,
build: Post,
from: "priv/posts/**/*.md",
as: :posts
@posts Enum.sort_by(@posts, & &1.date, {:desc, Date})
def all_posts, do: @posts
def all_development_posts,
do: Enum.filter(@posts, &(&1.category == :development))
end
lib/sour_shark/post.ex
defmodule SourShark.Blog.Post do
@enforce_keys [:id, :title, :category, :body, :description, :tags, :date, :path]
defstruct [:id, :title, :category, :body, :description, :tags, :date, :path]
def build(filename, attrs, body) do
path = "#{String.replace(filename, "priv/", "")}.html"
[year, month_day_id] = filename |> Path.rootname() |> Path.split() |> Enum.take(-2)
[month, day, id] = String.split(month_day_id, "-", parts: 3)
date = Date.from_iso8601!("#{year}-#{month}-#{day}")
struct!(
__MODULE__,
[id: id, date: date, body: body, path: path] ++ Map.to_list(attrs)
)
end
end
Now I wanted to test the markdown to HTML process. So I started by creating this post
touch priv/posts/2024/09-08-building-a-blog-part-1.md
Afterwards I put in some markdown to validate the HTML looked correct. I did so by opening the file in my browser. As expected the HTML looked right, but just ugly. The next step is making it look better
Styling and Scaffolding
With the core component validated making HTML, the next step is improving the appearance using CSS and designing pages for better navigation. I aim to keep the tooling simple, it only needs to build static assets for web server deployment.
Tools
- tailwindcss: A simple, community-supported styling library with a CLI that avoids the need for node and NPM management.
- highlight-js: Chosen for its ease of setup and extensive language support, as opposed to makeup which I was not able to get the styling I wanted after tinkering.
- EEx: Used for generating HTML files. Considering Phoenix for hot reloading and templating APIs in the future. For now will just use EEx with Bandit/Plug will serve locally.
Building the Site
I need to add an easy to invoke entry point in order to invoke the process of converting markdown files to HTML and to build any assets needed for browser rendering. This can be done by creating our own mix task.
Our building of the web site consists of the following components:
- Retrieving highlight.js JS and CSS files
- Running HTML files through EEx
- Converting markdown into HTML
- Bundling up all of the assets (images/js/css)
Our initial build task looks as follows:
lib/mix/tasks/build.ex
defmodule Mix.Tasks.Build do
use Mix.Task
@impl Mix.Task
@output_dir "./output"
@shortdoc "Creates all the static files for the site"
def run(_args) do
{micro, _} =
:timer.tc(fn ->
clean_output_dir()
download_highlight_js_assets()
build_pages()
build_blog()
build_assets()
end)
ms = micro / 1000
IO.puts("Finished building site in #{ms}ms")
end
defp clean_output_dir() do
IO.puts("Cleaning output directory")
{_, exit_status} = System.cmd("rm", ["-rf", @output_dir])
if exit_status != 0, do: raise("Failed to clean output directory")
end
defp download_highlight_js_assets() do
end
defp build_pages() do
end
defp build_blog() do
end
defp build_assets() do
end
end
This gives us the ability to create all the needed site files by running
mix build
Below is the breakdown of each component that composes the build
Highlight JS
lib/mix/tasks/build.ex
@highlight_js_version "11.9.0"
@highlight_js_cdn "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/#{@highlight_js_version}"
@highlight_js_languages ["elixir", "bash", "yaml", "json", "csharp"]
defp download_highlight_js_assets() do
IO.puts("Downloading highlight.js assets")
{_, exit_status} = System.cmd("mkdir", ["-p", "#{@output_dir}/js/languages"])
if exit_status != 0, do: raise("Failed to create directories for highlight.js assets")
{highlight_js_output, highlight_js_exit_status} =
System.cmd("wget", ["-P", "#{@output_dir}/js", "#{@highlight_js_cdn}/highlight.min.js"])
if highlight_js_exit_status != 0 do
raise("Failed to download highlight.js: #{highlight_js_output}")
end
Enum.each(@highlight_js_languages, fn lang ->
{lang_output, lang_exit_status} =
System.cmd("wget", [
"-P",
"#{@output_dir}/js/languages",
"#{@highlight_js_cdn}/languages/#{lang}.min.js"
])
if lang_exit_status != 0 do
raise("Failed to download highlight.js language #{lang}: #{lang_output}")
end
end)
end
This just uses wget
to get the specified version for highlight.js and each language I care about. These get downloaded into the output directory which will then be the staging area for what gets deployed to the web server.
HTML EEx Templates
lib/mix/tasks/build.ex
defp build_pages() do
for source <- Path.wildcard("lib/sour_shark/templates/pages/*.html.eex") do
target = source |> Path.basename() |> String.replace(".eex", "")
IO.puts("Building #{source} -> #{target}")
File.write!(
"#{@output_dir}/#{target}",
eval_file("lib/sour_shark/templates/root.html.eex",
assigns: [extra_head: nil, content: eval_file(source, assigns: [])]
)
)
end
end
lib/sour_shark/templates/root.html.eex
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title><%= Application.get_env(:sour_shark, :product_name) %></title>
<link href="/css/app.css" rel="stylesheet">
<%= @extra_head %>
</head>
<body class="bg-white antialiased dark:bg-slate-800">
<header>
<div class="flex items-center justify-between border-b border-secondary-800 p-3 dark:border-secondary-800 dark:bg-gray-800">
<div class="flex items-center gap-4">
<a href="/" class="rounded-full bg-primary/5 px-2 font-medium leading-6 text-brand">
<%= Application.get_env(:sour_shark, :product_name) %>
</a>
</div>
<div class="flex items-center gap-4 text-[0.7525rem] font-semibold leading-6 text-secondary-800 active:text-secondary-800/70">
<a href="/projects">
Projects
</a>
<a
href="https://github.com/jflowaa/sour_shark"
class="rounded-lg bg-primary-100 px-2 py-1 text-[0.8125rem] hover:bg-secondary-200/80"
>
Source Code <span aria-hidden="true">→</span>
</a>
</div>
</div>
</header>
<main class="px-4 py-5 sm:px-6 lg:px-8">
<div class="mx-auto w-fit">
<%= @content %>
</div>
</main>
</body>
</html>
There isn't much going on here. This is only taking all the *.html.eex
found and inflating them inside a root.html.eex
. Where the root.html.eex contains things like the navigation bar and headers (title, stylesheets, scripts).
Building the Blog Pages
lib/mix/tasks/build.ex
defp build_blog() do
posts = SourShark.Blog.all_posts()
extra_head =
eval_file("lib/sour_shark/templates/blog_head.html.eex",
assigns: [languages: @highlight_js_languages]
)
for post <- posts do
if Path.dirname(post.path) != ".",
do: File.mkdir_p!(Path.join([@output_dir, Path.dirname(post.path)]))
IO.puts("Building #{post.path}")
File.write!(
"#{@output_dir}/#{post.path}",
eval_file("lib/sour_shark/templates/root.html.eex",
assigns: [
extra_head: extra_head,
content: "<article class=\"prose dark:prose-invert\">#{post.body}</article>"
]
)
)
end
end
lib/sour_shark/templates/blog_head.html.eex
<script src="/js/highlight.min.js"></script>
<%= for language <- @languages do %>
<script src="/js/languages/<%= language %>.min.js"></script>
<% end %>
<script>hljs.highlightAll();</script>
Slightly more complicated than the building of pages. Instead of getting all the *.html.eex
files it is getting all the markdown files. Additionally these pages will have an extra header template to load in. This extra headers are just the highlight.js script files as this library is only being used on the blog pages.
Building Assets
lib/mix/tasks/build.ex
defp build_assets() do
IO.puts("Building assets")
{_, exit_status} = System.cmd("mkdir", ["-p", "#{@output_dir}/css"])
if exit_status != 0, do: IO.puts("Failed to create CSS directory")
for asset <- Path.wildcard("priv/static/**/*.{js,txt,ico}") do
IO.puts("Copying #{asset}")
File.cp!(asset, "#{@output_dir}/#{String.replace(asset, "priv/static/", "")}")
end
IO.puts("Building Tailwind CSS")
{output, exit_status} =
System.cmd("sh", ["-c", "source ./tailwind/build_css.sh"], stderr_to_stdout: false)
if exit_status != 0 do
raise("Failed to build Tailwind CSS: #{output}")
end
end
tailwind/build_css.sh
#! /bin/bash
./tailwind/tailwindcss -c ./tailwind/tailwind.config.js -i ./priv/static/css/app.css -o ./output/css/app.css --minify
tailwind.config.js
/** @type {import('tailwindcss').Config} */
const colors = require('tailwindcss/colors')
module.exports = {
content: [
"output/**/*.html"
],
darkMode: 'class', // remove once dark mode styles are better
theme: {
extend: {
colors: {
brand: "#800080",
primary: colors.purple,
secondary: colors.emerald,
neutral: colors.gray
}
},
},
plugins: [
require('@tailwindcss/typography'),
],
}
This copies over the static assets we already have and then finally uses Tailwind to inspect HTML files and generates the CSS file needed
Serving the Site Locally
To get a more proper experience on serving the site locally I wanted to spin up a web server. This is done by using Plug and Bandit for handling the connections. The code to do is simple:
lib/sour_shark/application.ex
defmodule SourShark.Application do
use Application
@impl true
def start(_type, _args) do
children = [
{Bandit, scheme: :http, plug: SourShark.Router, port: 8080}
]
opts = [strategy: :one_for_one, name: SourShark.Supervisor]
Supervisor.start_link(children, opts)
end
end
lib/sour_shark/router.ex
defmodule SourShark.Router do
use Plug.Router
use Plug.ErrorHandler
alias SourShark.Plug.ServeFiles
plug(Plug.Logger)
plug(:match)
plug(:dispatch)
get _ do
ServeFiles.call(conn, %{})
end
@impl Plug.ErrorHandler
def handle_errors(conn, %{kind: kind, reason: reason, stack: stack}) do
IO.inspect(kind, label: :kind)
IO.inspect(reason, label: :reason)
IO.inspect(stack, label: :stack)
send_resp(conn, conn.status, "Encountered an error that cannot be handled")
end
end
lib/sour_shark/plug/serve_files.ex
defmodule SourShark.Plug.ServeFiles do
@output_dir "./output"
def init(options), do: options
def call(%Plug.Conn{request_path: path} = conn, _opts) do
if path == "/",
do: serve_file(conn, "/index"),
else: serve_file(conn, path)
end
defp serve_file(conn, path) do
desired_path =
if path |> Path.extname() == "",
do: "#{@output_dir}#{path}.html",
else: "#{@output_dir}#{path}"
case File.read(desired_path) do
{:ok, content} ->
conn
|> Plug.Conn.put_resp_content_type(MIME.from_path(desired_path))
|> Plug.Conn.send_resp(200, content)
{:error, :enoent} ->
conn
|> Plug.Conn.send_resp(404, "Not found")
end
end
end
mix/tasks/serve.ex
defmodule Mix.Tasks.Serve do
use Mix.Task
@impl Mix.Task
@output_dir "./output"
File.mkdir_p!(@output_dir)
def run(_args) do
Application.start(:inets)
# TODO: doesn't work as desired, use `mix run --no-halt`
:inets.start(:httpd,
port: 8000,
server_root: ~c".",
document_root: @output_dir,
server_name: ~c"localhost",
mime_types: [
{~c"html", ~c"text/html"},
{~c"htm", ~c"text/html"},
{~c"js", ~c"text/javascript"},
{~c"css", ~c"text/css"},
{~c"gif", ~c"image/gif"},
{~c"jpg", ~c"image/jpeg"},
{~c"jpeg", ~c"image/jpeg"},
{~c"png", ~c"image/png"}
]
)
end
end
Now after running mix build
the command mix serve
can be ran and the site will now be hosted on http://localhost:8080
HTML Detail Tag Support
The HTML generated from nimble_publisher (which uses earmark) does not support doing a <details>
tag. The detail block could be opened and collapsed but code blocks within would not be converted into <code>
.
So I did a test for using a different markdown to HTML library, I went with the first one I found after doing a search which was MDEx. In order to set it I added an HTML converter like so:
lib/sour_shark/markdown_converter.ex
defmodule SourShark.MarkdownConverter do
def convert(filepath, body, _attrs, _opts) do
if Path.extname(filepath) in [".md", ".markdown"] do
body |> MDEx.to_html!(render: [unsafe_: true])
end
end
end
lib/sour_shark/blog.ex
defmodule SourShark.Blog do
alias SourShark.Blog.Post
alias SourShark.MarkdownConverter
use NimblePublisher,
build: Post,
from: "priv/posts/**/*.md",
html_converter: MarkdownConverter,
as: :posts
...
This library was able to produce details blocks with code blocks within. I also didn't notice any other side effects, so I'll continue to stick with this library.
Deploying
Now that it is working as expected, all that is left to do is deploy it. I created a deployment script
#!/bin/bash
rm /tmp/sour_shark.tar.gz
mix build && \
tar --no-xattrs -czvf /tmp/sour_shark.tar.gz -C output . && \
scp /tmp/sour_shark.tar.gz root@digital-ocean:/tmp && \
ssh digital-ocean << EOF
rm -rf /var/lib/caddy/sour_shark/
mkdir -p /var/lib/caddy/sour_shark/
tar -xzvf /tmp/sour_shark.tar.gz -C /var/lib/caddy/sour_shark/ && \
rm /tmp/sour_shark.tar.gz
EOF
Then updated Caddy for a new entry to serve for
jack-develops.com:80, www.jack-develops.com:80 {
root * /var/lib/caddy/sour_shark/
file_server
encode gzip
try_files {path}.html
header ?Cache-Control "public, max-age=3600"
}
To deploy I ran:
./deployment/deploy_vm.sh
More on how this works under the hood can be found at Maintaining and Deploying to this VM