Blog Pipeline

The Initial Pipeline and Plan

Initially I started this blog following the tutorial and video of Network Chuck starting a blog I used his guide and even his mega script as a starting point. I however didn’t want to host this with GitHub so I had to change that part of the PowerShell script to sync with my web server in the way I wanted it to be synced. The theme I am using also works better with subdirectories as the blog posts with index.md files containing the content and the images.py script did not look through subdirectories. I also wanted to set up comments on the blog. Which are up but I am still working on locking them down.

Changing the Scripts

Updating the PS script and the images.py took the better part of the evening to figure out. Now I can just write these blog posts in Obsidian using Markdown then just run one script and they get published to my site.

In the case of the PowerShell script I wanted to use rsync to push the files to a remote server running hugo on docker but I has having trouble getting it installed in windows. I could install it but adding it to the path was just not picking it up. So my workaround was to use my WSL and run a command on the WSL through the script. Yay Linux as a subsystem!

omegahugo.ps1

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89


# PowerShell Script for Windows

# Set variables for Obsidian to Hugo copy
$sourcePath = "C:\Path\To\Obsidian\posts"
$destinationPath = "C:\Path\To\hugo\<project>\content\post"
$destinationImages = "C:\Path\To\hugo\<project>\static"

# Path within subsystem
$sitePath = "/mnt/c/path/to/hugo/<projejct>"
$remotePath = "user@hostname:/path/to/docker/blog"

# Set error handling
$ErrorActionPreference = "Stop"
Set-StrictMode -Version Latest

# Change to the script's directory
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Definition
Set-Location $ScriptDir

# Check for required commands
$requiredCommands = @('hugo')

# Check for Python command (python or python3)
if (Get-Command 'python' -ErrorAction SilentlyContinue) {
    $pythonCommand = 'python'
} elseif (Get-Command 'python3' -ErrorAction SilentlyContinue) {
    $pythonCommand = 'python3'
} else {
    Write-Error "Python is not installed or not in PATH."
    exit 1
}

foreach ($cmd in $requiredCommands) {
    if (-not (Get-Command $cmd -ErrorAction SilentlyContinue)) {
        Write-Error "$cmd is not installed or not in PATH."
        exit 1
    }
}

# Step 1: Sync posts from Obsidian to Hugo content folder using Robocopy
Write-Host "Syncing posts from Obsidian..."

if (-not (Test-Path $sourcePath)) {
    Write-Error "Source path does not exist: $sourcePath"
    exit 1
}

if (-not (Test-Path $destinationPath)) {
    Write-Error "Destination path does not exist: $destinationPath"
    exit 1
}

# Use Robocopy to mirror the directories
$robocopyOptions = @('/MIR', '/Z', '/W:5', '/R:3')
$robocopyResult = robocopy $sourcePath $destinationPath @robocopyOptions

if ($LASTEXITCODE -ge 8) {
    Write-Error "Robocopy failed with exit code $LASTEXITCODE"
    exit 1
}

# Step 2: Process Markdown files with Python script to handle image links
Write-Host "Processing image links in Markdown files..."
if (-not (Test-Path "images.py")) {
    Write-Error "Python script images.py not found."
    exit 1
}

# Execute the Python script
try {
    & $pythonCommand images.py
} catch {
    Write-Error "Failed to process image links."
    exit 1
}

# Step 4: Build the Hugo site
Write-Host "Building the Hugo site..."
try {
    hugo
} catch {
    Write-Error "Hugo build failed."
    exit 1
}

# Rsync with web server
wsl -e rsync -avzP --exclude '*.ps1' --exclude '*.py' --delete $sitePath $remotePath

Write-Host "All done! Site synced, processed, committed, built, and deployed."

For the images script I altered it to be able to look through subdirectories as well as getting jpg in addition to png.

images.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


import os
import re
import shutil

# Paths (using raw strings to handle Windows backslashes correctly)
posts_dir = r"C:\Path\To\hugo\<project>\content\post"
attachments_dir = r"C:\Path\To\Obsidian\attachments"
static_images_dir = r"C:\Path\To\hugo\<project>\static\images"

def get_all_files(directory):
    file_paths = []
    for dirpath, dirnames, filenames in os.walk(directory):
        for filename in filenames:
            file_path = os.path.join(dirpath, filename)
            file_paths.append(file_path)
    return file_paths

# Step 1: Process each markdown file in the posts directory
for filename in get_all_files(posts_dir):
    if filename.endswith(".md"):
        filepath = os.path.join(posts_dir, filename)
        
        with open(filepath, "r", encoding="utf-8") as file:
            content = file.read()
        
        # Step 2: Find all image links in the format ![Image Description](/images/Pasted%20image%20...%20.png)
        images = re.findall(r'\[\[([^]]*\.png|[^]]*\.jpg)\]\]', content)
        
        # Step 3: Replace image links and ensure URLs are correctly formatted
        for image in images:
            # Prepare the Markdown-compatible link with %20 replacing spaces
            markdown_image = f"[Image Description](/images/{image.replace(' ', '%20')})"
            content = content.replace(f"[[{image}]]", markdown_image)
            
            # Step 4: Copy the image to the Hugo static/images directory if it exists
            image_source = os.path.join(attachments_dir, image)
            if os.path.exists(image_source):
                shutil.copy(image_source, static_images_dir)

        # Step 5: Write the updated content back to the markdown file
        with open(filepath, "w", encoding="utf-8") as file:
            file.write(content)

print("Markdown files processed and images copied successfully.")

Web Server

Hugo Server

My web server was setup with a docker file similar to the one below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


name: hugo

services:
  server:
    image: hugomods/hugo:exts-non-root
    command: server -D
    volumes:
      - ./blog:/src
      - ./hugo_cache:/tmp/hugo_cache
    ports:
      - 1313:1313
    restart: always

Comment server

For comments I added remark42 as a comment server following their installation guide was pretty straight forward. I need to add authorization in the future but this is also running on the same server Hugo. Unfortunately I was having trouble getting it working while keeping them on the same docker network and using hostname for connections. I don’t know that it’s actually supported that way. So I ended up using a reverse proxy to host the site and link it into Hugo through integration set in the theme. I still need to set up more authentication so folks can comment but have to sign in with some known account like Google or Facebook. Maybe that will be next.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


services:
  remark:
    image: ghcr.io/umputun/remark42:latest
    container_name: "remark42"
    hostname: "remark42"
    restart: always
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "5"
    ports:
      - "8080:8080"
    environment:
      - REMARK_URL=https://example.com
      - SECRET=hahanicetrythisisntmysecret
      - SITE=example
      - AUTH_ANON=true
    volumes:
      - ./var:/srv/var