The last article stopped at the deployment gate. On October 11, the Pages workflow for the latest series push completed successfully, and the public Part 8 page returned HTTP 200 when I checked it. Good. That tells me the build and deployment finished and that one page answered one request. It does not tell me whether every old short link still reaches the right article, whether an image is missing, or whether an outside source I linked last month has disappeared.

That is the uncomfortable bit about a static site. There may be no application process to keep alive, but the published result still has moving parts: hosting, DNS, redirects, linked assets, and other people’s websites. Once the page leaves my build directory, I need to look at what a reader actually receives.

Ask the Served Site, Not Just the Source Tree

The most direct utility here is utils/bin/site_crawl_check.py. It fetches the site’s sitemap.xml, checks each same-host page listed there, then follows the same-host links and images it finds in those pages. It reports a failed fetch with the source page that exposed it. The script takes an explicit base URL, so I can point it at the public site when I choose to run a live check:

python3 utils/bin/site_crawl_check.py --base-url https://unixwzrd.ai

That is a real HTTP check, not another reading of Markdown. If an image path has the wrong case, or a link points to a route Pages did not emit, the response is the evidence. It also has a sensible boundary: the crawler stays on the supplied host. It does not audit third-party pages or images, and it does not recursively parse every page reached through a link unless that page is in the sitemap. A clean report means the requests it made succeeded, not that every possible route on the site was exercised.

Redirects need an extra question. The crawler follows them and accepts the final response if it is below HTTP 400. That catches a redirect ending in a 404. It does not verify where a successful redirect lands. An old article URL that quietly ends up on the home page can return 200 and still be wrong. I keep the source and generated redirect checks from Part 8 for the URL contract, and I spot-check important public redirects after deployment. Status and destination are different assertions.

A deployed site is observed through a served-site sitemap crawler and a separate source-derived reliability monitor. Both produce evidence for a human investigation; neither is shown as an automatic Pages gate.
Open full-size diagram

The build and the public site answer different questions. Monitoring only happens when something invokes it.

The editable Graphviz source is available with the diagram.

observation-boundaries.dot text View source
Download observation-boundaries.dot 
digraph ObservationBoundaries {
  graph [bgcolor="#111923", pad="0.38", rankdir="TB", nodesep="0.42", ranksep="0.62", fontname="Helvetica", fontcolor="#edf2f5", label="What happens after the build?", labelloc="t", fontsize=20];
  node [shape=box, style="rounded,filled", color="#587080", fillcolor="#20303d", fontcolor="#f4f5f6", fontname="Helvetica", fontsize=12, margin="0.19,0.13"];
  edge [color="#9db1bc", arrowsize=0.7, penwidth=1.5];
  source [label="Source tree\nposts, metadata, assets", fillcolor="#513d28", color="#d4a660"];
  build [label="Pages build and deploy\nartifact accepted", fillcolor="#263958", color="#80a6d7"];
  public [label="Served public site\nreader-facing HTTP", fillcolor="#263958", color="#80a6d7"];
  crawl [label="Sitemap crawler\nserved same-host links + images"];
  monitor [label="Reliability monitor\nsource-derived pages + external links"];
  review [label="Investigate results\nstatus, destination, context", fillcolor="#234243", color="#73b8b5"];
  { rank=same; crawl; monitor; }
  source -> build -> public;
  public -> crawl [style=dashed, color="#d4a660", label="when run", fontcolor="#d4a660"];
  public -> monitor [style=dashed, color="#d4a660", label="when run", fontcolor="#d4a660"];
  crawl -> review;
  monitor -> review;
}

The Larger Monitor Has a Different View

The repository also has utils/bin/site_reliability_monitor.py. It starts with site availability, checks a list of critical pages, probes images on those pages, and checks external links extracted from Markdown source. It can keep link metrics in its local configuration and send email through configured SMTP settings in post-commit or periodic modes. The site mode skips external links for a quicker pass. These are useful pieces, but they are not the same thing as crawling the rendered site.

Its critical-page list combines configured paths with paths it derives from the checkout. The discovery covers top-level pages and project posts, including their legacy redirects. It does not discover the nested series posts as a complete set. It can also encounter source for a page that has not been published yet. The two current unit tests exercise project-route and redirect discovery; they do not prove a successful remote sweep or an alert arriving in an inbox. Even constructing the monitor can save an updated local configuration, and the external-link pass writes metrics. I treat a run as an operational action, not a read-only audit.

The names in that directory can overpromise if I read them too quickly. The monitor’s check_links setting controls its external Markdown-link check. It does not turn run_health_checks into a rendered internal-link crawl. The separate sitemap crawler does that narrower served-site job. There is also a page-management helper whose missing-page and removal commands call methods the monitor class does not currently provide. I am not going to tell readers they have an automatic missing-page cleanup system when that path cannot run as advertised.

A Script Is Not a Schedule

There are shell wrappers for periodic checks and post-commit checks, plus a script that can install crontab entries. Those are ways to invoke monitoring. The Pages workflow in this repository does not call either monitor or the crawler, and it has no scheduled health-check job. The existence of periodic_monitor.sh tells me what would run if a schedule called it; it does not tell me that a schedule is active, or that mail delivery is configured and working.

The distinction is especially sharp in post-commit mode. Some of the older guide text says it waits for deployment. The current method calls the health checks immediately. A local commit is not the moment GitHub Pages finishes publishing, and the shell wrapper does not wait for that workflow either. If I want to verify a release, I first wait for the Pages deployment result, then check the public routes and their destinations. Calling the monitor immediately after a commit could inspect the previous deployment and label it with the new commit hash.

I keep logs and notifications in the same category: evidence only if the job actually ran and the message actually arrived. The monitoring code has file logging and SMTP paths, but I cannot infer either a reliable schedule or successful notification from configuration alone. A failed outside link also needs judgment. It might be a removed page, a rate limit, a temporary outage, or a site that blocks automated requests. I do not rewrite an article because one probe got an unhelpful response.

Try the Public-Side Question Locally

The Hands-On companion serves invented pages on loopback and tests three cases: healthy output, a missing image, and an old URL that redirects to the wrong page. It uses only Python’s standard library. The third case is deliberate: a status-only check sees HTTP 200, while a destination check sees the mistake. Nothing in the exercise touches this site’s monitor configuration, email settings, or public host.

Current State

I have two useful but separate tools: a served-site sitemap crawler for public pages and their same-host links and images, and a source-aware monitor for selected pages, images, and Markdown external links. The repository also has invocation and scheduling scripts. They do not become an active safety net merely by existing in utils/bin, and neither tool is part of the current Pages deployment job. The limits matter because a clean build, a clean crawl, and a correct redirect are three different claims.

Next Work

The final installment will put these checks back into the full publishing loop: deciding what to write, previewing and proofreading it, reviewing what the tools changed, choosing when to push, and checking what readers can reach afterward. I want that loop to be dependable without pretending the site has become a CMS that makes editorial decisions for me.