Local First AI and Agent Operations
This planned 14-part series follows the work of turning a local-first collection of AI models, agents, services, memory, and developer tools into an environment I can inspect, recover, and improve without hiding the hard parts behind another layer of automation.
Hermes Agent is the system I am operating throughout the series, so names such as Dashboard and Gateway refer to Hermes components unless I say otherwise. I do not expect every reader to use Hermes. What matters beyond this particular system are the boundaries and habits around it: clear ownership, OpenAI-compatible services, deterministic workflows, governed memory, passive observation, and tests run from the process that actually has to work.
The main installments appear in Technology. Runnable companion articles appear in Hands-On. The companions are optional: they sit beside the main sequence instead of interrupting it, so you can follow the engineering argument straight through or pause when you want to build the smaller working example.
Main Series
-
When a Local AI Stack Becomes an Operations System
The model may be running while the system around it is still broken. This is how a local-first AI experiment became an operations problem involving ownership, dependencies, readiness, recovery, and rollback.
-
Deterministic First: Building a Knowledge Intake Pipeline
Before a model can decide what matters, the system still has to identify what exists, what changed, where it came from, and whether a human decision has already been made. I moved those jobs into a deterministic intake pipeline.
-
Memory Is a Governance Problem, Not Just a Vector Database
A relevant memory from the wrong context is not a better result; it is a disclosure. I moved authorization ahead of ranking and kept durable authority separate from disposable retrieval state.
-
The Agent Memory and Optimization Tool Landscape
Memory, caching, compression, shell reduction, routing, and observability do different jobs. I stopped looking for one winner and built a correctness-first way to evaluate complementary layers.
-
Measuring Token Optimization Without Breaking the Agent
One test used fewer input tokens and still made the agent slower. That result changed how I measure optimization: preserve correctness, record the whole task, and keep the raw path available.
-
A Passive Model Proxy as an AI Debugging Instrument
A request can look correct while the rendered prompt is wrong. I built a passive model proxy to inspect the request, template output, and response without changing the evidence.
-
Multimodal Context Hygiene with a Jinja Chat Template
Old textual media results can consume context long after they stop helping the model. I moved that cleanup into a tested Qwen chat template while keeping the diagnostic proxy passive.
-
From Shell Scripts to an Operator-Ready LLM-Ops-Kit
The shell scripts worked, but they could not explain the system. I turned that useful experiment into a small control plane with one configuration model, inspectable plans, typed lifecycle boundaries, and recovery.
-
Voice Cloning Across Hosts: Making TTS Operational
A healthy speech bridge does not prove that the model behind it is ready. Part 9 follows the co-located bridge, registered references, runtime ownership, and recovery boundaries that made cross-host TTS operational.
-
Squeezing More Inference from Apple Silicon: llama.cpp Today, MLXForge Later
A slow response can hide model loading, queueing, prompt prefill, first-token delay, or decode. Part 10 separates those costs, documents the llama.cpp runtime used today, and sets the evidence boundary MLXForge must eventually meet.
Hands-On Companions
-
Hands-On: Build an Idempotent Intake Manifest
A first import is easy. I built and tested a bounded Python and SQLite manifest that keeps source identity and history intact across reruns, changes, disappearance, and restoration.
-
Hands-On: Build a Fail-Closed Memory Router
Authorization has to choose the candidate set before relevance ranking begins. This runnable Python router and its seven canaries make that ordering visible and testable.
-
Hands-On: Build and Test a Passive Proxy Lab
Run a passive proxy, deterministic fake model endpoint, and six privacy and byte-preservation canaries using only the Python standard library.
-
Hands-On: Put Multimodal Context Hygiene Through a Test Bench
Run sanitized conversations through stock and derived Qwen templates, verify exactly what changes, exercise the routing boundary, and inspect the passive model-proxy relationship.
-
Hands-On: Put a TTS Bridge Through a Model-Free Test Bench
Build a model-free TTS test bench with bounded capability and registry metadata, opaque registered references, a legacy compatibility path, layered health, redacted events, and deterministic cleanup.
-
Hands-On: Build a Fail-Closed Inference Benchmark Comparator
Build a standard-library benchmark comparator that rejects incompatible runs and correctness drift before calculating median, direction-aware performance deltas.
Across the Series
Later installments also cover measuring token optimization, passive model-proxy instrumentation, multimodal context hygiene, operator-ready packaging, cross-host voice services, Apple Silicon inference measurement, cache and quantization choices, speculative decoding and MTP, installable skills, safe multi-agent operation on one LAN, and an honest public-release readiness assessment.