Docs
Setup notes and command logs from building out our local AI infrastructure.
2026-08-05
5 Simple Tricks for Running Hermes Agent Against a Local LLM
A practical how-to distilled from the two earlier live Hermes Agent sessions: point it at any OpenAI-compatible local endpoint, use one-shot mode and --yolo deliberately, check the chat_format before trusting any tool-calling result, and register MCP servers with absolute paths to dodge a real PATH bug.
2026-08-03
10 Things to Demo: Hermes Agent CLI on DeepSeek V4
Live-tested against the real jay-z820 + DeepSeek V4 setup: web search and code execution worked and were independently verified, while MCP tool calling, skills, memory, and delegation all traced back to one shared chat-template gotcha — including one case where the model confidently claimed to save a memory that was never written.
2026-08-02
DeepSeek V4 Flash 0731 — Quick Summary
A discussion with Sanjay on DeepSeek V4 Flash's launch pricing, the wider market shakeup it's causing among providers, and a live local deployment on a single 128GB-VRAM GPU with llama.cpp.
2026-07-27
Adding a Hermes Agent Alongside Open WebUI
Running NousResearch's Hermes-3-Llama-3.2-3B locally for real tool-calling — VRAM constraints, a chat-template gotcha that silently broke tool calls, and a temperature fix, wired into the existing mcpo MCP bridge.
2026-07-24
Standalone ChromaDB for RAG in Open WebUI
Running ChromaDB as its own server instead of Open WebUI's embedded store, reconfiguring Open WebUI to talk to it over HTTP, and proving retrieval by querying the vector DB directly with three sample docs (.txt, .md, .pdf).
2026-07-21
RAG Proof-of-Concept in Open WebUI
A five-minute proof that Open WebUI's built-in RAG actually retrieves from an uploaded document — a knowledge collection, a deliberately absurd test claim, and a before/after chat comparison with citations.
2026-07-17
45-Minute Class Workbook: Open WebUI Configuration
A run-of-show pulling from the admin guide and MCP/tools docs: talk for 2 minutes on a config, make the change live, show the behavior change. Six segments, timed.
2026-07-16
20 Things to Administer in Open WebUI
Controls panel Advanced Params (temperature, top_p, mirostat, and more) plus a tour of Settings, Admin Panel, and every sidebar shortcut, with a one-liner example for each.
2026-07-16
Tool Calling & MCP in Open WebUI
Live-in-class walkthrough: a native Open WebUI tool, then a real MCP server bridged in via mcpo, including every gotcha hit along the way.
2026-07-14
llama.cpp + Open WebUI Setup
GPU-accelerated local inference with llama.cpp and CUDA, fronted by Open WebUI, reachable remotely via Tailscale and SSH.
2026-07-14
HP Laptop: llama.cpp + CUDA Setup
Downloading a Ministral 3B model, rebuilding llama.cpp with CUDA, and tuning GPU layer offload to fit laptop VRAM.
2026-07-15
HP Workstation Deployment (Part 2)
Adding a second, isolated Open WebUI plus two more models (Gemma 3 4B, Qwen2.5 3B) — with every mistake and lesson learned along the way.