Day 4 · Fri, Oct 2, 2026

RAG with Hermes

Two-panel recap: Day 3 covered LLM routing between models; Day 4's goal is grounding Hermes in real data with RAG since routing alone gives no model new knowledge

Day 3 taught Hermes to pick the right model for a task, but routing alone doesn't give any model new knowledge — every model still only knows what it was trained on. Today closes that gap with Retrieval-Augmented Generation: standing up a vector store loaded with real documents, wiring it into Hermes as a tool, and testing both a grounded question and one the documents don't cover.

1 / 10

A local LLM only knows what it was trained on — ask it about a document it's never seen and it will either say so or, worse, confidently guess. Today grounds Hermes Agent in real data instead: standing up a vector store, feeding it real documents, and connecting Hermes to it as a tool it can query before answering.

Objectives

Stand up a standalone vector store (ChromaDB) and load it with real documents
Wire retrieval into Hermes as a tool it can call, not a manual copy-paste step
Ask Hermes questions that require the retrieved context to answer correctly
Deliberately test a question the documents don't cover, to see how Hermes handles it

Lab: ground Hermes in a real document set

With a ChromaDB instance already loaded with course material (following the same setup as our earlier ChromaDB RAG walkthrough), register it as an MCP tool Hermes can call, then ask it a grounded question:

$ hermes mcp add chroma-rag --command uvx --args mcp-server-chroma

$ hermes -z "Using the retrieval tool, what does our course material say about MCP servers? Cite what you found." --cli --yolo

Before running this live:confirm which document set is loaded into the vector store — pointing the RAG tool at the wrong collection produces a confidently wrong answer instead of an honest "I don't know," which is worse for teaching than no RAG at all.

Session notes: to be filled in after the live session on Oct 2, 2026 — document set used, the real grounded answer we got, and what happened on the out-of-scope question.