The `index.md` hub pattern: organizing docs for scale and token budgets
You probably know the shape: a 3,000-line README, a sprawling `troubleshooting.md`, a runbook that quietly became a book.
One large file feels convenient at first because everything is "in one place." Then it starts working against you. People have trouble scanning it. Search returns too much noise. Contributors avoid editing it. Merge conflicts pile up. AI agents burn tokens loading irrelevant material just to find one small answer.
That last bullet changes the economics of your docs. A monolithic runbook is not merely inconvenient. It is expensive. Every token of DNS troubleshooting sitting in the prompt while the agent hunts for a database timeout is a token you pay for, a token that adds latency to prefill, and a token competing for the model's attention. Good structure is still an information architecture concern. It is also a cost control and an accuracy control.
The `index.md` hub pattern
Instead of one giant file, create a dedicated directory for a broad topic and put a short `index.md` inside it.
The `index.md` is not the deep dive. It is a hub.
Its job is to explain the topic at a high level, enumerate the subtopics, and link to smaller, focused spoke files. Each link gets a one-line description of what problem that file helps solve.
The pattern looks like this:
```text
/troubleshooting
index.md
auth-failures.md
database-timeouts.md
network-issues/
index.md
dns-resolution.md
```
`/troubleshooting/index.md` is the entry point. `auth-failures.md`, `database-timeouts.md`, and `/network-issues/` are where the detail lives. The hub links to those nearby files, and no procedure text leaks into the hub itself.
Why this helps agent context
An agent navigating this structure does something different from an agent handed a monolith:
1. Open the hub first, a small and cheap read.
2. See the subtopics that exist and what each covers.
3. Decide which file is relevant to the symptom at hand.
4. Load only that file into context.
That is a routing problem, not a reading problem. Routing is cheap. Stuffing the whole document into the prompt just in case is not a strategy. It is a token bill you have decided not to look at.
A concrete before and after
Say an agent is debugging a connection timeout on a Postgres-backed service. It needs maybe 15 lines of guidance: pool size, statement timeout, retry and backoff, and how to spot an idle-in-transaction leak.
Before, the only artifact is `troubleshooting.md`. The file is 4,200 lines long and covers auth, DNS, proxies, TLS certs, queue backpressure, disk pressure, and the last three years of incident retrospectives. That is roughly 30,000 to 60,000 tokens, and it enters the prompt in full whether or not the agent needs it.
After, the agent reads `/troubleshooting/index.md` at about 300 tokens, matches "requests hang, retry, or fail around database access" to `database-timeouts.md`, and loads that file at about 1,500 to 4,000 tokens.
| Shape | Tokens loaded | Typical outcome |
|---------------------------|-------------------------------------------------------------|-----------------------------------------------------------------------|
| Monolithic runbook | 30,000 to 60,000 | Full-payload prefill, noisy retrieval, slower first token |
| Hub plus spoke | 300 plus 1,500 to 4,000, total less than 4,300 | Direct hit on the relevant procedure |
That is on the order of a 90 percent reduction in per-step context tokens. The agent is reading only the material it actually needs, not skimming a wall of text for a needle.
How structure changes agent cost
Here is where the argument stops being about tidiness and starts being about money and error rates.
Per-step context shrinks by roughly an order of magnitude. Going from 30,000 to 60,000 tokens down to about 2,000 to 4,000 tokens is an 85 to 93 percent cut in what gets paid for and re-transmitted on that step. On a frontier model at, say, $3 per million input tokens, the monolithic load alone is $0.09 to $0.18 per read. The hub-and-spoke path costs about a penny.
Agents do not make one call. This is the part that turns a modest per-step saving into a real one. A single debugging task typically means several reasoning turns, a tool call and re-read after each result, a retry when a command fails, a branch into a subproblem such as "is this DNS, or is this the pool?", and increasingly a fan-out to multiple subagents that each get their own context window. If a task touches docs six times, the monolith costs you six full loads. The hub costs you one hub load plus six different targeted spokes, most of which are small.
Rough math on a 10-step agent run:
- Monolith: 10 times about 40,000 tokens is about 400,000 input tokens.
- Hub plus spokes: about 300 plus 10 times about 3,000 tokens is about 30,000 input tokens.
At context-dominated workloads, that is at least a 50 percent reduction in cost per task as a conservative, single-step-only estimate. The number grows larger once you account for the fact that the monolith gets re-read and re-truncated on every retry. Latency follows the same curve. Prefill time scales with prompt size, so the routed path also gets answers back faster.
Retrieval precision improves because noise goes down. Fewer irrelevant tokens mean less chance the model blends unrelated symptoms, fixes, or commands into its answer. Load the 40,000-token runbook and the agent has `iptables` flags, a Redis eviction anecdote, and a stale note about a retired load balancer all sitting in attention range of a Postgres question. Load `database-timeouts.md` and the only commands in context are the ones that could plausibly be correct.
Context bloat causes wrong-tool and hallucinated steps. Long prompts degrade instruction-following and encourage nearest-plausible-snippet behavior. The agent runs the command it half-remembers from three sections ago. Scoped files make that failure mode much harder to hit. Fewer unrelated procedures in the window mean fewer confident, wrong actions in the trace.
Pattern to outcome
| Dimension | Monolithic doc | `index.md` hub plus spokes |
|---------------------- |---------------------------------------------------------------------|---------------------------------------------------------------|
| Context loading | Whole file in every prompt | Small hub, then one routed spoke |
| Token usage | High and flat, driven by topic size | Focused, driven by the actual question |
| Latency | Full-payload prefill each step | Prefill 10 to 20 times smaller |
| Agent precision | Noise-prone; procedures contaminate one another | Scoped; symptom maps to one file |
| Cost per run | Expensive and hard to reason about | Cheaper and closer to linear in real work done |
| Prompt caching | Busts whenever anyone edits the megafile | Hubs stay short and stable, cache well |
| Maintainability | Hard; edit anxiety and merge conflicts | Incremental; one file per subtopic |
Beyond tokens: structure as system optimization
The token count is the most legible win, but it is not the only one. A hub-and-spoke doc tree improves the whole retrieval substrate.
Prompt caching efficiency. Caches are keyed on prefixes and die on edits. A short, rarely-changed `index.md` sitting at the front of a family of requests is close to an ideal cached prefix. It gets a high hit rate and stays cheap to renew through TTL. A 4,000-line runbook that someone touches in three places per week invalidates everything downstream of the change. Stable hubs, mutable spokes: that is the shape you want if you are paying for cache writes.
Search relevance from routing metadata. Directory names, filenames, and the one-line summaries in the hub are effectively structured metadata. `database-timeouts.md` with the description "Timeout, connection-pool, and slow-query symptoms" gives search, grep, embeddings, and the model a precise, low-dimensional signal to match a symptom against. A monolith gives search one hit that returns 4,200 lines.
Better model reasoning. Agents plan well when the problem-to-document mapping is explicit. "If requests hang around database access, read this file" is a decision the model can make reliably with 300 tokens of overhead. Making it infer the same routing from headings buried inside 40,000 tokens of prose is a much harder task, and a much more expensive one.
Easier evals. When each subtopic is a file, each class of issue becomes a measurable unit. You can instrument tokens loaded, retrieval accuracy such as whether the hub routed the agent to the right spoke, and cost per resolved issue. Then you can watch for regressions when a file grows past its usefulness or a hub description goes stale. You cannot get that signal out of one big document.
A simple example hub
Here is what a short, practical `index.md` looks like:
```md
# Troubleshooting
Use this hub to find the right guide for a common operational issue.
Start here if you are unsure where to look, then open only the file you need.
## Authentication
- [auth-failures.md](./auth-failures.md)
401/403 responses, expired tokens, session resets, RBAC denials. Use this if
users cannot sign in or API calls fail with auth-related status codes.
## Database Issues
- [database-timeouts.md](./database-timeouts.md)
Connection-pool exhaustion, statement_timeout, idle-in-transaction, slow-query
symptoms. Use this if requests hang, retry, or fail around database access.
## Network Issues
- [network-issues/](./network-issues/index.md)
DNS resolution, proxy, TLS, and firewall problems. Start at this sub-hub if the
issue looks transport-related rather than application-level.
```
That is about 300 tokens. Enough to route a human and an agent. Deliberately not enough to answer anything in detail.
Rules for writing hubs that agents can route on
The pattern only pays if the hub stays a hub. Concretely:
- Keep `index.md` to one screen. Target about 150 to 500 tokens. If it exceeds a screen, you have a spoke hiding inside it.
- Use descriptive filenames. `database-timeouts.md` is a routing signal. `stuff.md` and `troubleshooting-v2-final.md` are not.
- Summarize every link in one or two lines, symptom-first. Write the trigger, not the table of contents: "Use this if requests hang or retry around database access."
- Never put full procedures in the hub. No copy-paste blocks, no step lists, no shell commands. The moment the hub is executable, it is a spoke.
- Split nested topics when the hub starts absorbing detail. When a section grows past a handful of links, promote it to its own directory with its own `index.md`. The same entry point keeps working, and the new level stays cheap.
- Include the literal error strings. Paste the exact message into the file that handles it and reference it in the hub line. Messages such as `connection pool timeout after 30000ms`, `EAI_AGAIN`, and `FATAL: remaining connection slots are reserved` help users grep and help LLM routers match. The string is one of the strongest signals available.
- Treat the hub as an API surface. Renaming or moving a spoke is a breaking change. Keep paths stable, or leave a redirect stub.
The human benefits are the agent benefits
Discoverability, scalability, and maintainability are not a separate, softer story. They are the same lever pulled once.
A contributor who can open one small file without scrolling 4,000 lines writes more accurate, more current docs. Current docs are what keep an agent from confidently recommending a retired command. A topic that grows by adding files instead of adding lines keeps its hubs small, which is exactly what keeps per-step context cost flat as your surface area expands. A structure that resolves merge conflicts in one narrow file is a structure that stays correct enough to be routed on automatically.
Poor docs do not just annoy people. They show up as a line item, and they show up in the failure trace.
A pattern that compounds
Turn your biggest file into a hub and push the details outward. Start with whatever doc you have already had to apologize for.
The `index.md` hub pattern is simple, and it scales because it treats structure as part of the interface shared by three audiences at once.
Humans find things faster because they get a menu instead of a monolith. Contributors update docs safely because edits are narrow and reversible. Agents load only the context they need because routing is cheap and comprehension is scoped.
So the honest framing is not "nice organization for documentation." An `index.md` hub is an optimization layer for the readers and agents that share the same context surface. It buys you a smaller prompt cache footprint, a lower token bill, less latency, fewer misrouted steps, and measurably better retrieval precision. It also makes your docs easier to read and maintain, which is what lets the pattern keep paying off.
Have you used the `index.md` pattern in your repos? What folder structures do you prefer for documentation, internal runbooks, or agent context?