The retrieval engine

This is the part of specd that makes the specs worth trusting. It is a knowledge graph, not just a vector store, and every stage is bounded so a prompt cannot quietly fill with noise.

Stage 1 · Rank fusion

Reciprocal Rank Fusion over two arms: vector similarity in pgvector, and Postgres full-text search over a generated tsvector. Merging by rank rather than by score is what lets two incomparable scoring systems vote together.

• Headings outrank body text — a match in a section title is a stronger signal about what a passage is about.
• One document cannot take every slot, so a long document does not crowd out the corpus.

Stage 2 · One hop across the graph

The seed documents are expanded one hop across resolved links — the five deterministic edge kinds extracted at index time. The expansion is edge-kind weighted, hub-gated (so a hub document does not pull in everything), query-ranked, and budgeted at four additions.

Stage 3 · Real source code

Up to two code snippets: the actual source of symbols the seed documents reference, read from the repository at retrieval time, fenced in the prompt, and citable as path#Class.method. The doc says what the code is for; the code says what it does.

Embeddings, and the honest ceiling

The default embedder is a deterministic local hash — no second API key, works offline. The README-level truth about it is that it is lexical: the full-text arm carries relevance until you point the index at a real model.

SPECD_EMBEDDING_PROVIDER=voyage   VOYAGE_API_KEY=...

# or any OpenAI-compatible /v1/embeddings endpoint —
# Ollama, LM Studio, llama.cpp, vLLM
SPECD_EMBEDDING_PROVIDER=openai   SPECD_EMBEDDING_BASE_URL=http://localhost:11434/v1

The second option is the interesting one: the ceiling comes off without a cloud key and without a repository's knowledge leaving the machine. Misconfiguring either fails loudly rather than degrading silently — the API probes the endpoint at startup and refuses a dimension mismatch by name.

The honesty rules

A retrieval engine that overstates what it found produces specs that overstate what they know. Three rules stop that:

Truncation is announced only when real matches were cut

A notice that fires on every request is a notice nobody reads.

Freshness says unmeasured, not fresh

A document with no commit date has an unknown age. Reporting it as fresh would be inventing a fact.

unknown is not unsupported

The corpus failing to answer and the corpus answering "no" are different results, and only one of them is evidence.

Health as numbers the UI can badge

Broken links, dangling anchors, orphans and stale code references are counted, and they move the score. They are not advisory notes on a page nobody opens — they are the metric that tells you when the corpus your specs rest on has started to rot.