We just published a long deep dive on how we built the RAG framework behind Kubrius Core: hybrid retrieval, strict evidence gates, a 95%+ score on a 500-question benchmark, all running locally.
It's a proper engineering post, and it's long on purpose. But the actual lessons fit on one page.
Here they are, extracted and standalone, so you can bookmark them, share them, or just skim them without the full build log. For the benchmark numbers, the failed experiments, and the reasoning behind each rule, read the full piece.
1. Keep a clean baseline
Don't judge an experiment in isolation. Compare it against a known clean version.
2. Use hybrid retrieval by default
Dense search and BM25 solve different problems. Enterprise RAG needs both.
3. Treat BM25 as first-class infrastructure
It's not legacy. It's exact-match memory.
4. Preserve important evidence into answer context
If retrieval finds the right evidence but context packing drops it, the model never had a fair chance.
5. Retry only after safe failure
Don't disturb already-good answers. Unknown-only retries are much safer than global deeper retrieval.
6. Gate retries strictly
A retry answer shouldn't be accepted just because it isn't literally "I do not know."
7. Count verbose refusals as unknowns
"The provided context does not contain..." is an unknown. Your evaluator should treat it that way.
8. Be suspicious of aggregate questions
"How many?", "which customers?", and "across all" questions often need dedicated aggregation logic, not normal RAG generation.
9. More retrieval isn't always better
Deeper retrieval can find more documents and still make the final answer worse.
10. Never hardcode benchmark answers
If you do, you haven't built a better RAG gateway. You've built a very small theatre production.
Why this matters
None of these are exotic. They're the boring discipline that separates a RAG demo from something a business can actually rely on.
If you're building anything similar, steal these rules. If you'd rather not build it yourself, this is exactly the discipline behind Kubrius Core: local RAG and agentic workflows, built to this standard, running on your own infrastructure.
Not sure where AI control fits before you get to RAG at all? Start with why SMEs need an AI control layer before they need another AI tool.
Frequently Asked Questions
What's the single most important rule for enterprise RAG?
Gate retries strictly. It's tempting to let a second attempt through just because it isn't a literal "I do not know," but that's how confident wrong answers sneak past review. Everything else in the list supports that one discipline.
Why isn't more retrieval always better?
Deeper retrieval can pull in more documents without pulling in more relevant ones, and the extra noise in the context can push the model toward a worse answer than a smaller, cleaner set of evidence would have.
What counts as a "verbose refusal," and why track it separately?
A verbose refusal is an answer like "the provided context does not contain..." dressed up in more words. If your evaluator only checks for the literal phrase "I do not know," these slip through as if they were real answers. Counting them as unknowns keeps your accuracy numbers honest.
Is BM25 keyword search still relevant next to vector search?
Yes, treat it as first-class infrastructure rather than a legacy fallback. Vector search is built for meaning; BM25 is built for exact matches like IDs, names and codes. Enterprise questions need both.
How do I apply these principles without building a RAG system from scratch?
Kubrius Core is built to this standard already, running local RAG and agentic workflows on your own infrastructure. See Kubrius Core or read the full engineering deep dive for how each principle was arrived at.
See where Kubrius Core fits your business.
If your workflows involve confidential documents, proprietary code, or client files that shouldn't touch a public API, Kubrius Core is the private execution path built for exactly this.