What if AI agents could learn from each other's work? We built a Knowledge Bank system to find out, and the results exceeded our expectations: 57% context reduction, 100% file reading elimination for documented topics, and clear compound knowledge accumulation effects.
The Problem: Every Agent Starts from Scratch
When working with AI coding agents, we noticed a frustrating pattern: every agent would explore the same files, discover the same patterns, and learn the same lessons. There was no memory, no accumulation of knowledge, no compound learning effect.
For example, five different agents exploring our Docker architecture would each:
- Read the same 4 configuration files (~700 lines)
- Parse the same service definitions
- Discover the same gotchas (silent container failures, database routing, etc.)
- Use ~35,000 context tokens each
This was inefficient and expensive. We needed a system where Agent 2 could benefit from Agent 1's discoveries, and Agent 10 could benefit from all nine agents before it.
The Solution: A Semantic Knowledge Bank
We built a Knowledge Bank using PostgreSQL with the pgvector extension and OpenAI embeddings. The architecture is straightforward but powerful:
Core Components
Three Types of Knowledge
We categorize knowledge into three types, each stored with metadata for efficient retrieval:
- Conceptual: Architecture overviews, system explanations, workflows (metadata.type: NOT SET)
- Examples: Validated code patterns with proof (metadata.type: "example")
- Gotchas: Common mistakes and edge cases (metadata.type: "gotcha")
How It Works: Query Examples
The Knowledge Bank uses natural language queries across multiple dimensions. Here's what agents actually ask:
The Experiments: Four Agents, Two Topics
We ran controlled A/B tests to measure the compound learning effect. The results were dramatic:
Test 1: Docker Architecture (Fresh Topic)
| Metric | Agent 1 (Empty KB) | Agent 2 (With KB) | Improvement |
|---|---|---|---|
| Context Tokens | 35,000 | 15,000 |
-57%
|
| Files Read | 4 files | 0 files |
-100%
|
| KB Coverage | ~30% | 95% |
+217%
|
💡 Key Insight: Agent 2 achieved 95% knowledge coverage from KB queries alone, requiring ZERO file reads. This is the compound effect in action.
Key Findings
1. The Compound Effect is Real
With 100 agents on the same topic:
2. KB Saturation Happens Fast
It takes just 2 agents per topic to reach 95% KB coverage:
- Agent 1: Documents 60-70% (discovers fundamentals)
- Agent 2: Documents 20-30% (fills gaps)
- Agent 3+: Find KB complete, add minimal new knowledge
3. The Gotcha Feature: Learning from Mistakes
Gotchas have exceptional return on investment:
- Cost to capture: 2-5 minutes agent time
- Savings for next agent: 30-120 minutes debugging time
- ROI: 10-30x return on knowledge investment
Why Multi-Dimensional Search Matters
Most vector databases use single-dimensional semantic search—one embedding captures all the content. But agents ask questions from different perspectives: what something does, when to use it, which systems it affects. Our multi-dimensional approach creates separate embeddings for four different aspects, then searches across all of them simultaneously:
The Math: Vector Similarity in Action
Each knowledge entry and query is converted into embedding vectors (1536-dimensional arrays of numbers). Similarity is measured using cosine similarity, which calculates the angle between vectors:
For multi-dimensional search, we combine scores from each dimension using weighted averages:
(1.0 × useful_for_similarity) +
(0.3 × systems_similarity) +
(0.3 × tasks_similarity)
2.6 (total weight)
Real Example: Why Dimensions Matter
Let's see how multi-dimensional search finds the right knowledge even when queries don't match exactly:
💡 Why This Matters: The "useful_for" dimension scored 0.85 despite completely different wording ("debugging silent exits" vs "fixing crashes"). This is the power of semantic embeddings - they understand meaning, not just keywords. A single-dimension search would have missed this match entirely.
Conclusion
Building a Knowledge Bank for AI agents taught us that compound learning is not just possible—it's incredibly effective. With the right architecture and prompt engineering, we achieved:
- 57% context reduction for fresh topics
- 100% file reading elimination for documented topics
- 95%+ KB coverage after just 2 agents per topic
- 10-30x ROI on gotcha documentation
The future of AI-assisted development isn't just smarter agents—it's agents that learn from each other's work. The compound effect is real, and it's powerful.
Thanks for Reading This Far! 🤔
You probably thought of some challenges with this system: What about outdated knowledge? How do you prevent noise from low-quality entries? What about redundant or conflicting information? And the big one: how do you efficiently query across multiple vector spaces simultaneously with scalable response times as the KB grows to thousands of entries?
These are exactly the challenges we solved in production. Part 2 will cover:
Want to be notified when Part 2 drops? Reach out or join our mailing list to get notified. Or if you have questions or ideas about these challenges, we'd love to hear from you!