Before the Second Brain, I Had to Tame the Inbox

· 3 min read

My goal is to build an AI-powered knowledge base that can instantly answer complex questions about our projects. But before that knowledge workspace can provide answers, it needs clean, reliable data. And a lot of that critical data starts its life as an email attachment, lost in the chaos of an inbox.

So I built an email agent to automatically file every inbound attachment into its correct folder. This wasn’t just about automation — it was the foundational step. If a document is misfiled, it’s invisible. For an AI that relies on finding the right information to chunk and analyze, “invisible” means “useless.”

My first version was a “pure AI” approach.

I gave the cheaper GPT-4o-mini a document summary and a list of 200+ possible folders, asking it to provide a shortlist of likely folders. Then I sent the shortlist and summary to the more powerful GPT-4o to make the final choice.

The result was a painful 37.5% accuracy rate.

I quickly realized the problem wasn’t the AI’s intelligence, but its context. A general-purpose model has no way of knowing our internal logic: the institutional knowledge that dictates why some contracts go into the construction loan folder instead of the main contracts folder.

It was trying to organize our library without knowing our unique Dewey Decimal System.

That failure was a reminder of what I’d always known: you cannot build a reliable knowledge system on an unreliable data foundation.

So we pivoted. We replaced the single, probabilistic guess with a deterministic, “Chain of Thought” engine:

  1. Trained classifier. First, a custom-trained model makes a simple, high-confidence call: what type of document is this? Report, contract, invoice, etc.
  2. Targeted extraction. Next, a focused LLM call extracts key metadata based on that type. Discipline, status, counterparty.
  3. Deterministic routing. Finally, simple rule-based code uses that structured data to build the exact folder path. No guesswork.

By breaking the problem down, we’re not just improving folder accuracy. We’re building a trustworthy, scalable data pipeline. Every piece of information lands in the right place, ready to be indexed, chunked, and embedded for the knowledge system to use.

The journey to build the “second brain” didn’t start with fancy RAG models. It started with the unglamorous, critical work of getting the data house in order.

The “failed” agent taught me that for any ambitious AI project, reliability at the source is the only foundation that matters.