rainforest canopy layer top view
← Back to blog

Building an AI Server We Actually Own: What Fourteen Months Taught Us

Company News•28 October 2026•TREEO Indonesia•3 min read

A hosted AI assistant would have been faster to adopt. TREEO built its own instead — a router in front of several models, a library of tools they can use, a document memory, and a research engine that returns cited documents rather than chat — and fourteen months later, the reasons for building rather than buying have held up.

The system today

193

Python modules

37,720

lines of code

690

commits

22

written architecture designs

37

agent tools available

186

test files

Three Reasons Not to Buy

Step by step

  • Cost

    Several models sit on a ladder, cheapest first. Work escalates only when a cheaper model fails — never because an answer merely looked weak. Usage is metered per run and reported in the output.

  • Access

    It reads our own systems — logs, documents, mail and calendar, data pipelines. A hosted product cannot be given that kind of access, and should not be.

  • Control of data

    Conversations, documents and memory stay on infrastructure we run, and each person's are kept apart from everyone else's by the system refusing to look outside their own pool.

Two Systems, One Boundary

The clearest architectural decision was splitting the product in two and keeping the boundary honest: the server owns models, tools and data; the workspace layer owns the desktop experience and each person's own files. Neither reaches into the other.

Identity lives somewhere else, on purpose. Who someone is, and what they may use, is decided by a separate identity system — not by the AI server itself. The server validates a signed token and derives the person's role, their tool permissions, and their private data namespace from it. Adding a colleague became an administrative act rather than an engineering one, and every data boundary in the system traces back to a single function rather than being scattered across the codebase.

The same tool loop serves every surface the system has — desktop app, team chat, developer tools, internal web apps. Each surface differs only in what it's allowed to do. That took a deliberate rewrite: before it, every surface had its own loop, and its own subtly different bugs.

A Ladder, Not a Model

The single most effective cost decision was refusing to pick one model. Requests enter at the cheapest tier and move up only on failure — never because an answer merely looked weak. A tier is repointed at a different vendor by changing one setting; the provider adapters are built as swappable plugins, not hardcoded choices.

Two rules took real incidents to learn. Escalation is failure-driven, never quality-driven — "this answer seems weak, try a better model" is an unbounded budget, because a failure is a fact and a judgement of quality is not. And a pinned tier is locked: if a caller asks for a specific tier, the router waits for that backend to recover rather than quietly answering from a different model. Silent substitution turned out to be worse than a slow answer.

Access is gated by role, not by what a request claims about itself. Each role carries its own tool allowlist, gated to that role and that specific surface. A tool that's unavailable says so, and says why — an earlier version failed opaquely, and the model filled the silence by inventing capabilities it didn't actually have.

Parallel Research That Returns Documents

Most AI tools answer a question in one go. This system takes a brief, splits it into research workstreams, runs them in parallel, verifies the results, retries what failed, and delivers a folder of cited documents.

The decision that makes it affordable: no model decides to search. A separate layer gathers sources — prior findings, the document store, web search, a fallback path — and the model only reads what it's given. An earlier, more agentic version paid for the tool catalogue, the search decision, the round trip, and the result re-entering context on every single round, and cost roughly ten times as much for the same output.

A real twelve-workstream run measured 8 minutes wall clock, 13 model calls, 195,849 tokens, and 11 documents delivered — about 110,000 characters, each with citations. Each workstream reads roughly six sources and writes once; it's deliberately not a multi-hour, hundreds-of-sources research agent, because that's a different cost profile the team switched off on purpose. Being explicit about that scope prevented a lot of disappointed expectations.

Three things about the design are worth copying into any similar system: the plan is rendered from the actual run state rather than written by a model, so it can never disagree with what really happened; verification runs as plain code — required fields, valid links, duplicates collapsed — rather than a second AI opinion; and each parallel worker sees only its own task and its declared dependencies, never the whole run, because shared mutable state across parallel branches is a reliable bug generator.

Six Lessons, Paid For

Step by step

  • Write the design down, and the wrap-up afterwards

    Twenty-two designs and a dozen wrap-ups. Designs record intent; wrap-ups record what actually happened, including conclusions that later work reversed. The wrap-ups get consulted far more often.

  • ️ Never hardcode a model name anywhere

    Not in docs, not in client configs, not in advertised identifiers. Clients persist whatever name they're shown, then quietly pin themselves to a model that's since been retired.

  • Documentation drifts faster than anyone believes

    A core reference file claimed 17 tools where 37 were actually registered, and described a model ladder that hadn't existed for months. A script now fails the build when the numbers drift — it has caught its own author twice.

  • Ask what the conversation already contains

    A recurring bug came from asking "does this look like a research request?" without asking "what does this chat already hold?" A follow-up question then started a whole new research run — including one where the user had explicitly said not to.

  • A plugin that loads is not a plugin that works

    One feature was hooked onto the wrong extension point — it wrapped a call that streams, received a stream object, and never saw any text. It logged "loaded" on every start and did nothing for days.

  • ️ Cheap and deterministic beats clever, until measured otherwise

    Every time a model decision was replaced with ordinary code — verification, source gathering, plan rendering — the system got cheaper, faster, and easier to debug, and lost nothing a user ever noticed.

Frequently Asked Questions

Three reasons held up over fourteen months: cost control through tiered model routing that only escalates on failure, access to internal systems a hosted product can't safely be given, and keeping data and per-person isolation on infrastructure the team controls directly.

We're extending this toward spatial compliance work

The same platform behind this post is what we're now extending into geospatial reasoning for EUDR and carbon dMRV. Read where that's headed, or get in touch if you're solving a similar problem.

Building an In-House AI Agent Platform: Lessons Learned | TREEO Indonesia