The Tour

You've heard the stories. The wind-down, the son with the FortiGate, the wife getting bullet-pointed by an unexpected butler, the SIM that hadn't arrived yet. Eleven weeks of building, told one moment at a time.

But I've never actually shown you what's running.

So this post is the tour. The map of the house. What's on the rack, how the pieces talk to each other, why I picked each one. Less story, more architecture. If the earlier posts were what it felt like to build, this one is what got built.

Read it like you're walking through someone's home for the first time. Some rooms you'll want to peek into; others you'll just nod and keep walking. There is an appendix at the bottom with a stack diagram and a tech-stack table for anyone who wants to look closer.

Let's go.

The shape of the thing

Tangerine Home runs on a single PC sitting under a desk in my living room. Modest hardware by data-centre standards, ample by family-of-three standards: an NVIDIA RTX PRO 4000 Blackwell GPU, enough memory to run a few language models at once, and a UPS humming away beside it for when the power flickers.

What I built on top of it is best understood as layers, not as one big AI app. Each layer does one thing well, exposes a small clean interface, and stays out of the others' business. When something breaks I can usually reach in and fix one layer without disturbing the rest.

There are roughly five of these layers, working from the inside out.

Layer 1: The model gateway

At the very centre is the part that decides which AI model handles what request.

Different jobs want different models. Long agentic workflows want a careful, expensive flagship model that won't lose the plot on the seventh tool call. Quick conversational replies want something fast and cheap. Local document classification wants something that runs on the GPU under the desk so private documents never leave the house.

Rather than wire each application to a specific model, I run LiteLLM as a gateway. Every model — OpenAI's flagship, the local GLM-4.7-Flash on my GPU, embedding models for memory, smaller models for digest jobs — sits behind a stable alias. jimi-model, email-local, memory-extract, and so on. Applications ask for the alias. The gateway routes the request.

The win is hidden but real: when I want to swap out the model behind the WhatsApp butler — which I've now done three times since launch — I edit one line in a config file. The applications don't care. They keep asking for jimi-model and the gateway points it somewhere new.

Layer 2: The tool servers

Above the gateway sit the tools. The actual capabilities of the system — read email, search the document archive, look up a password, check the calendar, store a memory, manage a task list.

Each capability lives in its own small server, written in Python, doing one thing. Email is one server. Photos is one server. Documents is one server. Memory is one server. Calendar, passwords, tasks, system health — each one is its own.

They all speak the same protocol: MCP, the Model Context Protocol that's become the standard for letting AI models call tools. From the model's point of view, asking the email server to list unread mail is exactly the same shape as asking the calendar server to find tomorrow's events. The interface is uniform; the work behind each tool is whatever it needs to be.

This is the part that took me longest to internalise. The temptation, when you're new to building with AI, is to put all the cleverness in one big application. The mistake, which I made twice, is that the resulting thing becomes impossible to maintain. The right move is the opposite: many small servers, each boring, each doing one thing, all composable. The intelligence is what uses the tools, not what's inside them.

Layer 3: The faces of the system

A system that nobody opens is a system that doesn't exist. So above the tool servers sit the faces — the surfaces through which a person actually meets the platform.

There are three so far.

Claude.ai is the face I use most. The same Claude interface anyone can sign up for — except mine is connected to my platform's tool servers as MCP connectors. When I'm planning a blog post, debugging a service, or running a system check, I'm doing it through Claude on my laptop, with the platform's own tools sitting one click away. From Claude's point of view it's calling tools. From my point of view it's a co-pilot that happens to know about my house. This is the power-user face.

Jimi is the face the rest of the family uses. The WhatsApp butler. He sits behind a regular WhatsApp number, picks up messages, asks the model gateway for the right model, and uses whichever tool servers he needs to answer. From the user's point of view he's just another contact. From the system's point of view he's the front door for ninety percent of family interactions, because WhatsApp is where family conversations already happen.

The web apps are the third face. There are several — a photo library, a document archive, a password vault, a task manager — each a self-hosted version of a tool you've probably used elsewhere. They run on the same server, behind the same Cloudflare tunnels, accessible from any phone or laptop with the right login. These are for the moments when chat isn't the right shape for the job. You don't text a butler "show me all family photos from 2019." You open the photo app.

Three faces, three audiences. Power user, family, anyone with a browser. Same platform underneath.

Layer 4: Memory, sessions, and state

Underneath the faces and beside the tools is the part that gives the system continuity.

Three different kinds of remembering, each with its own home. Personal memory — the slow-changing facts about who someone is, who their family is, what they like. Session logs — verbatim records of what was discussed and decided in a particular conversation. Platform facts — the system's knowledge about itself, like which port a service runs on or what the structure of a database looks like.

I learned the hard way that these three things look the same from a distance and behave very differently up close. Trying to handle them with one storage layer was the bug; splitting them into three was the fix.

For now, what matters is that every layer above this one — Claude, Jimi, the web apps, the digest jobs — can ask the memory layer "what do you know about this?" and get an answer that's actually useful. The system isn't just answering queries in isolation. It's accumulating context.

Layer 5: Access and the outside world

The last layer is how all of this reaches the outside world without getting overrun by it.

There are three doors, each for a different kind of visitor.

The front door is for me, working through Claude.ai. My platform's MCP servers are exposed via a Cloudflare tunnel at a single domain, with a small piece of network plumbing called Caddy sitting at the edge. Caddy looks at the per-service key on each request and routes it to the right tool server. From Claude's side it's just a list of connectors. From my side it's the same set of MCP servers Jimi uses, reachable from any browser I happen to be logged in on.

The family door is WhatsApp. Jimi's number is a regular WhatsApp number on a regular phone account. Family members don't need a new app, a new login, or a new mental model. They just text. The whole point of him is that the door doesn't look like a door.

The web app door is the same Cloudflare tunnel as the front door, but pointed at the human-facing apps — the photo library, the document archive, the task manager. Public URLs, individual logins, fully encrypted in transit. Reachable from any device, anywhere, with a browser.

Beside all three sits a private network — Tailscale — for the cases where someone needs direct access to an internal service that isn't meant to be public. Useful, but not the primary path. The primary path is one of the three doors above.

One front edge. Three doors. Each one locked separately, each one shaped to who's coming through it.

How it actually feels to use

Here's a real moment, end to end.

My wife sends a WhatsApp message to Jimi: "can you find the receipt for the air-con servicing last year?"

Jimi's WhatsApp gateway picks it up. It asks the model gateway for the right model. The model decides it needs to search the document archive, so it calls the document server's search tool. The document server queries its database, finds the right receipt, returns the metadata. The model, seeing the result, calls the document server again to get a download link. It composes a reply with the answer and the link. The reply goes back through the WhatsApp gateway and lands in my wife's chat thread.

Total time, from her sending the message to her getting the answer: about twelve seconds.

What she sees: she asked a question, she got an answer.

What's actually happening: five different services on a PC under my desk, talking to each other in a coordinated little dance, running on a model that may or may not be hosted on the same continent, retrieving a document that was scanned and filed weeks ago.

The whole point of the architecture is that none of that complexity reaches her. She just texts and gets an answer.

What's missing from this picture

Plenty.

I have a small metrics agent — Beszel — running on the server, shipping system stats to a dashboard so I can see CPU, memory, disk, and container health at a glance. What's missing is application-level observability — how many tool calls a given request used, whether a particular service is degrading, whether a model is misbehaving. The system tells me when a service is down. It doesn't yet tell me when a service is wrong.

It has no formal access control beyond per-user API keys — fine for three people in one household, insufficient if I were ever to share this with another family.

It has no automated backup verification. The backups run nightly but I haven't yet built the part that proves they'd actually restore.

It doesn't yet know how to handle Irene, my wife, in the way it handles me. Different person, different patterns, different vocabulary. The system was built around how I think, and that bias is going to surface every time someone who isn't me tries to use it. That's the next big piece of work, and probably the hardest.

What this is, and what it isn't

It is a working personal AI platform that three people use, that has survived eleven weeks of daily use and several rebuilds, that costs less to run than most cable subscriptions, and that does real work — answering questions, filing documents, managing tasks, remembering things — without any of my data leaving the house except when I explicitly choose to use a hosted model.

It isn't a startup. It isn't a product. It isn't something I'm trying to sell you.

It's a house I built. The blog is the record of building it. This post is the floor plan.

Next: everyone's building guardrails for what AI agents do. I started building something for how they think.

Appendix A: The stack at a glance

Appendix B: How the layers fit together

                    ┌─────────────────────────────────┐
                    │        OUTSIDE WORLD            │
                    │  (laptops, phones, WhatsApp)    │
                    └─────────────────────────────────┘
                                   │
        ┌──────────────────────────┼──────────────────────────┐
        │                          │                          │
   Claude.ai                  Web apps                    WhatsApp
   (power user)               (browser)                   (Jimi)
        │                          │                          │
        └──────────────────────────┼──────────────────────────┘
                                   │
                            ┌──────▼──────┐
                            │ Cloudflare  │
                            │   tunnels   │
                            └──────┬──────┘
                                   │
                            ┌──────▼──────┐
                            │    Caddy    │   ← edge router
                            └──────┬──────┘
                                   │
        ┌──────────────────────────┼──────────────────────────┐
        │                          │                          │
   ┌────▼────┐               ┌─────▼─────┐              ┌─────▼─────┐
   │ Web     │               │   Tool    │              │   Jimi    │
   │ Apps    │               │  Servers  │              │ (WhatsApp │
   │ (Ghost, │               │  (MCP)    │◄─────────────┤  butler)  │
   │ Immich, │               │           │              │           │
   │ etc.)   │               └─────┬─────┘              └─────┬─────┘
   └─────────┘                     │                          │
                                   │                          │
                            ┌──────▼──────┐                   │
                            │   LiteLLM   │◄──────────────────┘
                            │  (gateway)  │   ← model routing
                            └──────┬──────┘
                                   │
                  ┌────────────────┼────────────────┐
                  │                │                │
            ┌─────▼─────┐    ┌─────▼─────┐    ┌─────▼─────┐
            │  Hosted   │    │  Local    │    │  Memory   │
            │  models   │    │  GLM-4.7  │    │  layer    │
            │  (OpenAI) │    │  (Ollama) │    │ (custom   │
            │           │    │           │    │  pipeline │
            │           │    │           │    │ +pgvector)│
            └───────────┘    └───────────┘    └───────────┘