The shape of an AI architect

An AI architect doesn't just write prompts. They model the whole stack: where the data lives, how retrieval happens, what the model decides, and where humans stay in the loop. That is the difference between a demo that wins a meeting and a system that survives real users.

If you are still deciding whether you need this role at all, I wrote a separate piece on what an AI architect is and when you actually need one. This one is about the work itself: the seven decisions I make in the first days of almost every project, before a line of feature code gets written.

1. What is the one job to be done?

Most AI projects fail because they try to be impressive instead of useful. Before anything else, I find the single workflow where AI moves a number the business already tracks. One job, one metric. A support team drowning in tickets. A sales team losing hours to research. A content pipeline stuck in review. Everything else waits until that one thing works.

2. Build, buy, or skip?

The most valuable sentence an architect says early is "you don't need to build this." If an off-the-shelf tool does 80% of the job, we use it and spend the budget where it actually matters. Custom only earns its keep when it moves a metric a tool can't reach. Saying this out loud has cost me projects. It has also earned me every long-term client I have.

3. Retrieval, fine-tuning, or neither?

Teams reach for fine-tuning when they mean retrieval. If the model needs to know your facts, that is almost always retrieval, not training. Fine-tuning changes behavior and tone, not knowledge, and it is expensive to maintain. Most products in 2026 need good retrieval and a sharp prompt, nothing more.

4. Where do the guardrails live?

A model will eventually produce something wrong, off-brand, or unsafe. The decision is not whether that happens but where you catch it: input validation, output checks, a moderation pass, or a human gate. I decide this on day one, because retrofitting guardrails into a shipped system is painful and usually means a rebuild.

5. How will we know it works?

If you can't measure quality, you can't improve it, and you certainly can't trust it. I build a small eval set early: real inputs, expected behavior, and a baseline number. Then every change is measured against it. "It feels better" is not a launch criterion, and any architect who offers it as one is guessing.

6. Where do humans stay in the loop?

The best AI systems are honest about uncertainty. They cite sources, surface confidence, and escalate to a person at the right threshold. That handoff is a design decision, not an afterthought. Get it right and users trust the system even when the model is unsure.

7. Who runs this after launch?

An AI feature is not done when it ships. Someone has to monitor it, watch cost, and catch drift as the world changes around it. I design for that from the start with clean code, observability, and a runbook, so the team owns it without me. If only the architect can keep it alive, the architecture failed.

The thread that connects all seven

None of these decisions are about the model. They are about the system around it. That is the whole job, and it is why the same prompt can be a toy in one product and load-bearing infrastructure in another.

This is exactly the thinking behind the AI Audit & Roadmap I run at the start of most engagements. You can poke at the systems it produces in the AI Lab, or book a 15-minute call and tell me about the one job you would want AI to do.