Anyone can generate an architecture now. That was never the hard part.

What a weekend-sized tool tells you about where the value in system design has moved, and why the expensive half is the part nobody automated.

Someone published a small command-line tool recently. You describe a product in plain English and it hands you back a system architecture: the components, the tradeoffs it considered, the assumptions it made. A few hundred lines of Python around a language model. Written, by the look of it, in a weekend.

My first reaction was not the interesting one. My first reaction was to check what it got wrong.

The second reaction was better, and it is what this piece is about. Because that tool existing at all is a fact about our industry, and the fact is this: producing a credible-looking system design has become nearly free. Not good. Credible-looking. Those are different things, and the distance between them is where all the remaining money is.

The design that appears in ninety seconds

Imagine a founder called Rachel. She has a product idea, no architect, and a board meeting on Thursday.

She types four sentences about what she wants to build. Ninety seconds later she has a document. It has a component diagram. It names a message queue and explains why. It lists three tradeoffs and picks one, with reasons. It flags two assumptions it had to make about her data volumes.

Ten years ago that document was two weeks and a consultant. Rachel now has it before her coffee cools.

Here is the thing worth sitting with. That document is not obviously bad. It is not the confident nonsense people expect from these tools. It is a competent first-pass design of the kind a mid-level engineer would produce on a decent day. For a large class of ordinary systems it is roughly right.

So the honest position is not that these tools are useless. It is that they have taken a task which used to be a bottleneck and made it cost almost nothing.

The teaching: "When something that used to be scarce becomes free, the value does not disappear. It moves somewhere else, and usually to the step immediately after it."

The one thing that tool does that most do not

There is a moment in using it that I did not expect.

Give it a description that is too thin and it does not produce a design. It comes back and asks you what you actually meant. What is the scale. Who uses this. What happens when it fails.

That sounds small. It is the most sophisticated thing in the whole project.

Almost every AI tool is built to always produce an answer. Emptiness feels like failure, so the system fills the space. Ask a vague question, get a confident generic response that looks like an answer and functions as a distraction. A tool that says your question is underspecified and here is what is missing is doing something genuinely harder, because it has to model the shape of a good question rather than the shape of a good-looking output.

That instinct is worth stealing, and I mean stealing it as a habit rather than as code. In your own work, the moment you notice you have given a machine an underspecified question, the useful move is not to accept what comes back. It is to answer the question it should have asked you.

The teaching: "A tool that refuses to answer a bad question is more valuable than one that always answers. So is a colleague."

The failure mode that survives review

Now imagine Rachel takes her ninety-second document into Thursday's board meeting.

Nobody in the room can tell. It has the right shape. It uses the right words. It reasons in the register of someone who has done this before. The one person who might have caught the problem is the engineer who left in March.

Six months later the thing is built and it does not hold. Not because the design was nonsense, but because of one choice: a queue where the ordering guarantee was wrong for their case, or a data model that made a later regulatory requirement almost impossible to meet. One decision, made confidently, in a document nobody was equipped to challenge.

This is the actual risk with generated designs, and it is not the one people worry about. Everybody worries about obvious hallucination — the invented library, the API that does not exist. Those are cheap failures. You find them in an hour, because the code does not run.

The expensive failure is a design that is plausible everywhere and wrong in one place. It passes review because it reads like competence. It costs six months because you only discover the flaw when you meet the condition the model never considered.

The teaching: "The dangerous output is not the one that is obviously wrong. It is the one that is right about nine things and quietly wrong about the tenth."

What the research says about people like Rachel

This is not a hunch. It has been measured, and the number is unkind.

Researchers at Harvard Business School ran a field experiment with the Boston Consulting Group, involving 758 consultants doing realistic work. On tasks that sat inside the AI's capability, the quality of what people produced went up substantially. On tasks that sat outside it, consultants using AI performed meaningfully worse than colleagues doing the same work without it.

Same tool. Same people. The task decided the outcome.

The researchers named the mechanism: mis-calibrated trust. People leaned hardest on the tool exactly where it was weakest, because nothing in the output distinguishes the two situations. The frontier of what these systems do well is jagged and unpredictable, and it looks identical from the outside whichever side of it you are standing on.

The finding I keep coming back to is a quieter one from the same work. The people who got the best results were not the ones who delegated the most. They were the ones who knew which parts to hand over and which parts to keep.

The teaching: "The skill is not using the tool. It is knowing where the tool stops being reliable, and checking hardest right there."

Why nobody automated the judging

There is a reason the weekend project generates designs rather than reviewing them.

Generating is a closed problem. You take a description and produce a plausible artefact. The output is judged on whether it reads correctly, and language models are extraordinarily good at producing text that reads correctly.

Judging is open. To review a design properly you need things that are not in the document: what the organisation is actually capable of operating, which of these components someone will still understand in two years, what the regulator will ask for, which constraint is real and which is habit dressed up as a constraint. You need to know what is missing, and absence is precisely what a text generator cannot see.

That is why the review half has not been automated, and why I do not think it is about to be. It is not a harder version of the same task. It is a different task that happens to arrive in the same file format.

The teaching: "Generating is a question about the document. Judging is a question about the organisation the document has to survive."

The engineering answer: give the model a smaller job

There is a fix for a lot of this, and it is old engineering rather than anything new.

Software has always separated two kinds of work. Some of it has a right answer: validating an input, applying a constraint, filling a default, handling a known error. We have spent fifty years learning to put that kind of work somewhere it cannot fail quietly — type systems, schemas, validation layers, tests that run before anything ships. The other kind has no single right answer: weighing one pattern against another, judging which trade-off suits this situation, noticing what looks risky.

The mistake almost everybody is making with language models is handing them both kinds at once.

Put the work with a right answer in code, where a rule holds every time and breaks loudly when it cannot. Leave the model the work that genuinely requires judgement. Not because the model is untrustworthy, but because a guarantee and an opinion are different things, and only one of them can be enforced.

That principle cuts against both of the popular positions.

Against the maximalists. The common instinct when a model gets something wrong is to write a better prompt. You ask it more nicely to validate its own output, to check its own constraints, to not invent things. This almost works, which is what makes it dangerous. You are asking a probabilistic system to provide a deterministic guarantee, and it will comply most of the time, which is precisely the failure profile that produces a six-month surprise. If a rule must always hold, it belongs in code, where it holds every time and fails loudly when it cannot.

Against the refusers. The opposite reaction is to conclude these systems cannot be trusted with anything serious. That throws away the half where they genuinely help. Weighing a dozen architectural patterns against a described situation is real work, and a model does it quickly and often well. The problem was never that it reasons. The problem is that it reasons and validates in the same breath, in the same voice, with the same confidence.

So the useful question stops being how much can I trust it and becomes which parts of this job have a right answer. Anything with a right answer should be checked by something that cannot be persuaded. Everything else is where the model earns its place.

There is a limit to this, and it is the part that matters most to me. Software can verify that a design is well-formed — that the constraints hold, the required fields exist, the output matches its contract. It cannot verify that the design is wise. Whether this is worth building, whether your team can operate it, whether the regulator will accept it: no contract check reaches those. That is the layer above the code, and it is still a person's job.

The same discipline applies to a roadmap, not just an architecture. Settle where the boundary sits between what is enforced and what is judged, before adding agents, retrieval, or anything else that widens what the system can reach. Most teams do the reverse: add the capability first, and work out the boundaries after something goes wrong in front of a customer.

The teaching: "Do not ask a probabilistic system for a deterministic guarantee. If a rule must always hold, put it somewhere that cannot be talked out of it."

Independent, meaning nobody in the room wants a particular answer

Think about who reviews an architecture today.

The vendor whose platform it is built on. The integrator being paid to build it. The internal team who designed it and would rather not redo it. The consultancy hoping this becomes a programme.

Every one of them is competent. Not one of them is neutral. And the questions that matter most are the ones with uncomfortable answers: is this worth building at all, is this constraint real, what happens when the model is wrong and who finds out.

That is the case for an independent review, and it has become sharper rather than softer now that first drafts are free. When designs were expensive, the design itself was the scarce thing and the review was a luxury. Now the design is cheap and the review is the only step left that requires a human who has watched systems fail.

There is a version of this that would be worthless, and it is worth naming because it is the version most people will build: an assurance review generated by a language model. If you are going to sell someone confidence, it cannot be produced by the thing whose failure mode is confident wrongness. The tool can draft the questions. A person has to sign the findings and be accountable for them.

The teaching: "Assurance produced by the thing being assured is not assurance. It is a second opinion from the same mind."

What to actually do on Monday

If you have a design in front of you that came out of a model, or out of a vendor, or out of a team that likes it, three questions will tell you more than another read-through.

What has to be true for this to work, and who checked whether it is true? Every design rests on assumptions about volume, skills, budget and time. Generated designs state them explicitly, which is a genuine gift. Take the list seriously and go and verify one.

What happens when it is wrong, and who finds out? Not the components. The failure path. A design that has no answer here is not finished, however good the diagram looks.

And what would you refuse to build? If the answer is nothing, the review has not happened yet. The most valuable thing a reviewer says is usually not fix this. It is do not build this part at all.

The generated first draft is a real gain. It compresses the boring, slow part of design and it puts a competent starting point in front of people who could not previously afford one. I would rather live in a world with those tools in it.

But a first draft that nobody is qualified to challenge is not a shortcut. It is a decision that has been made without anyone noticing they were making it.