In the previous essay, I argued that a product lives or dies in the experience of using it. For an AI model, that experience is the product. The more I sit with that, the stranger it becomes that companies with more money than most governments keep shipping software that falls apart on first contact. I do not mean that they lack the people or money to do better. Increasingly, they do not need to.
Reid Hoffman’s line about being embarrassed by the first version of a product made sense in its original setting. A founder with little cash, no users, and a few months of runway has to learn in public. They put something rough in front of the small number of people willing to try it, then find out whether the idea deserves to survive. Eric Ries turned that instinct into a method: the minimum viable product. It was advice for people short of money and time. It was not a permission slip for companies that have neither constraint.
So what happened to that distinction?
What has been largely lost is that distinction. The companies using this logic most aggressively now launch a minimum viable version to hundreds of millions of people and finish it in front of them. The embarrassment that was once the cost of getting started has become a defense for companies that could have cleared a higher bar. “Move fast and break things” came from a period when the thing being broken was a social network. Facebook retired the phrase in 2014. The habit was apparently too useful to retire with it.
Tim O’Reilly described “the perpetual beta” as a virtue of the early web, and at the time the label at least meant something. Gmail carried its beta tag for more than five years, an admission that the service holding your mail had not yet settled into being reliable. With that in mind, the later disappearance of the label was convenient. Unfinished products did not go into nonexistence; users were simply expected to take an upbeat release note, “we’ve improved,” as normal. Taken literally, the thing you used yesterday was worse. Figuring that out was now part of using it.
Wait a second. I am by no means against frequent releases. Good release engineering depends on testing a change before a broad rollout and being able to undo it when it goes wrong. Google’s own account of release engineering delineates that boundary well enough. A company can move quickly without making every customer the first person to discover its mistake. The argument becomes questionable only when frequent releases mean shipping widely before anyone has earned the confidence to do so.
With the arrival of AI, the habit has acquired another costume: measurement. Much of the testing is now automated and models often judge other models. LLM-as-a-judge is useful and cheap, and strong models often correlate well with human raters. I do not think that makes the approach worthless. Still, an evaluator drawn from much the same distribution as the model it evaluates ought to make anyone less comfortable with the score. Labs also optimize against a small set of public benchmarks, many of which leak. Contamination studies continue to find test items in training data, so a better score can reflect memorization as much as capability.
So why does a good score matter?
A benchmark can be an excellent regression test, and a model judge may catch a change that is obviously worse. The relationship with the person on the other side of the screen is another matter. A model can satisfy the judge and improve the benchmark while leaving the next person’s actual conversation no better. That is what the dashboard cannot see.
Once a proxy is allowed to set the shipping bar, a known gap stops looking like something that ought to be closed before launch. It becomes material for a future release. “Room for improvement” is the polite phrase for a disappointment, and it describes the business model rather well. A finished product gives you one announcement. A half-finished one keeps producing reasons to appear in the feed. If a feature can be finished after launch, leaving it unfinished beforehand starts to look less like necessity and more like incentive.
Some post-launch improvements are genuine; there are failures a team can only find in use, and release notes can be an honest attempt at accountability. The distinction becomes clearer when the experiment fails. Did users know they were in it, and could the company have found the fault first? Does anyone inside the company pay for getting that judgment wrong? A small beta group that knows it is testing something and a worldwide release dressed up as a learning exercise are not the same arrangement. They should not be treated as one.
What bothers me most is who pays the difference. The user becomes the quality-assurance team, unpaid and often with no idea that the test is still running. The Saudi user whose thread I wrote about last time was told he was using a national flagship, not a beta. He still spent a week discovering its failures for its makers. Scale that across every major launch and work moves from the party that can afford it to people whose time cannot be invoiced. You spend their trust as well. When every version is provisional, responsibility turns foggy: a failure becomes a known issue, and a regression becomes an iteration. The product was never quite meant to work in the state in which you found it. A version number and a public record of what changed are not bureaucratic niceties. They are how a user establishes what the product promised when they relied on it.
There is another complication now: the user is increasingly not the person.
Who is actually using the product?
Agents increasingly handle the small software errands people would rather not spend their afternoon doing. They are unusually tolerant beta testers. An agent does not get tired of a five-step flow or resent a broken endpoint; it retries until the task is completed or cannot be completed. It can fail spectacularly, of course, but it does not carry resentment forward in the way a person does. The old guardrail is that a human eventually gets annoyed and leaves. It weakens when the thing using the product feels nothing.
Calling this “user experience” is beginning to feel too narrow. A company can satisfy the agent and still fail the person behind it. The task may be completed and the dashboard may turn green, while the human comes out worse for having used the product. What a product owes has always been the human outcome, regardless of what does the clicking. Call it human experience if a new term is useful. The agent’s report is not the verdict for the person who has to live with it.
Building for agents remains worthwhile. Vercel and Stripe already publish an llms.txt file, and a site that a model can parse without driving a browser removes real friction. I only object when serving the agent turns into mistaking it for the customer. The agent is a means to an end. The person it works for is still the point.
Nobody needs to pretend that iteration is itself the issue. Some products cannot be specified in advance; they have to be learned with the people using them. Software is cheaper to correct than a bridge or a vaccine, and a product shipped confidently but wrongly can do more harm than one that admits uncertainty. I ship unfinished work too. This site has gone live with mistakes I noticed only after other people saw them, though nobody needs my permission to leave it behind. A bank or a health product is by no means in the same position. Neither is an AI assistant that people have begun to trust with their work.
My objection is to the well-funded company that launches early because launching is cheaper than finishing, then sells the gap back to us as a feature. “Finished enough” does not mean perfect. It means the first encounter respects the user’s time, and that beta is a state you eventually leave rather than a permanent waiver of responsibility. It also means the cost of finishing belongs in the launch budget instead of being quietly charged to the customer. An imperfect recipe is one thing. A model being trusted to interpret a contract or a medical result, let alone a government decision, is another.
For AI, I think the standard ought to be higher. The product is a conversation, and its failures land as somebody’s stupidity—sometimes the model’s, sometimes the user’s. A contaminated benchmark and a model grading its cousins do not create much of a floor. The test comes the next morning, when a real person arrives with a real question and has no use for the excuse that the apparatus was still learning.
The MVP was a promise to finish later, made by people who might not last long enough to keep it. When companies that will outlast their users make the same promise, the promise has largely lost its meaning. There are two ways to read the gap: as a debt the company owes the people who trusted it, or as work it has learned to leave to them. To no avail, we keep choosing the kinder reading for companies that can evidently afford the other one.
what a great post with so many insights I had not considered. This was illuminating