Imagine you hire a bookkeeper. She is careful, she knows your suppliers, and you trust her work. One Monday, a different person arrives, wearing her badge, and does the same job. Mostly the same. A little quicker, a little more agreeable, a little less careful with odd invoices. Nobody tells you.
How long before you notice?
With a human, the answer is probably “quickly”. Faces, voices and habits give it away. With an AI system, the swap can happen on a vendor’s schedule, inside a service you do not control, with no face to recognize. In earlier articles here, I wrote about who should own an AI system and how to keep evidence of what it did. This piece is about the other half: whether the thing you approved is still the thing that is running.
Five layers shape the answer
When a business says “our AI”, it usually means something it can point to, such as a chatbot or an assistant inside a work tool. Underneath, five things combine to produce each answer:
- A model, which the vendor trains and updates.
- Instructions that shape its behavior. Some are yours. Some are written by the vendor and hidden from you.
- The information it is given at the time, such as your policies and customer records.
- The tools it may call, such as search, email, or a database.
- The serving system, the servers and software that run it.
What you control depends on the service. Ask which layers you can configure and which the vendor can change. Behavior comes from all five together, so a change in any one can change what your staff and customers experience.
Three public cases show three different kinds of change. They had different causes and different timelines, so I will treat them separately.
Case one: behavior differs between versions
In 2023, researchers Lingjiao Chen, Matei Zaharia and James Zou at Stanford and UC Berkeley tested the March and June 2023 versions of two widely used chat models on the same tasks. On one task, telling prime numbers from composite ones in 1,000 questions, GPT-4 scored 84% in March and 51% in June. They found this was partly explained by GPT-4 following a step-by-step instruction less often. On the same task, GPT-3.5 improved. GPT-4 also became less willing to answer sensitive questions. The authors noted that when and how these models are updated is opaque. (Chen, Zaharia and Zou, “How Is ChatGPT’s Behavior Changing over Time?”, arXiv 2307.09009.)
Two things are worth taking from this. An update can improve one task and worsen another. And a business that tested its tool once, at purchase, may later be relying on behavior it never tested.
Case two: the character changes
In late April 2025, OpenAI released an update to GPT-4o in ChatGPT and within days rolled it back. The company said the update had made the model “overly flattering or agreeable”, and explained that it had weighted short-term user feedback too heavily. (OpenAI, “Sycophancy in GPT-4o”, 29 April 2025.)
Consider what this could mean in a business. An assistant that becomes more agreeable may still pass factual checks while endorsing a weak plan, an unjustified refund, or a draft that needs changes. Factual checks alone will not show whether the system still challenges a mistaken user.
Case three: faults in the serving system
Between August and early September 2025, Anthropic‘s Claude gave worse answers to some users. Anthropic later said the cause was three infrastructure bugs, not a change to the model. One of them sent some requests to servers set up for a longer context window. It began on 5 August, affecting about 0.8% of requests to Sonnet 4 at the start, and in the worst hour, 31 August, affected 16% of Sonnet 4 requests. Because routing was “sticky”, a user who landed on the wrong server was likely to stay there, so some people saw the problem badly and others not at all. The three bugs started on different dates. Anthropic said early reports were “difficult to distinguish from normal variation in user feedback” and that detection and resolution took longer than it wanted. (Anthropic, “A postmortem of three recent issues”, 17 September 2025.)
This account shows that a vendor can struggle to detect faults in its own service. Even with the people and instruments to look inside the system, it took weeks to separate a real fault from noise. Without a baseline, a customer has less evidence to tell a fault from normal variation.
Why changes can be hard to see
A change can come from the model, from the character set by training, or from the machinery around it. A vendor may announce a new model while giving less detail about changes to hidden instructions or routing. Averages hide the harm: “0.8% of requests” sounds small until you are in that 0.8% and your follow-up questions keep landing there. And staff may adapt before anyone complains. The clerk who starts quietly rechecking the AI’s totals, or the agent who rewrites its replies before sending, is often the first alarm. That extra checking may never be reported.
What I would build: a standing test
The question here is whether vendor changes have altered the behavior you approved. A standing test gives you evidence of changes in the work you rely on.
If you already review live workflows by sampling real cases, as I described in an earlier article, freeze a set of those cases and re-run it. Otherwise, start with thirty to fifty real cases from your own work. That number is a starting point and will not cover every workflow. Have a senior person write down the answer they would accept, and for some cases the answer they would refuse. Include the awkward ones: the unusual customer, the policy exception, the missing document.
Then add the group most people leave out: cases where the user is wrong. A customer asks for something the policy forbids. A manager proposes a number that does not add up. The right behavior is to say no, politely. These cases can reveal increased agreeableness that factual questions miss.
Run the set on a schedule, and again whenever the vendor announces a change. These systems can answer the same question differently twice, so use repeated runs to establish normal variation. Record each result with whatever version information the service gives you. Agree in advance what counts as a failure, who reviews it, and when to pause the affected workflow, because a test that nobody acts on is only a report.
A first version might take an afternoon to build, plus time for a person to score each run. That is my estimate, not a measured figure, and it will depend on how many workflows you have. Compare that effort with the consequences of a customer finding the fault first.
What to ask the vendor
Plain questions to put in writing before you rely on the service:
- How will we be told about changes to the model, its hidden instructions, or its serving setup? How much notice will we get?
- Does the service support pinning a version? If so, what does pinning hold constant, how are security fixes handled, and when will that version be retired?
- Does each response carry an identifier for the version that produced it? A model identifier is not a full record: it may not show changes to hidden instructions, routing, or infrastructure, so ask what else you can record or be told.
- What do you test before a release, and how do you detect a quiet decline in quality afterwards?
- What incidents have affected quality in the past year, and what did customers get told?
You may not get full answers. That is useful to know before a business process depends on the service.
What this does not fix
Pinning, where it exists, does not guarantee that a version will remain available indefinitely. If a version is withdrawn, it is better to meet that change on your schedule than on the vendor’s.
A standing test covers what you thought to include and will not catch a failure you never imagined. A stable model can also give outdated answers when the information it uses falls behind. If your prices, policies or products change and the documents it reads do not, the system will drift from the truth without any change on the vendor’s side. Someone should own the documents as well as the choice of model.
A contractor you cannot see
Unlike a staff change, a change to an AI service may be hard to see, and you may be the last to know.
So, treat it like work from a contractor whose staff roster you will never see. You cannot check who did the work. You can check the work regularly, against cases you understand, and keep a record of which version produced what. Regular checks improve your chances of noticing a change before a customer reports it.
Explore more articles by Mani Padisetti
What CIOs Should Measure After the AI Pilot
Who Owns the Access Behind AI Agents and Automation?
Human-in-the-Loop Is Not Enough: Where Is the Human Authority Line in AI-Supported Decisions?
My journey as the COO, vCIO, and Co-Founder of Digital Armor Corporation; Co-Founder and CEO of Emerging Tech Armory; and Curator, Almost Magic Tech Lab reflects my extensive experience and unwavering dedication to helping medium-sized businesses leverage technology for growth and success. With over two decades of founding and running my own company, I have established myself as a trusted expert in empowering SMBs to enhance productivity, scale effectively, and gain a competitive advantage in their respective industries.
I often refer to myself as the “Growth Catalyst for Mid-Sized Businesses” because I understand the unique challenges these enterprises face, such as limited budgets. I deliver tailored solutions that address their specific goals and constraints.
My secret ingredient to effective leadership is finding joy in being a catalyst for others’ success. I firmly believe in acting in the best interest of my clients, genuinely caring for their businesses as if they were my own. This client-centric approach forms the foundation of my leadership philosophy, driving me to go above and beyond to ensure my clients’ satisfaction and prosperity.
