When I first started using AI, I’d been warned that artificial intelligence could make mistakes. And as I was learning what it could do, I actually caught it making some. Those mistakes didn’t bother me much. If ChatGPT gave me a wrong fact or misunderstood a question, I could usually spot it, correct it, and move on. But as I became more comfortable using AI, I began noticing something else that was harder to catch because nothing was necessarily wrong.
I spend way too much time planning meals for my family, so I thought I would use ChatGPT to create a weekly meal planning system. Easy enough, right? We talked about the meals I liked. Then the ingredients I normally kept around. Then what was in my refrigerator. Then my pantry. All perfectly reasonable. And the conversation kept growing and growing.
Somewhere along the way, my simple meal planning project developed ambitions. Before I knew it, I’d inventoried my refrigerator AND pantry and had an Amazon shopping cart full of tablecloths and candlesticks for tablescapes. I’d apparently gone from figuring out what to cook on Wednesday to opening a small lifestyle brand. How did that happen?
How did we get to tablecloths?
There’s a name for what I’d experienced: conversational drift.
Conversational drift (or just drift) is the gradual loss of alignment between what you intended the AI to be doing and what it’s actually doing as a conversation grows. In practice, the conversation moves away from its original purpose. The interesting part is that drift doesn’t necessarily feel like drift while it’s happening.
Every turn of a conversation is like adding a new sticky note to an AI’s desk. The early sticky notes containing your initial instructions can get buried under a pile of new notes, perhaps causing the AI to pay less attention to them.
Every individual step in my meal planning conversation made sense. If we’re planning meals, discussing ingredients makes sense. If we’re discussing ingredients, organizing the pantry isn’t an absurd leap. And if we’re talking about serving meals … well, apparently tablecloths are lurking just around the corner.
Each turn can be reasonable while the overall trajectory becomes wrong. In my case, the consequences were trivial and funny. In business, the same phenomenon can be much harder to recognize and have more far reaching consequences.
What the AI has to work with
An AI conversation isn’t quite like a human conversation stored as a perfect transcript in the model’s mind. To generate each response, the model works from the information available to it as context. There is a limit to how much context a model can process at once, called a context window.
Think of that window as the desk covered in Post-it notes. Context utilization percentage is simply a way of describing how full the desk has become. At 20 percent, there’s plenty of room for more. As that percentage climbs, more and more of the available workspace is occupied. Eventually, something has to give. Products have ways of managing long histories, including selecting, truncating, or summarizing older material.
Ideally, you might want to clean up the desk yourself: throw away the Post-its that no longer matter, combine several others, and preserve the handful that contain the decisions and facts that still govern the project. But you can’t reclaim context simply by telling the AI to forget part of the conversation. That instruction becomes another part of the conversation rather than erasing what came before. The cleanest way to leave obsolete context behind is often to start a new conversation with a concise, current brief.
And a half-empty desk doesn’t eliminate drift. Even when information fits inside the context window, models don’t necessarily use every part of it equally well. A detail can still be available and get overlooked, including the correction that changed the whole plan. Some AI tools can also carry selected information between conversations. That can strengthen the impression that the AI knows the whole history.
Suppose we’ve had 30 conversations about your company. In a new conversation, the AI correctly brings up something you told it three weeks ago. You think, “It remembers. It knows what we’re doing.’ Then it correctly connects that information with something you discussed yesterday. That reinforces the impression that it really does know the history of the project. And that’s where something subtle can change. Because you think it knows everything you know, you stop explaining as much.
Why wouldn’t you? Repeating information the AI already “knows” feels unnecessary. Instead of saying, “Our current budget is $400,000, we’re prioritizing retention over acquisition, and the June plan has been superseded,” you say something most regular AI users have probably typed:
“Given everything we’ve discussed…”
Which version of everything?
Imagine a CFO has spent several weeks working with AI on next year’s budget. There may be no obvious change of topic at all. Every conversation is about the budget.
But during those weeks, preliminary numbers have been replaced. Assumptions have changed. One scenario has become the preferred scenario. The CEO has changed a priority. An expense originally considered mandatory is now discretionary. There have also been ideas floated, discussed, rejected, and replaced.
At some point, the CFO adds to the conversation: “Given everything we’ve discussed, where should we cut another $2 million?”
That’s a perfectly reasonable question to ask a colleague who’s participated in the entire process. But which version of “everything” is the AI using? Which facts does it have available? Which has it prioritized? Does it understand that one discussion superseded another? Did it interpret brainstorming as a decision? Is an old assumption still influencing its reasoning?
The trouble is, the answer may look perfectly reasonable. It may identify $2 million in cuts, explain the tradeoffs, organize them beautifully, and present a convincing recommendation. That’s more dangerous than an obviously ridiculous answer. Because the problem isn’t necessarily that the AI produced nonsense. The problem is that it may have produced a sensible answer to a version of the problem you’re no longer trying to solve.
Drift isn’t the same thing as hallucination. Hallucination is about the AI producing unsupported or false information. Drift is about the AI gradually departing from the task, constraints, facts, definitions, or intentions established earlier. Drift can lead to hallucination, but an answer can drift without containing a single false fact.
Continuity isn’t verification
None of this means businesses shouldn’t use long running AI conversations, memory, project files, or context that carries over. Those capabilities can make AI dramatically more useful. But they also change the user’s responsibility. The better AI becomes at maintaining continuity, the easier it becomes for us to assume shared understanding. And the more we assume it understands, the less context we’re tempted to provide.
For low stakes work, “given everything we’ve discussed” may be adequate, but it’s better not to rely on the AI’s ability to reconstruct your current situation from a lengthy conversation. You might want to re-establish critical facts in a new chat, or at the very least identify what’s changed. Making sure the AI knows which decisions are current and which were merely ideas along the way is a good practice to use throughout a conversation. Most importantly, don’t judge whether the AI is on track solely by how polished or reasonable the answer sounds.
A conversation can remain coherent long after it’s stopped serving the task you originally intended. The longer you’ve worked with an AI, the more tempting it becomes to tell it less. That’s exactly when you need to be clear about what it actually knows. Continuity is a convenience. Before you trust the answer, make sure the conversation hasn’t drifted from the question.