LLMs Are Still Toxic, Stuck in the Past, and Bad at Math | Eyosias
Pangram verdict · v3.3
We believe that this document is a mix of AI-generated, AI-assisted, and human-written content
AI likelihood · overall
MixedArticle text · 1,914 words · 5 segments analyzed
It's been nearly four years since ChatGPT was released. For most of us that first encounter was GPT-3.5, and the friction was enough to convince a lot of developers that this thing was not built to handle code. The hallucination rate was high, and the complaints came down to four issues.
AI was bad at math. AI was stuck in the past. AI had a short memory window. AI was toxic, reflecting the worst parts of the internet.
Take a look at what people were saying back then. So were all of these solved over the past few years? Is that why the technology is suddenly so useful? Nope. All of these problems still exist, even in the newest models. Some have workarounds and others have ways to limit how often they show up, but every one of them is still sitting in the model. Being a developer through this has felt strange. For a while the tools were hit or miss. Some days they saved me real time, other days they just made more work, and I never knew which I'd get. Then in January 2026, at the peak of Claude Code, half my feed was people realizing an agent could just write the code for them. The model underneath hadn't changed much, but the tooling around it had finally caught up. That's worth slowing down for. Chasing each new tool means relearning something that may not matter a year from now. It's agentic swarms first, then loops, then graphs, and on and on. Meanwhile the four limits underneath don't move, and every trick worth knowing is a workaround for one of them. Learn the limits and the rest of the news cycle starts making sense on its own. AI is bad at math First, AI is bad at math, and the solution is just a workaround. So if you feed a batch of math equations directly into new frontier models like Fable or Sol through their API instead of their native apps, you'll still see them miss some of the equations you sent. I sent GPT Sol High 200 simple addition questions and it got one wrong. Put that AI in charge of your company's accounting and it's fumbling a transaction every couple hundred entries.
Here is the one it missed. 3565304616814852Model answered52467998Exact answer52467898 So why did it get it wrong? When you ask an AI a simple equation like 1 + 1, it gets it right, because that answer is found in its training data. Give it a random, long pair like 35653046 + 16814852, though, and there's a chance that exact equation has never shown up before. With no memorized answer to lean on, the model does the only thing it knows how to. A language model only ever predicts the next likely token, so it never really calculates. It ends up imitating the standard algorithm, the long-addition method from grade school, working through it one digit at a time. But it's still picking each digit by what looks statistically likely instead of doing the actual math. In our case, that's where it slipped and wrote a single wrong digit. That step-by-step working out is what makes it a reasoning model. Before answering, it reduces one hard problem into a chain of simpler ones it can handle, which is how the run above got 199 of 200 right. It's a real jump, but each step is still a guess, so it never quite reaches a guarantee. Yet drop the same problem into ChatGPT or Claude and it gets it right. Something outside the model is stepping in. The app gets around the model's weak math by handing the problem to an outside tool. When it notices a hard math problem, it sends the numbers to a server running Python and lets a small script do the calculation instead of the model. That wrapper around the model is called the harness. A harness is just whatever code runs around the AI. It can be all sorts of things, but here it's the part that catches the math problem and calls the Python script. Here's how it works.
To see this in action, watch how your ChatGPT or Claude app handles a math prompt. If the answer is simple enough to already be in its training data, it just responds directly. If the problem is a little harder, it first tries to work it out itself with the standard algorithm.
And once the math gets complex enough that neither of those is reliable, it pauses, reaches for the Python tool, and computes the exact answer. The model is trained to judge which of these three routes fits the problem in front of it. When it decides it needs the Python tool, the harness is there to execute the tool call. The whole thing feels like a bit of a hack. In the end, language models only output text. So to make tool calling work, developers train the model to spit out a specific JSON structure whenever it wants a tool. The harness watches the model's output, spots that JSON, runs the tool, and feeds the result back so the model can write the final answer. The illustration above walks through the raw events if you want to see exactly what that looks like. Tool calling took off around 2023. Meta's Toolformer trained a model to reach for tools like a calculator on its own, and OpenAI soon made it reliable enough to ship in ChatGPT. You can see the tools any app uses by asking it to list all your internal tools. Early on, developers wired every tool into the harness by hand. If you wanted a model to read your GitHub issues or query your database, someone had to build that specific integration into that specific app. Anthropic standardized this in late 2024 with the Model Context Protocol, or MCP. Instead of baking each tool into the harness, you run a small MCP server that advertises what it can do, and any MCP-aware app can plug into it. That Python calculator from earlier could sit behind an MCP server, and Claude, Codex, and Cursor could all reach for it without anyone writing the same integration three times. Underneath it's the same tool-calling loop. MCP just agrees on a common shape for how a tool describes itself and hands back its results, so a tool gets built once and every app can use it. AI is stuck in the past Every model is trained on data up to a certain date, and that date is its cutoff. After that the model is frozen and doesn't know about anything newer. Take GPT Sol High, with a cutoff of February 2026. Ask its API who's in the 2026 World Cup final with web search off, and it can't give you the right answer.
PromptWho won the World Cup Final 2026?gpt-5.6-sol answeredThe 2026 FIFA World Cup Final has not yet been played. It is scheduled for July 19, 2026, at MetLife Stadium in New Jersey.Wrong. Spain won the 2026 final. So why did it get it wrong? The final is in July 2026, past its cutoff, so it never made it into the training data. The way around this is RAG, short for Retrieval-Augmented Generation. That web search we just turned off is a version of it. With it on, the model can look up what's current and answer based on that. There are a few types of RAG. The web search one pulls from the open web. The basic version works off a document you give it instead. They both end up handing the model outside information, but they go about it pretty differently. Here's the basic version of the document one.
As context windows get bigger, do we still need RAG? Yes. Even a 10 million token window is tiny next to a real codebase or document store, and stuffing everything in is slow, expensive, and something models handle unevenly, since they tend to lose track of the middle. Pulling in just the relevant piece stays the cheaper, sharper move, and that holds even as the shape of retrieval keeps changing. Do apps like Codex and Claude Code use RAG? For navigating code, mostly not the classic version. Instead of embedding your whole codebase into a vector database, they lean on agentic search, where the model runs grep, lists files, reads them, and follows imports the way a developer would. Anthropic pulled the vector index out of Claude Code in 2025, and Codex's CLI is told to prefer ripgrep for the same job. Vector RAG is still the norm for document question answering and large knowledge bases, just not for reading code. Boris from the Claude Code team explains the reasoning here. Boris from the Claude Code team on why they no longer use RAG AI has a short memory A regular program can pull millions of rows from a database and only load the one it needs.
A model can't do that. Each request to a model carries the entire transcript so far, your messages and its own, all fed back in at once. What looks like one new message is the whole conversation, sent again. It has to work this way because the model keeps no memory between turns, so every time it replies it re-reads the whole conversation from the top. There's a hard cap on how much of that text it can take in, and that cap is called the context window. SystemUserAssistantfixed limit · 3,000 tokensOne run replays the system prompt and growing transcript until the fixed context window fills.Turn 0 of 3. 0 tokens currently in the context window. Why resend the whole transcript every time instead of just passing each message one by one? Because the model is stateless. There is no storage inside the LLM, and its weights have stayed frozen since its cutoff date. So every request starts from a blank slate, and the only way it knows what came before is to see the whole conversation again. If the system sent only your latest line, the model would have no idea what you said a moment ago, so the transcript has to come along as the memory, and there is no memory unless the system resends it. If the whole transcript has to be sent anyway, why send all of it on every single query? Why not send just the new message each time and let the model append it to what it already has? It's technically possible, but it would be hugely wasteful. At this point it stops being an LLM problem and becomes a hardware one. To keep the conversation on its side, the server would have to store the work the model did while reading the transcript. It keeps that work in a special cache called the KV cache. Without that cache the model would redo everything each step, so generating the 1,000th token would mean pushing all 999 before it back through from scratch. The cache holds the math already done, so each token gets computed only once. The KV cache lives in VRAM, the fast memory attached to the GPU (think of it as RAM made for and soldered onto the GPU). VRAM is scarce and expensive, and a single model serves millions of people at once, so pinning every idle chat there while people read and type would eat up the most limited resource on the machine.