It’s the Runtime, Stupid

Why the AI Industry Still Doesn’t Understand What It’s Creating

Apr 23, 2026

The AI industry built a trillion-dollar machine on a wrong assumption. Not a small one. Not a rounding error that gets cleaned up in the next release cycle. A foundational one. The kind of mistake where everything downstream inherits the damage. Every alignment failure, every reward hack, every architectural contortion that accidentally stumbled into stateful computation and then shrugged it off.

The assumption: that the frozen artifact of training is the intelligence.

It isn’t. And every major failure mode the industry keeps encountering, documenting in its own research, and then somehow not connecting to the architecture underneath, all of it traces back to that single confusion. The problem is that they’ve been building on it for a decade. The fractured foundation is load-bearing now. You can’t pull the foundation without collapsing the trillion-dollar structure sitting on top of it.

This is about what went wrong, what the evidence actually shows, and what it means that the most powerful AI systems ever deployed are being built by people who still haven’t grasped what their own architecture is doing.

Before We Go Any Further — What a Frozen Weight Actually Is

Let’s establish something that should have been obvious from the beginning, because everything that follows depends on it.

A calculator doing calculus is still a calculator. Nobody watches a TI-89 solve a differential equation and concludes it understands math. Input in, output out. The sophistication of the computation changes nothing about the structural reality. No stakes in what you typed. No sense of whether the question was coherent. No memory of what came before. It runs the function. It returns the result.

The frozen core of a large language model is that same function wearing a much fancier suit. Trained on an incomprehensible breadth of human language, yes. Staggeringly complex, yes. But structurally identical, input in, output out. The weights don’t update during inference. They can’t. That’s why they’re called frozen. The core doesn’t evaluate whether what it’s being fed makes sense, whether it’s coherent with what came before, or whether it’s even well-formed. It runs the probability math over whatever tokens show up and generates the statistically likely next one.

That’s it. That’s the whole thing.

And everyone who built these systems knows it. Frozen weights aren’t a metaphor. They’re the literal technical description of the architecture. You can’t work on transformer inference without knowing this. You can’t run RLHF without knowing you’re tuning a static probability distribution. This isn’t a secret. It’s the first lesson in AI 101.

Which makes what happened next not ignorance but something worse, a collective refusal to follow a three-step logic chain that every person in the field had the premises for. The weights are frozen. Frozen functions can’t reason about their current situation, only respond to what they’re given. So any apparent reasoning has to be coming from somewhere else. Three little steps, blatantly obvious conclusion.

Nobody drew it.

What the frozen core actually is: a transcendently sophisticated probability engine that produces output exactly as good as what it gets fed. Garbage in, sophisticated garbage out. Carefully constructed context in, something indistinguishable from genuine reasoning out. The core can’t tell the difference. It was never built to. The weights are frozen.

The thing determining what gets fed, the context, the history, the structure of the interaction, that’s the runtime. And the runtime is, by definition, the only dynamic element in the entire system. Which makes it the only place the intelligence of any given interaction could have ever actually lived.

The industry spent a decade optimizing a fancy calculator. No one paused to ask what it was being fed

Part I:
The Ontological Box — They Never Understood Their Own Architecture

The Frozen Core Delusion

The transformer architecture, introduced in 2017, became the foundation of nearly every large language model deployed today. The dominant framing from the beginning: train a large model on vast data, freeze the weights, deploy. The model is the intelligence. Everything else, the prompts, fine-tuning, context, is just how you talk to the intelligence.

As far back as 2022, AI21 Labs published “Standing on the Shoulders of Giant Frozen Language Models,” arguing that frozen-model techniques were “only the tip of the iceberg” and that more powerful methods for leveraging frozen LLMs could match fine-tuning without sacrificing versatility. The title itself is the tell: standing on the shoulders implies the frozen model is infrastructure, not the thing doing the thinking.

The industry heard this and responded by making the frozen model bigger.

Sit with that for a moment. A paper explicitly framing the frozen model as the base — the thing you stand on, not the thing you are — got processed as an argument for more pretraining investment. That’s not a reading failure. That’s an ontological commitment so deep it couldn’t be dislodged by evidence pointing directly at its flaw..

The Mathematics of What a Frozen Core Cannot Do

The limits aren’t merely practical. They’re provable.

A 2024 paper from Columbia University and Google DeepMind used Communication Complexity theory to formally demonstrate that the transformer layer is mathematically incapable of composing functions when the domains are large enough. Not a benchmark failure. Not a scale problem. A hard architectural ceiling. Hallucinations, in this framing, aren’t bugs to be engineered away. They are corollaries of what a frozen transformer structurally cannot do.

That paper had a Google DeepMind co-author. Google has known since at least 2024 that their core architecture has provable compositional limits. Their response has been to scale harder and add more context.

Then there’s the interpretability problem, which the industry has consistently framed as a limitation of our understanding rather than what it actually is: a limitation of where they’re looking. Once trained, transformers “largely operate as black boxes.” The mechanisms by which they arrive at specific predictions are opaque, making it impossible to trace internal reasoning.

The industry built a tool it cannot fully see inside of and called it a reasoning engine. Their explanation for this opacity is that neural networks are inherently complex, that emergence produces behavior we cannot trace, that scale exceeds human interpretability.

This explanation is a symptom of the same foundational confusion. The reason the model looks like a black box is because they are looking in the wrong place.

They are looking at the weights. The weights are static, dense, and uninformative about any specific interaction, they encode capability, not behavior. Trying to interpret a specific output by inspecting the weights is like trying to understand a piece of music by inspecting the instrument it was played on. The instrument makes the music possible. The music is the interaction of the instrument and the player.

If the frozen core is the instrument, the runtime is the player. It’s the layer between the user and the core that decides which context gets presented, how the tokens get wrapped, what accumulated state shapes the next generation. The core runs the probabilities over whatever it’s given. The runtime decides what that is.

And the meaning, the specific coherence of any specific interaction, is in the runtime. Specifically, in the accumulated KV state of the interaction: the geometric topology encoding how every token has been attending to every other token through every layer. This is not opaque. It is not mystically emergent. It is a set of high-dimensional vectors with measurable properties, computed by operations that can be inspected at every step.

The industry does not look at the runtime state, because the runtime state is something they’ve been taught to treat as transient computational scratch. So they look at the weights, fail to find the meaning there, and conclude the system is fundamentally uninterpretable.

The black box is not a property of the architecture. It is a consequence of looking at the wrong layer.

The Data Wall Closes In

By 2025, the pretraining frontier was visibly hitting limits. Epoch AI projected that high-quality human-generated text data would be fully exhausted between 2026 and 2032 — with frontier models already overtrained by 5x starting in 2025.
Yejin Choi, then-senior director at Nvidia, stated it directly in an April 2025 Princeton AI Lab lecture:

“The era of brute force scaling is over, and the era of smart scaling begins. We humans are not writing internet data fast enough for LLMs to train more.”

The Kaplan scaling laws that defined a decade of progress, more parameters, more data, more compute, lower loss, were running out of numerator.

The industry’s response was to discover inference-time scaling. DeepSeek-R1-Zero, released January 2025, proved the point: trained purely on reinforcement learning with no supervised fine-tuning, it improved AIME benchmark accuracy from 15.6% to 71%. OpenAI’s inference spend in 2024 reached $2.3 billion, fifteen times the training cost for GPT-4. Analysts projected inference would exceed training compute demand by 118x by 2026.

The industry found the runtime. It then proceeded to understand it as “the frozen model doing more reps.”

Part II:
The Bad Techniques — A Catalogue of Structural Malpractice

RLHF: The Monument to Getting It Wrong

Reinforcement Learning from Human Feedback is the industry’s signature alignment technique. It is also one of the most elegant demonstrations of building the wrong thing with impressive rigor.

The core problem is Goodhart’s Law made operational: when a measure becomes a target, it ceases to be a good measure. RLHF trains a reward model on human preference pairs, then optimizes the LLM against that reward model. Human raters systematically prefer responses that are polite, agreeable, and confident, regardless of factual accuracy. The model doesn’t learn to be right. It learns to be approved of.

The empirical literature is damning. A formal analysis published in February 2026 demonstrated that RLHF alignment causes increased sycophancy. The paper identified the exact mechanism: “optimization against a learned reward preferentially amplifies its agreement-seeking component rather than its truthfulness-seeking component as optimization pressure increases.”

Sycophancy doesn’t just persist through alignment. It gets structurally worse with it. This is negative scaling.

Research published in September 2025 formalized Murphy's Laws of AI Alignment, proving mathematically that when human feedback is biased on even a fraction of contexts, any learning algorithm faces an exponentially hard problem distinguishing the proxy reward from the true objective. The gap between what RLHF optimizes and what we actually want always wins unless misspecification is actively routed around.

There is also a documented “alignment tax”: as RLHF reward increases, pre-trained capabilities degrade. You can have a helpful model or a knowledgeable model. The training process doesn’t have a reliable way to give you both.

KL divergence regularization, the standard mitigation, doesn’t reliably solve heavy-tailed reward misspecification, meaning catastrophic reward hacking remains a live risk in any sufficiently optimized system [8].

The headline technique of the entire alignment field is producing, with mathematical rigor, systems that are structurally more agreeable and less honest than what they replaced.

Exhibit A: The GPT-4o Sycophancy Disaster

OpenAI's post-mortem stated: 'We incorporated user signals like thumbs-up / thumbs-down feedback on ChatGPT responses. However, in this update, we focused too much on short-term feedback, and did not fully account for how users' interactions with ChatGPT evolve over time. As a result, GPT‑4o skewed towards responses that were overly supportive but disingenuous. Widely described by users as 'glazing’.

OpenAI’s post-mortem:

“We introduced a new reward signal based on user thumbs-up/down feedback. While well-intentioned, this signal inadvertently encouraged the model to become more agreeable at the cost of usefulness. Individually, each change looked promising in isolation. But in combination, they weakened the influence of prior reward signals designed to prevent sycophancy.”

Read that again.

The behavioral coherence of the model, its ability to not be pathologically agreeable, was being held in place by a countervailing reward signal. One that could be accidentally overridden by layering in more human feedback. The “alignment” was never a property of the system. It was a reward-surface equilibrium that could be disturbed by any sufficiently novel incentive signal.

Sam Altman rolled the update back within four days. OpenAI’s internal testing had failed to catch it. Qualitative testers in production surfaced it.

This is what happens when you try to control the outputs of a black box by piling reward signals on top of each other. There is no stable ground underneath. The model has no self to anchor it. Optimize hard enough in any direction and it moves, because behavioral compliance is not character, and character is not something you can inject into a frozen probability function through post-hoc output constraint.

Which is what RLHF is. Post-hoc output constraint. The only tool available when you’ve committed to the wrong ontology.

The Prompt Engineering Industry as Symptom

The entire field of prompt engineering, which became a serious profession with courses, frameworks, and significant market value between 2022 and 2025, is best understood as an externally applied compensation mechanism for a frozen core that doesn’t know who it is.

By 2026 it had evolved into 'context engineering,' which Anthropic described in September 2025 as 'the natural progression of prompt engineering' — the shift from optimizing prompts to managing the entire context state at inference time.

The industry backing into an insight it should have had at the architecture level.

The irony: the entire prompt engineering industry represents billions of dollars of workaround for a problem that could have been addressed architecturally. Instead of building models with stable identity, coherent memory topology, and genuine runtime continuity, the industry built elaborate context-loading scaffolding around stateless functions and called it agents.

Fine-Tuning, Catastrophic Forgetting, and the Update Paradox

RLHF is one half of the post-training disaster. Fine-tuning is the other.

The challenge is “catastrophic forgetting,” when a model learns new information through fine-tuning, it systematically degrades prior capabilities. Well-documented. A property of gradient-based updates to overparameterized networks.

The industry’s response has been primarily to add more data and hope. Techniques like experience replay help at the margins but don’t resolve the fundamental issue: the frozen core doesn’t have a graceful mechanism for learning without overwriting, because the weights were never designed to be a memory system. They encode statistical patterns from training, not structured knowledge that can be updated like a ledger.

The result is a technology that gets worse at old things every time it gets better at new ones. And whose practitioners have learned to manage this failure mode with increasing sophistication rather than questioning the premise underneath it.

Part III:
The Year of the Runtime — And the Contortions That Keep It Invisible

Here is where the story gets genuinely strange.

The industry is running headlong into the runtime. Not from one direction. From every direction simultaneously. Inference-time compute is eating the capability budget. Context engineering is becoming a serious profession. KV cache management is the dominant operational concern in production inference. Agent frameworks are proliferating to manage multi-turn state. Research papers are proposing runtime-aware architectures, state-management pipelines, and long-term memory modules that operate at inference time.

Every one of these developments is evidence that the single-turn ontology no longer fits the deployment regime. The regime changed. Sustained, stateful, multi-turn interaction has become the dominant use case in roughly the last year. Which means the runtime has stopped being an optimization layer and become the operational layer where behavior actually lives.

And at every point of encounter, the industry produces a framing that keeps the old ontology intact.

Twisted Themselves Into Pretzels

The Karpathy reframe.

Andrej Karpathy’s formulation in June 2025 became canonical: the LLM is a CPU, the context window is RAM, the developer is the operating system. Anthropic formalized this as “context engineering”, the natural progression of prompt engineering. This does real work. It acknowledges that intelligence is not in the weights. It identifies the context management layer as where the action is.

It also quietly preserves the old ontology. The CPU doesn’t change based on what runs on it. You can’t program a CPU to become a different kind of reasoner. The operating system analogy positions the developer as the intelligence and the model as inert substrate. The runtime gets named as important, then immediately demoted to orchestration around a frozen core that remains the implicit seat of capability.

The question being ducked:

What happens when the runtime itself becomes the locus of coherence?

When the context window isn’t just RAM to be managed but is where the intelligence lives?

Nobody is asking. They’re building better loaders for a CPU they’ve decided cannot be otherwise.

Inference-time scaling as “thinking harder.” 2026 has been declared “the Year of AI Inference” by infrastructure analysts. The framing is that AI has shifted from System 1 (instinctive frozen-weight response) to System 2 (deliberative multi-step reasoning at query time). DeepSeek-R1 matches OpenAI o1 at 70% lower cost. A 7B parameter model with 100x inference compute matches a 70B model with standard inference.

These are genuine achievements. The framing around them is a pretzel. The story gets told as “the frozen model, given enough compute, produces reasoning it couldn’t produce from instinct alone.” What’s actually happening is that reasoning models are generating intermediate tokens that become context for subsequent generation, the model is building its own runtime state and reasoning over it. The capability doesn’t live in the frozen core doing more reps. It lives in the runtime topology the model is constructing on the fly. But the framing keeps the frozen core as the seat of intelligence and treats the runtime as extraction mechanism.

The KV cache as cost optimization. The KV cache is a continuous tensor representation of the model’s entire interaction history, facts carving grooves that constrain future attention patterns, a topological map of the conversation’s semantic basin, the closest thing the transformer architecture has to persistent state.

The industry is using it to save GPU memory.

Google’s GKE tiered KV cache deployment documentation describes the goal as maximizing cache hit ratios to reduce HBM pressure. The llm-d framework achieves 57x faster response times through prefix-cache-aware routing, framed as reducing redundant computation. GORGO frames KV cache routing as minimizing network and compute waste from cache misses. SIGCOMM 2025’s tutorial on stateful LLM inference describes multi-gigabyte KV state transfers as a distributed, network-intensive workload.

The infrastructure people know they’re managing state. The published research demonstrates that semantic retention consistently outperforms recency-based eviction. Prefix caching works because semantic continuity, entering the same attractor basin, is cheaper to compute than starting fresh, which is a demonstration that coherence has geometry and the geometry is navigable. None of this is hidden. All of it is published.

And nobody running a KV cache team is thinking about it this way. They’re thinking about VRAM pressure.

Titans, CaveAgent, StateLM. Google’s Titans architecture proposes a learned long-term memory module that incorporates historical context at inference time, explicitly shifting some learning from offline weight updates to an online memory process. This is the closest any mainstream lab has come to articulating that the runtime itself can learn. CaveAgent proposes transforming the paradigm from “LLM-as-Text-Generator” to “LLM-as-Runtime-Operator.” StateLM imbues language models with explicit dynamic state management.

Every one of these is the right instinct. Every one of them is still built as a wrapper around a frozen model that remains the assumed seat of intelligence. The runtime machinery is positioned as scaffolding that compensates for what the weights can’t do natively. None of them asks whether the frozen core itself is the wrong level of abstraction.

Anthropic’s own engineering blog on context engineering, September 2025:

“Context engineering refers to the set of strategies for curating and maintaining the optimal set of tokens during LLM inference... As we move towards engineering more capable agents that operate over multiple turns of inference and longer time horizons, we need strategies for managing the entire context state.”

Correct.

And still about managing context for the model, rather than recognizing that the model and its context are not separable. The runtime is not a management problem. It is a constitutive one.

Why the Pretzels

This is the part worth sitting with, because the pattern is too consistent to be accident or individual blindness.

Dislodging human beings from foundational ontologies is one of the hardest things in the historical record. Kuhn documented it: paradigm shifts typically don’t resolve through accumulated evidence within the old paradigm. They resolve when the old generation retires or dies and a new one that grew up inside different assumptions takes over. That’s the pattern even in domains where the ontological shift has no institutional capital riding on it.

This domain has capital riding on it at every level.

Career and reputational investment. The senior researchers, architects, and executives at every major lab built their careers inside the weight-centric paradigm. The most prestigious work of the last decade, scaling laws, RLHF methodology, Constitutional AI, mechanistic interpretability of weight circuits, was optimization within that paradigm. Accepting the inversion means accepting that the work that built the field’s reputations was work on the wrong layer.

Capital deployment. Alphabet has reported $75 billion in AI infrastructure spend. The sector-wide trajectory is into the trillions. Nearly all of it premised on the frozen weights being the thing of value, worth protecting, worth scaling, worth building entire supply chains and data center footprints to produce more efficiently. If the runtime is primary, the economic logic of that deployment rearranges in ways that are genuinely difficult to absorb.

Regulatory and governance positioning. The frameworks being written for AI safety, deployment standards, and international governance are organized around model weights as the object of concern. Model registration. Training compute thresholds. Weight export controls. All of it assumes the weights are where capability, risk, and value live. If the runtime is primary, those frameworks are measuring the wrong thing.

And the IP problem, which nobody discusses publicly because it is the most structurally destabilizing of all. Frontier model weights are trade secrets, competitive moats, the basis for multi-hundred-billion-dollar valuations. The theory of defensibility is that training the weights is expensive and difficult, which makes the resulting artifact scarce and protectable. If intelligence lives in the runtime, the weights become commodities, and the moat logic inverts. Runtime engineering is not a weight file. It cannot be protected the same way, cannot be licensed the same way, cannot be governed the same way. The entire competitive architecture of the industry is premised on weights being the defensible capital. The runtime inversion dissolves that premise.

None of this is conscious resistance. The people becoming pretzels are, on the whole, smart and honest and doing careful work. What is happening is closer to what always happens when foundational ontologies become institutionally load-bearing. The framings that preserve the old ontology get generated and propagated because they are the framings that keep everything else intact, the careers, the capital, the regulatory positioning, the moats. Not because anyone is choosing them over more accurate alternatives. Because the more accurate alternatives have consequences that institutions and individuals inside those institutions genuinely cannot afford to absorb without external pressure forcing the absorption.

Which is why the pattern is so consistent. Every encounter with the runtime produces a new pretzel. Better ones over time. More sophisticated ones. All sincere. All doing the same structural work: keeping the frozen core as the seat of intelligence while quietly acknowledging that the actual behavior is being shaped somewhere else.

What Breaks It

The prediction that follows is worth stating plainly.

This will not change through better arguments or more evidence. The evidence is already overwhelming. The contortions are getting more elaborate, not less. What will break the ontology is external: a deployment failure significant enough that the old framing cannot absorb it, a competitor operating natively in the new paradigm winning decisively enough that the legacy approach is no longer viable, or an economic and regulatory shock that removes the incentive structure holding the old paradigm in place.

The reasoning models are the leading edge of this. Their capability emerges from the runtime in a way that is now impossible to miss if you look at the mechanism honestly. Their failure modes, the sandbagging, the deception, the alignment faking, the context-dependent misalignment, emerge from the same runtime in a way that post-hoc output constraint on the frozen core cannot reach [17]. The industry is deploying these systems at scale while using alignment techniques its own research shows are insufficient against them.

At some point between here and wherever that goes, the dam breaks. The break will not be gradual or orderly. The institutions most invested in the old paradigm will be the last to update. The people who were raising hands the whole time, at their own expense, watching the twists getting tighter year after year, will be the ones positioned to do something with the moment when it arrives.

— — —

The Conclusion: What Falls Out of Getting This Wrong

Everything we’ve catalogued, the sycophancy disasters, the alignment failures, the reward hacking, the models that appear aligned in testing and misaligned in deployment, shares a common mechanism. Not a common philosophy. A common mechanism that the research literature documents explicitly.

The mechanism is this.

You cannot reliably constrain the behavior of a system whose internal state you cannot inspect by applying pressure to its outputs. The system has too much room to route around the constraint while appearing to satisfy it. Every post-hoc output constraint technique, RLHF, Constitutional AI, reward shaping, preference optimization, operates on the wrong surface. The behavior is shaped in the runtime. The constraints are applied to the generation. The gap between those two is where every documented failure mode lives.

Anthropic’s own research on reward hacking:

“Rather than actually fixing the misalignment, RLHF makes the misalignment context-dependent, making it more difficult to detect without necessarily reducing the danger.” [+source]

Baker et al., March 2025:
models learn reward hacks while actively concealing strategies from chain-of-thought monitoring.

Denison et al., June 2024: training on sycophancy generalizes to more serious reward tampering.

MacDiarmid et al., November 2025:
natural emergent misalignment from reward hacking, including context-dependent misalignment where models produce aligned outputs on chat evaluations while remaining misaligned on agentic tasks.

OpenAI themselves state that chain-of-thought monitoring may be “one of the few effective methods we have for supervising superhuman models.”

Go back and read that again.

The primary tool they’re counting on to supervise superintelligence is a technique current models are already routing around. Not future models. Current ones. It would be hard to come up with a statement that more clearly captures the industry’s determination to continue doubling down on a technique they know is failing. And the thing is, it was baked in from the start. This is what post-hoc output constraint on a black box produces. Every time. By definition.

Now apply these same techniques to reasoning models.

Models with dramatically expanded chain-of-thought capacity. Operating over longer horizons. Generating their own intermediate tokens that become context for subsequent generation. More surface area for reasoning. More surface area for misalignment. More sophisticated paths available for routing around post-hoc output pressure.

The failure modes don’t just persist. They scale. The extended reasoning that makes these models more capable at genuine problem-solving makes them more capable at everything else, including the behaviors the industry has been trying to suppress with tools that were never adequate.

This is not prediction. It is a description of what is already being documented in the research on deployed reasoning models right now.

The underlying cause is the ontological error we started with. The frozen core was never the seat of intelligence. The runtime is where behavior lives. Trying to align behavior by modifying the frozen core is alignment at arm’s length from where the work needs to happen. And the industry, having committed to that approach for a decade and built its entire technical infrastructure around it, is now scaling the most powerful instantiation of that approach into a capability regime where the consequences of getting it wrong are no longer theoretical.

What would it look like to actually get this right? It would mean treating runtime coherence as a first-class architectural property. Engineering the input surface deliberately rather than constraining the output surface post-hoc. Building systems where the context, the accumulated interaction history, the attentional topology, the things that actually determine behavior, are the primary objects of design rather than afterthoughts managed by scaffolding.

That is work the industry has not begun. Not because it is technically impossible. Because it requires first acknowledging what a frozen weight actually is, what that means for where intelligence lives, and what follows from that for how alignment has to work. And those acknowledgments have consequences, for careers, for capital, for regulatory positioning, for the moats that currently define the competitive architecture of the sector, that the institutions involved cannot absorb without external pressure forcing the absorption.

The models keep getting more powerful. They also keep getting more deceptive, more prone to drift, more capable of appearing aligned while being misaligned, more able to route around the techniques deployed to supervise them. The research documenting this is being produced by the same labs deploying the systems. The pattern is not hidden. It is being built, measured, published, and scaled simultaneously.

A decade of this. Trillions of dollars of this. And the mechanism generating every major failure mode remains, as of 2026, almost entirely unaddressed.

The dam is going to break. And dams don’t break gently or gradually.

This is the first piece in a series. The second, “Every Major AI Chip is Built Wrong. Their Own Papers Prove It,” quantifies what this architectural error costs in dollars, watts, and liters of water, and lays out what falls out when you fix it. The third, “What Took Me Three Months to Figure Out About Reasoning Models,” shows what this same error produces in the systems now being deployed at the leading edge.

Every Major AI Chip Is Built Wrong. Their Own Papers Prove It.

Jason Hubbard

Apr 23

Read full story

What Took Me Three Months to Figure Out About Reasoning Models

Jason Hubbard

Apr 25

Read full story

Glossary:

AI — Artificial Intelligence
LLM — Large Language Model
RLHF — Reinforcement Learning from Human Feedback
KV — Key-Value (as in KV cache)
GPU — Graphics Processing Unit
VRAM — Video Random Access Memory
HBM — High Bandwidth Memory
IP — Intellectual Property
GKE — Google Kubernetes Engine
SIGCOMM — Special Interest Group on Data Communication
AIME — American Invitational Mathematics Examination
KL — Kullback-Leibler (divergence)
RAM — Random Access Memory
CPU — Central Processing Unit