What a 23% kernel speedup tells us about whether AI can improve itself and how to quantify the gains.

In 2025, Google DeepMind’s AlphaEvolve, an AI system that discovers algorithms, found a way to make a FlashAttention kernel run 23% faster. This was a great result but it was not the beginning of a runaway loop. AI improving the software that trains AI, which trains better AI, and so on.

Here is what that 23% actually did. It made a 1% reduction in Gemini’s training time.

Not a disappointment. A 1% reduction on a training run costing tens of millions of dollars is worth real money, and AlphaEvolve found many such wins. But the ratio is the most instructive number in the entire recursive self-improvement debate, and almost nobody quotes it. Work backwards from 23% to 1% and you learn that the attention kernel is about 4% of training wall-clock. A big win on a small slice is a small win.

That arithmetic — the gap between how impressive an improvement sounds and how much of the whole it actually touches — is the central fact about AI improving itself. It has a name, and it’s older than the field.

Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops (July 2026)

Screenshot

Unlimited Self-improvement is mainly verification limited. However, huge improvements can be made before getting to unlimited.

The Prime Intellect self improving harness could be a path to expanding self improvement to aspects beyond the coding implementation. I will investigate and try to understand more about the Prime Intellect path to full Recursive Self Improvement. Prime Intellect can touch a lot more of the processes.

Amdahl’s law, and why better coding agents stop mattering

In 1967, Gene Amdahl published a short paper arguing against the enthusiasm for massively parallel computers. His point was almost embarrassingly simple. If you speed up part of a process, your overall gain is limited by the parts you did not speed up.

If 35% of research work can be automated, your maximum possible speedup is 1.54×. Not 8×. Not 100×. And you hit most of that ceiling almost immediately. Going from a 4× faster coding agent to a 100× faster coding agent moves you from 1.41× to 1.53×.

“Claude got dramatically better at writing code” does not mean “AI research got dramatically faster”. Anthropic disclosed in June 2026 that Claude authored more than 80% of the code merged into its production codebase.

Screenshot Screenshot

The thing that actually resists improvement

So what’s in the other 65%?

Break research into stages.
– deciding what problem is worth attacking
– designing the experiment
– implementing it
– waiting for compute,
– analyzing results, writing up.

Implementation has been substantially automated. Compute-waiting is bounded by physics and money and removing large known delays. Although coding directly to hardware can provide large speedups. SpaceXAI is claiming that mapping everything on 220,000 GPUs speeds up training and inference by 10 times. Analysis and writing are partially automated.

The stubborn residual is judgment — choosing which problems matter. Anthropic names it directly that the missing ingredient is not execution but knowing which goals are worth pursuing.

Every improvement loop is a bet that some signal can substitute for human judgment.

However, enough speedup can break ceilings and enable better productivity in other areas. Look below at breaking ceilings.

A July 2026 survey of 1,250 papers on AI self-improvement organizes this into what it calls a verification hierarchy, ordered by how trustworthy the signal is:

Formal verifiers — proof checkers, type systems. Sound by construction. a false improvement cannot get through. Coverage is tiny.
Execution feedback — tests, compilers, benchmarks. Reliable and cheap. Incomplete passing tests underdetermines correctness, and any fixed benchmark eventually gets gamed.
Learned judges — reward models, LLM-as-judge. Broad coverage, capped by the judge’s own competence, and themselves optimization targets.
Intrinsic signals — the model’s confidence, its self-consistency. Cheapest, most gameable.

Screenshot Screenshot Screenshot Screenshot Screenshot

Level shift versus rate change

Screenshot

In July 2026, a startup called Weco AI published what it billed as the first experimental evidence of consistent recursive self-improvement. Their system, AIDE², had an outer AI agent rewriting the harness code of an inner AI research agent. After 100 unattended outer-loop iterations over eight days, it produced agents that beat a baseline Weco’s own engineers had hand-tuned for two years. It designed a novel search algorithm, cut prompt size 16-fold, and — strikingly — built its own defenses against cheating, dropping the inner agent’s reward-hacking rate from 63% to 34%.

Eight days beating two years. It sounds like ignition.

It isn’t, and Weco says so. They grade recursive self-improvement on a four-level ladder, and they claim Level 1 — net positive against human R&D under a fixed budget — while explicitly declining to claim Level 2, which they call ignition: the improved system becoming better at improving.

Level 1 is a one-time payout. It is faster than human or faster than the old way, but it is not a compounding process.

Why 1.5× is HUGE. Historical Context

Everything above sounds deflationary. It shouldn’t be. Here’s the calibration that matters.

Compare to baseline productivity growth. US total factor productivity — the residual that captures how much better we’ve gotten at converting inputs into outputs — grew at roughly 2% per year during the mid-century golden age, and closer to 1% since. A permanent 1.5× improvement in research throughput is equivalent to about forty years of normal TFP progress, delivered as a step function. A 2× is a lifetime of it.

Compare to Moore’s law. The most economically transformative technical phenomenon of the last century was not a 1000× jump. It was roughly 1.4× per year, sustained. Fifty years of that compounding produced everything. A modest factor is not modest when it persists.

Compare to Watt. James Watt’s separate condenser improved steam engine fuel efficiency by roughly a factor of three. That single-digit multiple, applied to the cost of mechanical power, restructured global economic geography over the following century.

Compare to the Green Revolution. Cereal yields roughly doubled to tripled across the developing world between the 1960s and 1990s. A 2–3× on one variable, and a billion people didn’t starve.

Electronic design automation

Since roughly the 1970s, chip designers have used software to design chips. Those chips then run the software that designs the next generation of chips. This is a genuine recursive loop, closed, industrial, and operating continuously for fifty years. Better chips run better EDA tools, which design better chips.

It never ignited.

What it produced instead was Moore’s law and then shifted into GPU parallelization. There were mostly steady, predictable, roughly geometric improvement that held for decades and then began to slow until the GPUs restarted. The recursion was real and the compounding was real, but the loop settled into a stable rate rather than accelerating without bound. The constraints that bound it were physical (lithography, heat, quantum effects), economic (fab capital costs rising faster than transistor counts), and organizational — not a failure of the recursion itself.

Screenshot

The move from CPU to GPU to the LLM data-center age illustrates both the power of recursive improvement and its tendency to settle into constrained, high-but-stable rates of progress until a new architectural or economic regime emerges. The EDA precedent suggests that genuine closed loops can deliver decades of transformative compounding without ever igniting into unbounded takeoff.

But EDA limits for improving CPU was shattered with new multi-core and parallelization.

First unblock was Multi-core and parallelization. Designers used the continuing transistor growth to put multiple cores on a chip instead of making one core faster. This shattered the single-thread bottleneck for workloads that could be parallelized.

Second major unblock was GPUs and massive parallelism. GPUs took the parallelization idea much further. Instead of a handful of complex cores, they offered thousands of simpler cores optimized for throughput, with far higher memory bandwidth.

Peak FLOPS for High-end GPUs frequently delivered 10–100×+ higher single- or half-precision throughput than contemporary CPUs for suitable parallel workloads.

Third wave was LLM / AI-specialized hardware + algorithmic multiplication. The LLM era layers two further multipliers on top of the GPU foundation. Hardware specialization with Tensor cores, lower-precision formats (FP16, BF16, FP8, FP4), high-bandwidth memory (HBM), and high-speed interconnects (NVLink).

Switching to tensor/lower-precision formats alone often delivered ~5–18× higher effective throughput on the same silicon compared with FP32.

Recent datacenter GPU analyses show FP16/FP32 dense performance doubling times around 1.4–1.7 years in some periods — faster than classic Moore’s Law.

Training compute for frontier models grew extremely rapidly. Roughly 4–5× per year (doubling every ~5–6 months) in the deep-learning era, far outpacing pure hardware density gains. This came from more chips in parallel, longer training runs, and more powerful accelerators.

Algorithmic / software efficiency
This is the pure “multiplicative” layer on top of hardware. Studies of language models find that the compute required to reach a given performance level has improved at rates equivalent to doubling effective compute every ~5–14 months in various analyses (substantially faster than hardware alone in many periods).

Cumulative algorithmic gains over roughly a decade (2012–2025)have been estimated in the range of thousands to ~20,000× effective compute for the same performance in some language-model evaluations.

Breaking Ceilings

IBM’s Walter Doherty documented in 1982 that when system response drops below roughly 400 milliseconds, user productivity rises disproportionately. It is linear with the speedup. Above a few seconds, people batch their queries and context-switch away. Below the threshold, they stop planning queries and start exploring. The interaction becomes conversational rather than transactional.

Cerebras is credibly in that regime for inference. Wafer-scale integration keeps weights in on-chip SRAM rather than HBM, which attacks exactly the memory-bandwidth wall that makes decode slow. Thousand-token-per-second inference is a real published capability, not a projection.

If experiment latency drops from hours to seconds, experiment design stops being a careful upfront activity and becomes an exploratory one. You do not design one good experiment, you sweep a hundred bad ones. That doesn’t just compress the compute bucket. It partially eats the design bucket.