RSI becomes the new AGI, equally elusive to define

The term "recursion" has become the latest catchword in artificial intelligence. Two distinct startups have adopted it as their name, and many others now reference Recursive Self-Improvement (RSI) in their development plans. Much like AGI before it, RSI has turned into a three-letter shorthand for a dramatic AI breakthrough — even if its exact definition still sparks some debate.

At its core, RSI describes an AI system capable of continuously upgrading itself. Once these systems can handle the improvement cycle more effectively than humans, the process becomes a closed loop, constrained only by available computing power, with human involvement becoming unnecessary or even counterproductive.

Whether unsettling or not, that vision is one many AI labs are eager to pursue.

Earlier this month, renowned AI researcher Richard Socher launched a company aptly named Recursive Superintelligence, with RSI as its explicit mission. "Our primary focus is building truly recursive, self-improving superintelligence at scale," Socher told TechCrunch at launch. "This means the entire process of generating, implementing, and validating research ideas would be automated."

Several other prominent researchers are chasing the same goal, hoping for a breakthrough that would make recursive self-improvement a reality.

One of the most notable is Alex Karpathy, a legendary figure from Tesla and OpenAI, who is using agent swarms to train LLMs on simple tasks for a project called Auto-Research. Karpathy has been unusually transparent about the project, regularly tweeting milestones and making the building blocks available via a public GitHub repo. So far, the work has largely focused on making minor improvements to a GPT-2 scale model — as Karpathy noted in March, "It’s not novel, ground-breaking 'research' (yet)" — but it has been enough to convince many other researchers to pursue the RSI dream. With Karpathy now working on pre-training at Anthropic, he will have ample opportunity to apply the concept at a larger scale.

Adaption — founded by Cohere and Google alum Sara Hooker — recently launched a similar tool called AutoScientist to automate frontier training. Like Karpathy’s auto-researchers, the system trains agents to make incremental improvements, but for Adaption the goal is to simplify the training of a full-scale frontier model. If those same researchers start pushing the frontier forward, the system could quickly evolve into something very close to RSI.

Disarray founder Doris Xin drew more specific RSI interest when her self-trained machine learning agent took home 28 medals in a recent Kaggle competition, outperforming many human-trained agents. From her perspective, the main challenge is reliability.

"I would argue that with infinite compute and an infinite time horizon, we are already there," Xin told me. "I want to argue that this isn’t really a creative endeavor. It’s mostly solid engineering work."

Not there yet

There is also plenty of evidence that the AI industry is far from meaningful recursive systems — and still struggling to communicate its progress to a wary public. Google CEO Sundar Pichai essentially admitted this in a recent podcast interview.

"It's a continuum, and we are all definitely making progress," Pichai said. "But the way people describe R.S.I., that would represent a next level of acceleration with many implications, and we aren't quite there yet."

Yet that continuum includes a great many self-improving AI systems. In January, one of Anthropic’s lead programmers for Claude Code estimated that "close to 100%" of his team’s code was written by the tool — a frank admission that Claude Code was literally writing itself.

Just because engineers use an AI tool doesn’t mean it can replace them — but Anthropic appears to be getting close to replacing engineers as well. In a recent survey tied to the Mythos preview, five out of 18 Anthropic engineers believed that with harness improvements, this version of Mythos could soon substitute for an L4 engineer — a mid-level programmer who can handle complex projects without supervision.

Still, some expected weaknesses remained.

"Some of Claude’s major reported weaknesses compared to an L4 include: self-managing week-long ambiguous tasks, understanding org priorities, taste, verification, instruction-following, and epistemics," the report reads.

In other words, its weaknesses encompass everything related to self-direction, which is the foundation of RSI. But for everything else, Claude is ready to step in.

Just like the AGI term before it, the AI industry also cannot tell us how far it is from demonstrating a meaningful recursive system. When Georgetown’s Center for Security and Emerging Technology assembled a group of experts to study RSI last year, the group found a major split in assessments — some predicted an imminent "superintelligence" explosion, while others expected slower progress and eventual plateauing. But all agreed that recursion made the future especially difficult to predict.

Helen Toner, director of CSET and a former board member at OpenAI, told TechCrunch that simply using AI tools to conduct AI research isn’t sufficient to qualify as RSI. "They’re just using AI as much as they can," Toner tells TechCrunch. "And I think that is different from the classic definition of RSI, which really means no humans are needed."

Toner points to a recent post by METR’s Ajeya Cotra, which distinguishes different milestones on the path to AI research takeover. One step, which Cotra calls "adequacy," would occur when the system can still perform research after all humans are removed — even if the resulting research is less valuable or efficient. "Parity" comes when an AI-only system is as good at research as a human-only system. "Supremacy," the final stage, comes when an AI-only system outperforms a collaborative system of humans and AI.

Ultimately, Cotra concludes that AI is very close to the adequacy threshold of being able to produce some work on its own — similar to the incremental changes made by Karpathy’s Auto-Research system. "I wouldn’t be totally shocked if you told me this milestone had already passed, and I expect it to happen in the next couple years," Cotra writes.

She is less certain about when parity will occur, but once it does, she believes it would "massively accelerate the pace of AI progress, leading to AI research supremacy within another year."

Bumps in the road

Given how much of AI is built on scaling laws, there is a strong tendency to assume RSI will follow the same curve. Toner thinks many of those pursuing AI research and development through RSI "think of it as a pretty smooth ladder, where you can just keep scaling up."

But even if AI researchers can make incremental improvements like Karpathy’s auto-researchers, larger challenges remain in handing off the entire research process. Toner frames it in terms of computing history, where humans have handed off more and more of the process while still directing things from the top.

"We went from machine languages to assembly language and compiled languages; you’re getting further and further from the guts of the computer," Toner says. "But the human is still, in some intuitive sense, running the show."

Moving beyond that paradigm will require significant challenges, both in engineering and alignment. But even with massive investments, infinite compute is not available — and the fundamental tradeoff between human labor and machine intelligence will be hard to overcome.

As for a fully recursive AI system of apocalyptic visions? The only thing researchers largely agree on is that, like AGI, it is not here yet.