Arcpize and ARC-AGI in 2025: What Progress Toward General (Broad) Intelligence Really Looks Like
In the world of artificial intelligence, it is easy to get the impression that everything has already been solved. It is hard to deny the impressive abilities of models that write code, provide medical advice, and pass mathematics competitions. And yet there is one benchmark that remains a genuine frontier beyond reach. We are talking about the ARC-AGI family of tests, designed to assess what comes closest to “true” intelligence: the ability to create new rules on the fly, learn from a small number of examples, and solve problems that do not exist in any training data.
In recent years, alongside the ARC benchmark itself, the term Arcpize has also emerged. It is an informal term for methods that combine several approaches at once: iterative solution refinement, synthetic thinking, concept libraries built during operation, and evolutionary techniques. Arcpize is not the name of an official architecture. Rather, it is a way of thinking about how to make modern AI models truly reason instead of merely predicting the next tokens. In practice, the best results on ARC-AGI in 2025 come precisely from systems with an arcpizelike character: hybrid, unconventional, and repeatedly iterating on their ideas.
To understand why this is needed, it is worth starting with what ARC actually is.
Table of contents
ARC-AGI: A benchmark that defies scaling
ARC-AGI (Abstraction and Reasoning Corpus for AGI) is a set of visual puzzles that look trivial: colored grids ranging from a few to several dozen pixels in size, several input examples and their correct outputs, and one new input for which the model must generate the correct output. In theory, the task sounds like a classic logic puzzle. In practice, it is a test of deep abstraction.
.
Each puzzle is unique, and its logic is built around symbols defined locally. The model has never seen these rules before, so it cannot rely on training memory. This fundamentally distinguishes ARC from benchmarks such as MMLU, HumanEval, or Big-Bench, where models draw on billions of patterns stored during training.
In 2025, ARC-AGI-2 is the version used most commonly. This iteration introduces cost and construction constraints to limit brute force. It focuses not only on the percentage score, but also on efficiency, namely computational costs per task. A human solves such puzzles for the price of a cup of coffee. Frontier models can “spend” ten, twenty, or even thirty dollars on a single problem and still achieve scores in the range of a dozen or so percent.
Why Are Modern LLMs Performing So Poorly?
Models such as GPT-5, Claude 4.5, Sonnet 4.5, and Grok 4 are impressive when they have lots of text, context, and the ability to carry out a lengthy chain of reasoning. Unfortunately, a large part of their “reasoning” stems from broad text statistics rather than from an abstract thinking mechanism.
On ARC-AGI-2, this looks very clear:
- pure LLM models (without additional techniques) usually achieve 0–3 percent;
- models using Chain-of-Thought achieve 10–18 percent;
- the most advanced bespoke systems based on iterative refinement reach approximately 30 percent.
The results show that contemporary generative AI is excellent at reproducing patterns, but still has enormous difficulty creating new rules.
https://arcprize.org/leaderboard
What are CoT, refinement, and arcpizowe methods?
Chain-of-Thought (CoT)
This is a technique that asks the model to “think aloud.” In typical prompting, you add the instruction “think step by step.” As a result, the model produces a sequence of justifications. This significantly helps with mathematical or logical tasks, but often leads to hallucinations and an explosion in computational costs.
On ARC, CoT provides a moderate improvement, but it is not a panacea. At a certain point, the model starts to get lost in the excess of steps.
Refinement
Refinement is the idea of not expecting the model to produce a perfect answer immediately. Instead, an iterative loop is built: the model generates several initial ideas, the system tests them on \training data, and then asks the model to improve and develop these candidates.
It’s somewhat like designing with a machine that can improve itself after every iteration. The best systems perform hundreds of such iterations during a single task. It’s extremely costly, but effective.
Evolutionary Computing and Program Synthesis
The arc-puzzle approach uses ideas inspired by evolution. Instead of generating a single solution, it generates many candidates, evaluates them, and retains only the best ones. During subsequent iterations, the candidates mutate, combine, or learn new operations. That is exactly how the best solutions to ARC-AGI are created.
Program synthesis, in turn, is a technique in which the system generates not only a solution to a specific problem but also builds a library of functions or concepts that it can use later. Over time, such a library becomes a resource of structured knowledge that can be reused.
.
This is precisely the essence of arcpizing: the system not only generates a response but also develops its own conceptual framework.
Who Are J. Berman and E. Pang—and Why Do Their Systems Win?
Most would expect the top of the ARC-AGI leaderboard to belong to the largest labs. Yet in 2025, it is dominated by independent researchers working with their own pipelines.
Jeremy Berman
He is a researcher known for Reflection AI. His system achieved approximately 29.4 percent on ARC-AGI-2 and approximately 79.6 percent on the older ARC-AGI-1. The most interesting aspect of his work is that it uses evolutionary test-time compute. The system generates hundreds of solutions, tests them, selects the best ones, and continues refining them. Surprisingly, it turned out that generating ideas in natural language works better than generating them in Python.
Berman makes most of his methods publicly available, from blogs to code, allowing his techniques to quickly become an inspiration to others.
E. Pang
E. Pang (often signing as epang080516) takes second place with a score of approximately 26 percent. His approach is based on concepts from DreamCoder. While solving tasks, the system develops its own library of functions and ideas. As a result, it improves not only within a single puzzle but also throughout the entire process. Pang stands out because his solutions are significantly less computationally expensive.
It is precisely from such work that the concept of arcpizing emerged: a creative, agile, iterative approach to ARC.
What does the ARC-AGI leaderboard look like at the end of 2025?
On ARC-AGI-1, the best models can reach 70–80 percent, while Berman’s system can exceed 79 percent. Humans achieve nearly a perfect score. The difference between humans and algorithms is noticeable, but not dramatic.
The situation changes when we look at ARC-AGI-2. Here, the results decline:
- J. Berman: approximately 29%;
- E. Pang: approximately 26%;
- GPT-5 Pro and GPT-5.1 Thinking: approximately 17–18%;
- Grok 4 Thinking: approximately 16%;
- Claude Sonnet 4.5: approximately 13–14%.
An ordinary person achieves between 77 and 98 percent, depending on their level of education. This is a gulf we will not bridge with another neural layer.
Chart of AGI Model Performance and Costs
The ARC-AGI-1 chart serves as a map of the capabilities of today’s reasoning systems. The vertical axis shows performance, while the horizontal axis shows the cost per task in dollars. The upper-left corner contains the Grand Prize zone—the ideal combination of a high score and low costs, which no one has achieved yet. The distribution of points shows which approaches perform best and which rely on the power of pure scaling.
The highest orange points belong to Jeremy Berman and E. Pang. These are bespoke systems based on iterative refinement and program synthesis that understand ARC tasks not statistically, but structurally. Berman achieves the highest score, while Pang is cheaper; both confirm that the key to ARC lies beyond classic LLMs.
.
Nearby, though at enormous cost, is o3-preview (Low), OpenAI’s frontier model. It can reach the low-to-mid seventies, but consumes hundreds of dollars per task. This is a good example of the limits of the “more compute can solve everything” approach.
Below, sequences of points representing OpenAI model families are shown: GPT-5, GPT-5.1 Thinking, o3-Pro, and o4-mini. Their positions form logical trajectories: the larger the reasoning budget, the better the result, but also the higher the cost. GPT-5.1 Thinking (High) is the most balanced, achieving approximately seventy percent at moderate costs.
The Grok family is also included. Grok 4 in Thinking mode scores in the vicinity of the low sixties, making it xAI’s most effective model. More advanced, yet still conventional systems, such as Claude Opus 4, TRM, and HRM, come in at around thirty percent. Lower down are Qwen, Llama, and other inexpensive models, which perform poorly but are highly cost-effective.
The chart leads to several clear conclusions. The highest performance is achieved by iterative systems, not language models alone. Frontier models can reach high scores, but they are not efficient. Open source remains far behind. Above all, ARC-AGI-1 shows that true reasoning does not result from model size, but from the right strategy for acting, testing, and correcting one’s own hypotheses. This is not a ranking, but a compass for the future of artificial intelligence.
Why Is ARC-AGI Still “Unbeaten”?
For three reasons.
First of all: every problem is unique
ARC does not allow the use of training memory. No statistical data will help, because each task has its own previously unknown logic.
Secondly: rules are symbolic
Neural networks are great when patterns are continuous and statistical. ARC requires reasoning along the lines of: “a green pixel surrounded by blue pixels three times in a row means moving the object two squares to the right.” This is symbolic reasoning, which is difficult for neural networks to perform in their current form.
Third: Lack of scalability
ARC-AGI-2 is designed so that performance does not simply increase with model size. New ideas are needed, not larger GPUs.
What Does arcpize Tell Us About the Future of AGI?
The most important discovery of the past two years is this: progress today does not come from increasing parameters, but from the intelligent use of models during operation. Arcpizowe systems show that an LLM can be more than a token predictor. It can be a concept generator and an element of a larger structure.
Trends that will likely shape the next decade of AGI research:
- iterative refinement with cost control;
- evolutionary test-time compute;
- libraries of concepts synthesized during operation;
- combining language models with symbolic reasoning mechanisms;
- multi-agent systems in which models evaluate and modify their ideas.
Future AGI may not be a single model with a trillion parameters. A dynamic, modular architecture based on evolutionary exploration of the space of ideas is more likely.
Summary
Arcpize is not an algorithm, a protocol, or a new architecture. It is a way of working on tasks that require genuine reasoning. It involves iterating, testing, refining, and developing your own library of ideas. It is precisely through such approaches that independent researchers outpace powerful laboratories on the world’s most prestigious AGI benchmark.
As of November 2025, ARC-AGI-2 remains unsolved. The Grand Prize awaits a system that exceeds 85 percent. At present, no one is even close to half that level. However, the pace of change is rapid: just a year ago, 3 percent was considered a success. Today, 30 percent is the standard for the best ARC-AGI systems.
This is a clear signal that real progress toward AGI no longer comes from scaling, but from new structures of thought. And that means the most groundbreaking ideas are still ahead of us.