Saját hibáit másolhatja túl a Sakana AI önfejlesztő modellje

Sakana AI's MASS method lets one language model improve itself by solving tasks, judging results, and refining workflows without a stronger teacher model. The study finds gains across all three roles, but shared blind spots can let errors pass as progress and feed into the next training cycle. Researchers from Sakana AI and UC Berkeley have introduced "Multi-Agent Self-Supervision" (MASS), a method for recursive self-improvement (RSI). A single language model handles the task, optimizes how the work gets done, and evaluates the results. At the research frontier, even experts often struggle to judge results because verification can require costly supporting evidence or lengthy experiments, the authors argue. A single plasma study can take more than 120 million CPU hours, according to the paper.Ad The model itself may therefore be the best available judge, even though it can miss its own mistakes when reviewing its work. Using the same model to solve, check, and revise can reinforce those errors rather than correct them.Ad Inspired by "Society of Mind," a book by AI pioneer Marvin Minsky, the researchers focus on how the model divides up the work. In MASS, a workflow is text appended to the prompt that defines each sub-agent's role and instructions. A "contract" specifies what information each sub-agent must return and which checks it must perform, while a "hop" sets the order of calls. An orchestrator distributes the work among the sub-agents. When testing a trading strategy against historical prices, for example, the data specialist's contract must ensure the strategy uses only information available at each point in time.Ad During the inner loop, the model's weights stay frozen as the team runs a candidate workflow and the evaluator compares the result against the current "champion." The optimizer uses that feedback to propose the next workflow, mainly by changing contracts and hops. In the outer loop, the champion runs 18 times per task, with the model ranking the results itself and selecting up to 15 runs for training. The researchers remove the workflow text from the orchestrator's initial prompt in the training examples so the model learns to organize work based on the task alone. Specific assignments to sub-agents remain in the recorded traces, and the model takes on all three roles again after LoRA fine-tuning.Ad The base model is Qwen3.6-27B, with Qwen Code as the harness. The researchers use twelve research tasks generated by GPT-5.5 across finance, robotics, and pharmaceuticals, reserving three for testing and nine for training. Only eight yield training data because the search finds no workflow that beats the reference on one training task.Ad On the test tasks, the win rate against the base model rises from 53.9 percent after one round to 69.9 percent after two. On MLR-Bench, DSBench, ScienceAgentBench, and AstaBench, the model reaches 1.2 to 1.6 times the base model's score per output token. Gains in raw scores are sometimes small, with DSBench improving only from 0.473 to 0.482, while scores drop slightly on Terminal-Bench 2.0 and SWE-bench Verified. The paper's appendix shows that completed MLR-Bench submissions rise from 31 to 50 out of 60, while their average grade falls from 3.07 to 2.88. The delegation tool goes unused in all 180 runs, leaving it unclear whether teamwork explains the gains. Those gains are no longer statistically significant when results are compared task by task. The authors describe mixed results across research tasks, with AstaBench showing the same pattern. Although the model trains only on solution traces, its performance in the other two roles improves as well. The self-evaluator's agreement with external judges rises from 73 to 93 percent. Workflows found after one round beat the base model's workflows 64 percent of the time, even when the base model runs both. Without workflow instructions, the base model delegates to sub-agents in just one of 110 runs, compared with 87 af