The startup open-sourced its generated research artifacts on GitHub.
Towards a system that discovers knowledge autonomously!
New SOTAs on @karpathy’s nanogpt & nanochat
and @nvidia’s kernel benchmarks.
We are open-sourcing the artifacts of the system and contributing back to the research community!
http://x.com/i/article/2064569799313940481
Users congratulate Recursive on its autonomous AI research system achieving new SOTAs on NanoGPT and NVIDIA benchmarks for the visible progress shown, while some criticize the announcements as vague or overhyped.
AI is now doing our AI research.
At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method.
RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time.
Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three.
These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench.
AI is code and AI can code. The code and ideas that lead to these results were not invented by our team but by the AI system itself.
To do RSI safely, we need to investigate its inventions. That's best done transparently with the community. @Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas.
Link below.
Early results from Recursive 🚀🚀
SotA results from our open-ended knowledge discovery system:
1️⃣NanoChat 5min pre-training (0.9372 bpb -> 0.9109 bpb, 2.8% lower Bits-Per-Byte than long-standing community SoTA)
2️⃣NanoGPT SpeedRun (79.7s -> 77.5s, 2.8% faster than long-standing community SoTA)
3️⃣GPU kernel optimization (Overall 7.8% better than SoTA performance in SOL- ExecBench, hosted by NVIDIA)
To achieve that, our system automatically finds and combines innovations together to create better solutions than current ones carefully designed by expert humans in various domains.
We have open-sourced resulting artifacts found by our system so you can check the output yourself. See a full breakdown and technical writeup:
https://www.recursive.com/articles/first-steps-toward-automated-ai-research
http://x.com/i/article/2064569799313940481
very cool results!
i really believe speedruns by @kellerjordan0 @karpathy and others are a great testbed for RSI like tasks, both for evaluating and improving how models do research
there are still limitations in how current speedruns are designed, and work needed on the system side to make agents efficient. seeing what models can do in math/code, i believe one missing piece to achieve the same for ai research is giving them the right environment
http://x.com/i/article/2064569799313940481
In GPU kernel optimization, our framework achieves overall SoTA in NV's SOL-ExecBench, SoTA in all 4 sub-categories, and outperforms solutions that are (1) designed by GPU human experts, and (2) generated by other AI systems designed by GPU experts.
Our system is general. We don't have in-house GPU experts for now (I am regarded as a kernel "expert" internally😆).
🔍Check the details in NV's official leaderboard https://research.nvidia.com/benchmarks/sol-execbench
AI is now doing our AI research.
At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method.
RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time.
Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three.
These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench.
AI is code and AI can code. The code and ideas that lead to these results were not invented by our team but by the AI system itself.
To do RSI safely, we need to investigate its inventions. That's best done transparently with the community. @Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas.
Link below.
Lots of people have been asking me what we're up to at @Recursive_SI.
We still can't say much quite yet, but we thought we'd share a little early demo.
It turns out our automated researchers are pretty effective at performance optimization!
Excited to show results of the first steps towards automated AI research at @Recursive_SI. The same general system achieved state of the art on @NVIDIAAI's SOL-ExecBench GPU Kernel Optimization, nanoGPT Speedrun, and @karpathy's NanoChat autoresearch benchmarks.
This is probably what Fable and Mythos would want to avoid people doing.
We'll keep at it.
AI is now doing our AI research.
At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method.
RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time.
Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three.
These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench.
AI is code and AI can code. The code and ideas that lead to these results were not invented by our team but by the AI system itself.
To do RSI safely, we need to investigate its inventions. That's best done transparently with the community. @Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas.
Link below.
Excited to share these preliminary results on our internal autoresearch system @Recursive_SI, where we achieve SOTA on nanochat / nanogpt speedrun / kernel benchmarks using the same underlying system without task-specific adaptations.
blog: https://www.recursive.com/articles/first-steps-toward-automated-ai-research
http://x.com/i/article/2064569799313940481
Recursive’s autonomous research system just set new SOTAs on @karpathy’s nanogpt + nanochat and @nvidia’s kernel benchmarks!!
No humans in the loop. The system discovered these gains on its own.
Open-sourcing results today.
http://x.com/i/article/2064569799313940481
First results from Recursive on AI that improves AI! 🚀 ✨ 📈
Our automated AI research system incorporates principles from open-ended and AI-generating algorithms. 🧠 💡🌱🧬
It conducts key parts of the science loop: proposing ideas, implementing them, testing them, and picking the next ideas based on the data. 🔬 🔭 🧪⚗️
The same general system produces state-of-the-art results on three different problems (two on training language models, one on speeding up AI via kernel optimization).
• NanoChat Autoresearch: 1.3x faster to reach the same loss than the best solution produced by an entire community of humans+agents over months, and 1.8x faster than the initial hand-optimized solution
• NanoGPT Speedrun: 3% speedup of a very efficient solution produced by entire community of humans+agents over 2+ years
• GPU Kernel Optimization: 18% reduction in gap to theoretically optimal score on Nvidia’s SOL-ExecBench
These are early tests of our system. We’re very excited for what the future holds!
Post: https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Great work by the amazing team at Recursive!
https://www.recursive.com/articles/first-steps-toward-automated-ai-research
AI is now doing our AI research.
At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method.
RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time.
Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three.
These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench.
AI is code and AI can code. The code and ideas that lead to these results were not invented by our team but by the AI system itself.
To do RSI safely, we need to investigate its inventions. That's best done transparently with the community. @Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas.
Link below.
such a cool benchmark
Faster training to the same quality.
First results from Recursive on AI that improves AI! 🚀 ✨ 📈The same general system produces state-of-the-art results on three different problems (@karpathy's NanoChat autoresearch, NanoGPT speedrun, and kernel optimization). It automates key parts of the science loop: ideation, implementation, & testing.🧵👇
First results from Recursive on AI that improves AI! 🚀 ✨ 📈
Our automated AI research system incorporates principles from open-ended and AI-generating algorithms. 🧠 💡🌱🧬
It conducts key parts of the science loop: proposing ideas, implementing them, testing them, and picking the next ideas based on the data. 🔬 🔭 🧪⚗️
The same general system produces state-of-the-art results on three different problems (two on training language models, one on speeding up AI via kernel optimization).
• NanoChat Autoresearch: 1.3x faster to reach the same loss than the best solution produced by an entire community of humans+agents over months, and 1.8x faster than the initial hand-optimized solution
• NanoGPT Speedrun: 3% speedup of a very efficient solution produced by entire community of humans+agents over 2+ years
• GPU Kernel Optimization: 18% reduction in gap to theoretically optimal score on Nvidia’s SOL-ExecBench
These are early tests of our system. We’re very excited for what the future holds!
Post: https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Great work by the amazing team at Recursive!
Detailed explanation of our first results by our amazing teammate @cong_ml !
Recursive just came out of stealth, and the team has been cooking 🔥
Our first results: an automated AI research system that can improve AI across 3 very different settings across training and GPU kernel optimization.
https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Today we're sharing early results from Recursive’s open-ended knowledge discovery system.
The system automates the full research loop: generating ideas, implementing them, running experiments, validating outcomes, and using those results to decide what to explore next.
We evaluated it on three AI benchmarks, where it achieved state-of-the-art results across all three:
1. NanoChat 5-Minute Pretraining: improved from 0.9372 bpb to 0.9109 bpb, achieving 2.8% lower Bits-Per-Byte than the long-standing community state of the art.
2. NanoGPT SpeedRun: reduced training time from 79.7s to 77.5s, making it 3% faster than the state of the art solution by entire community of humans+agents over 2+ years.
3. GPU Kernel Optimization: delivered 7.8% higher performance than the state of the art on SOL-ExecBench, hosted by NVIDIA.
These results are still early, but they point toward a future where AI research becomes significantly faster and more cost-efficient.
Post: https://www.recursive.com/articles/first-steps-toward-automated-ai-research
http://x.com/i/article/2064569799313940481
A new state of the art in NanoGPT, NanoChat, and SOL-ExecBench from @Recursive_SI's automated AI research system -- all in one post, and all built with @modal.
Recursive just came out of stealth, and the team has been cooking 🔥
Our first results: an automated AI research system that can improve AI across 3 very different settings across training and GPU kernel optimization.
https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Faster training to the same quality.
Some more neat results. Better models within the same time budget
We are open-sourcing solutions from this system so that the community can check them and build upon them: https://github.com/recursive-org/first-steps-toward-automated-ai-research
More details in our blog: https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Excited to show results of the first steps towards automated AI research at @Recursive_SI. The same general system achieved state of the art on @NVIDIAAI's SOL-ExecBench GPU Kernel Optimization, nanoGPT Speedrun, and @karpathy's NanoChat autoresearch benchmarks.
Some more neat results. Better models within the same time budget
https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Official NVIDIA results: https://research.nvidia.com/benchmarks/sol-execbench/leaderboard/collection/1/B200
AI is now doing our AI research.
At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method.
RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time.
Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three.
These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench.
AI is code and AI can code. The code and ideas that lead to these results were not invented by our team but by the AI system itself.
To do RSI safely, we need to investigate its inventions. That's best done transparently with the community. @Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas.
Link below.
