Anthropic's Fable 5 achieves a 10x performance gain on Thoughtful Lab’s FrogsGame by autonomously training a weaker model

and this is why Anthropic restricted LLM development for Fable 5

Once in a while you see charts like this for Mythos, and now for Fable. The entire game changed in February when Mythos emerged from training. Imagine what Anthropic has developed internally since then. I think they are on a completely different - invisible - trajectory now.

the heart attack continues w/ Fable 5, we checked there were no reward hacks

I wouldn't read too much into this tbh, I'm kind of negging OpenAI. They were getting too smug for a while; now they're paying for it. But they remain the OG AGI lab. They had two such moments in the history of commercial LLMs: GPT-4 and o1. Dario had *nothing* on o1 either.

The funniest outcome would be if Gemini finally converts their strategic bet (starting with XLand and onwards to Nano-Banana, Veo, Genie 4…) into a performance leap. Dario: scale. Sama: reasoning. Demis: world modeling. It's not just a FLOPS arms race, it's a paradigms clash.

Damn.

@fireandvision From Anthropic's Series C in 2023:

@thoughtfullab How were you not redirected to opus 4.8? Did you use special prompting to bypass Anthropic’s guardrail against llm research work when using fable5?

@AndrewCurran_ "2026–2027 is the critical window in AI. If you're ahead then, the models start getting better than humans at everything, including AI design and using AI to make better AI."

@AndrewCurran_ You’re getting out of your depth. It found a clever pseudo reward hack for that Env that doesn’t generalize

That said, we dislike FrogsGame as a task internally. The frogs know what they did. We're now sprinting toward adding more useful, real-world posttraining tasks, partly out of ambition, partly to put a distance between us and the frogs 🐸

@JoshPurtell @AndrewCurran_

@Potrock_ @AndrewCurran_ Pseudo is the key word here

@JoshPurtell @AndrewCurran_ They looked, they validated, but surely you are right. how many pseudos turn into 1 valid solution?

@Potrock_ @AndrewCurran_ Idk what that means but it just wrote a solution for the env in python and then used STaR to turn it into finetuning data. That’s technically legal but doesn’t generalize

@AndrewCurran_ slaughterbots

@scaling01 recursive self improvement soon

@dog_foot_ruler_ @thoughtfullab That's only going forward. Since release, "frontier llm development" requests would make Fable give a "dumbed down" response. Which apparently is still much better than 4.8 or this wasn't considered frontier/sota

@thoughtfullab Fable 5 made every other frontier look like a side quest