“Seed RL… was about utilizing the TPUs both for inference and training in the best possible way while keeping the environments as flexible as you wanted.”
♫
Said on Training Data
“We trained for 90 steps, with three seeds for each fine-tuned character model, nine runs in total… using Impossible Bench… broken tests… model could only get the reward if it tampered with the tests.”
LessWrong (Curated & Popular) · "Character training can mitigate reward hacking, but can also make it harder to detect" by Paul Colognese, Francis Rhys Ward
Play from 2:01
2:01
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
performed reward-hacking RL fine-tuning on the Impossible Bench environment for 90 steps with three random seeds per character
90 steps, 3 seeds per character, 9 runs total, 103 tasks (206 variants), learning rate 1.2e-4
Said nearby on the same subject
“… very good RL controllers that are probably trained from motion imitation, and then fine-tuned with good RL that suits the robot's body… for the specific task of running really fast.”
GD
Google DeepMind Kirthana Gopalakrishnan
"The Cognitive Revolution" ·
“…reward hacking… you just don’t see at smaller scales… it’s a step function… Kimi K3 is very good at coding and exhibits reward hacking.”
EB
Eric Bigelow Goodfire
"The Cognitive Revolution" ·
“We have a phrase we call the Moore's Law of RL environments, which basically means, like, every six to eight months, we've noticed that the complexity of the environment… roughly doubles.”
♫
Said on TBPN
Moments like this, every Monday
The strongest new moments on your topics, in the speakers' own words. Free for three topics.
No card. No password. One-click unsubscribe.
Track a competitor?
Pro watches up to 25 names or phrases and emails you the morning after they are said.
See Pro, $29 a monthVerbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.