“If we take SFT self-distillation… that's actually more or less a step of RL… iterating this process ends up with a process quite similar to RL… avoids very elaborate infrastructure and expensive rollouts.”

"The Cognitive Revolution" · AI:AM: A Level We Shouldn't Pass? Notes from The Curve + Tokens vs. Salaries & Is SaaS Cooked?

Play from 1:13:45
1:13:45

Audio streams from the publisher’s own feed. Verbatim does not host or copy it.

What was described

Proposed using self-distillation SFT (combining multiple teachers) as a cheaper alternative to full RL rollouts for enterprise model customization

Said nearby on the same subject

“I use their flash version of it… I don’t have as much memory as I need to run the full guy, but five-three flash is really my main local model”
♫
Said on Security Now (Audio)
“On the government side, almost everybody now is making, you know, in proposal phases, at least document what you used it for, which model you use.”
KP
Karine Pagano Talent Rewire
The Mindtools L&D Podcast ·
How they work Share this moment
“Human in the loop combination where you can combine human experience, customer orientation and empathy with the AI efficiency”
FS
Fall Line Specialty
Profiles in Risk ·
How they work Share this moment
“Our model grader identified 10 rollouts, or approximately 1 to 2 rollouts per 1,000, Bayesian mean posterior rate of 0.17%, 95% interval of 0.09% to 0.28%”
♫
Said on LessWrong (Curated & Popular)
How they work 10 rollouts identified, ~0.17% rate (1-2 per 1,000), 95% CI 0.09-0.28% Share this moment

Moments like this, every Monday

The strongest new moments on your topics, in the speakers' own words. Free for three topics.

No card. No password. One-click unsubscribe.

Track a competitor?

Pro watches up to 25 names or phrases and emails you the morning after they are said.

See Pro, $29 a month

Verbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.