“Any in-context persona steer makes the model more agreeable, and all except psychopathy make the model less psychopathic.”
♫
Said on LessWrong (Curated & Popular)
“There are… neurons that seem to track model confidence where you can even like steer and make the model more or less confident generically about whatever concept it's learning.”
Eric Bigelow, research team · "The Cognitive Revolution" · How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning
Play from 14:09
14:09
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
Identified neurons that track model confidence and used them to steer the model toward more or less confidence on a concept
Said nearby on the same subject
“Collect data… fit a small local student model… route to student models when there is high confidence”
♫
Said on Last Week in AI
“We can go into the KV cache of the model and steer these positive or negative directions ... when you steer the zone with negative valence, the model is significantly more likely to avoid that zone”
♫
Said on "The Cognitive Revolution"
“You can set those parameters… if you want them only to merge if you're like 99% or more confident.”
JL
John Lindquist
How I AI ·
Moments like this, every Monday
The strongest new moments on your topics, in the speakers' own words. Free for three topics.
No card. No password. One-click unsubscribe.
Track a competitor?
Pro watches up to 25 names or phrases and emails you the morning after they are said.
See Pro, $29 a monthVerbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.