“The authors do show that a very small amount of additional post-training on the sparsely activating model is enough to recover a large majority of the performance loss.”

Kurt Tigges · "The Cognitive Revolution" · The Neuron Basis: Goodfire's Curt Tigges on Agentic Interp, Monosemanticity & Reverse Engineering

Play from 2:49
2:49

Audio streams from the publisher’s own feed. Verbatim does not host or copy it.

What was described

performed a small amount of additional post-training on the sparsely activating model to recover performance loss after heavy ablation

Said nearby on the same subject

“In short, Kurt shows that you can ablate roughly 90% of the individual neurons, keeping only those with the strongest activations, with surprisingly minimal performance degradation.”
KT
Kurt Tigges
"The Cognitive Revolution" ·
Why it worked 90% Share this moment
“Showing that the original model's behavior is substantially restored relative to ablating the neuron entirely when the other LLM's guesses are patched in to the model's activations.”
KT
Kurt Tigges
"The Cognitive Revolution" ·
Why it worked Share this moment
“We're still up an additional 2% year over year. So we're actually tracking growth, not a big amount of growth, but a small amount of growth in the trans-border segment.”
♫
Said on Sask Ag Today on GX94
Hard number 2% YoY increase Share this moment
“You are somewhat well known for tweaking the script after you watch the actors perform it”
♫
Said on Q with Tom Power
Why it worked Share this moment

Moments like this, every Monday

The strongest new moments on your topics, in the speakers' own words. Your first brief is free, then USD {{ TIERS.starter.price }} a month for three topics.

No card to start. No password. Cancel any time.

Track a competitor?

Pro watches up to 25 names or phrases and emails you the morning after they are said.

See Pro, $29 a month

Verbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.