“You need more pre-training data… give the model a bunch of data and think… transform this data… pre-trained on it would make the model smarter”
♫
Said on "The Cognitive Revolution"
“…my belief-updating paper initially we just used a LLaMA eight-billion and it actually updates belief very dynamically as it’s reading stories…”
Eric Bigelow, Goodfire · "The Cognitive Revolution" · How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning
Play from 49:29
49:29
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
Use a pre-trained 8 B LLaMA model to study belief-updating dynamics in stories across emotions and attributes, without fine-tuning
8 billion-parameter LLaMA
Said nearby on the same subject
“We asked inkling to fine tune itself using tinker. The model wrote its own fine tuning job, ran it and evaluated the result.”
♫
Said on Everyday AI Podcast - An AI and ChatGPT Podcast
“You may need a bit of fine-tuning of your probes so that they work on each person's particular fine-tuned model”
EB
Eric Bigelow
"The Cognitive Revolution" ·
“We have a bunch of fine-tuned models ourselves too… making us more innovative… couldn’t do with the frontier models because of cost and efficiency reasons”
♫
Said on All-In with Chamath, Jason, Sacks & Friedberg
Moments like this, every Monday
The strongest new moments on your topics, in the speakers' own words. Free for three topics.
No card. No password. One-click unsubscribe.
Track a competitor?
Pro watches up to 25 names or phrases and emails you the morning after they are said.
See Pro, $29 a monthVerbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.