“You need more pre-training data… give the model a bunch of data and think… transform this data… pre-trained on it would make the model smarter”
♫
Said on "The Cognitive Revolution"
“We trained the final model on the SpaceX cluster ... a few weeks of pre-training, then synthetic data NRL and then shipping it”
Misha Laskin, Beam · No Priors: Artificial Intelligence | Technology | Startups · Beam: The Great American Open Model with ReflectionAI Co-Founder and CEO Misha Laskin
Play from 1:02:47
1:02:47
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
trained the final model on a SpaceX compute cluster received in July, doing a few weeks of pre-training followed by synthetic data NRL and then shipping it
Said nearby on the same subject
“We did clustering of building blocks on similarity and avoided entire clusters from being overlapped between test and train… performance drops when we do that”
♫
Said on Latent Space: The AI Engineer Podcast
“We prioritized… a strong pre-training base… then amplified it with reinforcement learning at what we believe to be the largest scale… 10,000 GB-300 for four weeks run documented yet.”
ML
Misha Laskin ReflectionAI
No Priors: Artificial Intelligence | Technology | Startups ·
“We then collected those corrections and then basically fed them back into the training data, retrained the models. They then got better.”
♫
Said on The Leadership in Insurance Podcast - Insurtech & Innovation
Moments like this, every Monday
The strongest new moments on your topics, in the speakers' own words. Free for three topics.
No card. No password. One-click unsubscribe.
Track a competitor?
Pro watches up to 25 names or phrases and emails you the morning after they are said.
See Pro, $29 a monthVerbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.