“You can go off Megatron ... they take the open source code they make it their own thing and they just ... don't push upstream”

The a16z Show · Can Open Models Keep Up With the AI Frontier? | Nathan Lambert

Play from 45:59
45:59

Audio streams from the publisher’s own feed. Verbatim does not host or copy it.

What was described

Trains models on the open-source Megatron codebase, forks it, and does not push changes upstream

Said nearby on the same subject

“The point is to stop thinking only about takeoff and start thinking about landing… there are three questions… what does finished look like… what work am I pretending I can skip… what work can I move upstream.”
TD
Todd DeWalt
The Construction Leading Edge Podcast ·
How they work Share this moment
“He put me in a 12-week program that I lifted what he wanted me to lift. I trained how he wanted me to train. And within 12 weeks, I ended up not only changing my physique, I changed my life.”
BH
Bryce Henson
The Profit Lifestyle for Fitness Entrepreneurs ·
How they work 12 weeks Share this moment
“You have a lot more control of an open weights model that, or an open source model that you can configure, you know, whether it be through fine tuning or whatever, set up your own guardrails”
NK
Nick Kairinos
AI in the UK ·
Contrarian take Share this moment
“There are pretty strict top-down mandates… and… communities in open source… hybrid model… pockets of command and control… a lot of coordination.”
VP
Vijoy Pandey Cisco
DataFramed ·
Why it worked Share this moment

Moments like this, every Monday

The strongest new moments on your topics, in the speakers' own words. Free for three topics.

No card. No password. One-click unsubscribe.

Track a competitor?

Pro watches up to 25 names or phrases and emails you the morning after they are said.

See Pro, $29 a month

Verbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.