“You may need a bit of fine-tuning of your probes so that they work on each person's particular fine-tuned model”
EB
Eric Bigelow
"The Cognitive Revolution" ·
“Fine-tuning LoRa, rank 16, on 2,000 rows per arm, one epoch… 50 answers to each of 16 held-out questions at temperature 1, giving 800 answers per run.”
LessWrong (Curated & Popular) · "Personas in a Pre-Internet Language Model" by David Africa
Play from 4:44
4:44
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
fine-tuned models with LoRa adapters (rank 16) on 2,000 rows per arm, one epoch, using BF16 precision
4 training seeds per arm, 50 answers per 16 held-out questions at temperature 1 (800 answers per run)
Said nearby on the same subject
“Model portfolio is 16.4% per annum versus the SPDR200, 7.5%... QAV light portfolio is 21.8% per annum versus 9.2%”
♫
Said on Quality At Value - Value Investing In Shares
“We asked inkling to fine tune itself using tinker. The model wrote its own fine tuning job, ran it and evaluated the result.”
♫
Said on Everyday AI Podcast - An AI and ChatGPT Podcast
“We have a bunch of fine-tuned models ourselves too… making us more innovative… couldn’t do with the frontier models because of cost and efficiency reasons”
♫
Said on All-In with Chamath, Jason, Sacks & Friedberg
Moments like this, every Monday
The strongest new moments on your topics, in the speakers' own words. Free for three topics.
No card. No password. One-click unsubscribe.
Track a competitor?
Pro watches up to 25 names or phrases and emails you the morning after they are said.
See Pro, $29 a monthVerbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.