“Pre-training is exponentially more resources for debatably linear gains… seems very sub-linear”
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) · Why Jev Is Changing How We Build With AI with Diogo Almeida
Play from 42:27
42:27
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
Argue that pre-training large language models is sub-linear in returns and not the most efficient use of budget
Said nearby on the same subject
“What we tried to do is, you know, using large language models and AI, we tried to really ground probability of success in an empiric data set, where we use Bayesian statistics.”
“If you can get that information and you can turn that unstructured data into semi-structured data and connect it to the prediction, then you can use the storytelling power of the large language models”
“We try to keep it around 8 to 10% for most of our businesses… we take their marketing budget down when we need to get them more efficient with their calls.”