The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) · Your Ranking Model Is Optimizing the Wrong Thing with Zoe Weil
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
Use internal evaluation harnesses and a library to continuously test ranking models
“I used Phoenix, which is a little evals library that you can set up there to run tests, immense real information, and used a large model as a judge to pick the best model that could do the job.”
“We trained models on internal data and then predicted on external data… total number of tests … like 13 million across 500 protein targets.”
“They're using Pascal for the graph… we give them the entire infrastructure plumbing, and their internal tech team can build specific applications on top or use our application library.”
“What we do is basically to bake in a refusal guardrails into the models we open rights that we test internally”
The strongest new moments on your topics, in the speakers' own words. Free for three topics.
No card. No password. One-click unsubscribe.
Pro watches up to 25 names or phrases and emails you the morning after they are said.
See Pro, $29 a monthVerbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.