“Stress testing the models ... trying to expose weaknesses ... creates a baseline for the performance of your models”
DS
Dan Stocker
Cyber Risk Management Podcast ·
“The grabbing a drink task is just how we test language following ... you can just specify what you want if the model follows intent follows instruction”
Latent Space: The AI Engineer Podcast · Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy - Jason Ma, Dyna Robotics
Play from 1:06:01
1:06:01
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
Create synthetic task instances (e.g., "grabbing a drink") to stress-test language-following capabilities of their models
Said nearby on the same subject
“We now have a capability where we have an AI competence score factor. So it's not just can you complete the task, but how did you complete the task with AI?”
HB
Hack The Box
Smashing Security ·
“We not only stress test to show families how their retirement plan holds up in a variety of scenarios... when people see that their plan still works when bad things happen, that's when fear starts to fade.”
♫
Said on Retire Right Radio
“If you have 10 tasks in sequence... the ER model has a certain X success rate and the VLA model has a Y success rate, now the sequencing task compounds the error”
♫
Said on "The Cognitive Revolution"
Moments like this, every Monday
The strongest new moments on your topics, in the speakers' own words. Free for three topics.
No card. No password. One-click unsubscribe.
Track a competitor?
Pro watches up to 25 names or phrases and emails you the morning after they are said.
See Pro, $29 a monthVerbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.