“I have on my website BEP Research, cost per completed task… benchmarks based on the model and the vertical”

Monetary Matters with Jack Farley · Token Bill Dwarfs CPU Bill | Ben Pouladian on Meta's Muse, Anthropic S1, and Why All Roads Still Lead to NVIDIA

Play from 51:30
51:30

Audio streams from the publisher’s own feed. Verbatim does not host or copy it.

What was described

tracks cost per completed task by model and vertical on his BEP Research site to benchmark AI spend

Said nearby on the same subject

“I would do cost by app… or cost by model… I track spend and efficiency by model, so devs can look and see what’s the best model per their workload”
JW
Jason Ward
Agentic Conversations (formally mlops.community) ·
“We now have a capability where we have an AI competence score factor. So it's not just can you complete the task, but how did you complete the task with AI?”
HB
Hack The Box
Smashing Security ·
Why it worked Share this moment
“We have an algorithm that will determine the best model… for 99% of the tasks… we spend $15-$5 million in AI-related spend… frontier models are used only for supervision”
♫
Said on Prof G Markets
“The idea generally that we've taken so far is that it's going to be kind of this cost-per-review model. So you're spending into this retargeting structure.”
♫
Said on Ecommerce Braintrust

Moments like this, every Monday

The strongest new moments on your topics, in the speakers' own words. Free for three topics.

No card. No password. One-click unsubscribe.

Track a competitor?

Pro watches up to 25 names or phrases and emails you the morning after they are said.

See Pro, $29 a month

Verbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.