“We audit how we assess understanding of the outputs that were produced... if they can't explain why they built the model that way, that's what we're looking for.”
Unsupervised Learning with Jacob Effron · Ep 94: Applied Compute CEO on the Limits of RL, the New AI Hyperscaler & Why Post-Training Wins Inference
Play from 48:45
48:45
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
In interviews, ask candidates to explain why they built AI-generated outputs the way they did, auditing understanding of tool usage
Said nearby on the same subject
“I think we've seen a pattern of employees being evaluated on AI usage… tracking usage of tools is the absolute wrong way of going about evaluating output.”
“We do audit logs, but now in the agentic world, we have to also give more insights why an agent arrived to a decision. What's the evidence they use? And when did they use what evidence? It's traceability and explainability.”
“Ship Fusion wanted to give operators a better way to understand how their fulfillment experience actually compares to the rest of the market. So they placed and audited hundreds of actual real life DTC orders”