“What if you could provide logit biases to your tool calls to fine-grainy control when things are inserted or not without poisoning your context?”
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) · Why Jev Is Changing How We Build With AI with Diogo Almeida
Play from 1:20:38
1:20:38
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
apply logit bias to tool-call logits so the model preferentially selects desired tools and avoids context-heavy native tool calls
Said nearby on the same subject
“We're actually able to bias the model based off of the direct user feedback using some of these like online training methodologies. We're able to see like real upticks in user satisfaction, completion rates, decreased tool call failures”
“A separate copy of the model that’s running on the tool calls to judge whether it’s a safe thing to do… or if it is safe, is it what the user asked for”