“If you have some GPUs, you put a VLM on it and you throw it up on an open router, you kind of have a production inference offering”
Unsupervised Learning with Jacob Effron · Ep 94: Applied Compute CEO on the Limits of RL, the New AI Hyperscaler & Why Post-Training Wins Inference
Play from 33:08
33:08
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
deploy a VLM inference engine on GPUs behind an open router to create a production inference offering
Said nearby on the same subject
“Jev is a router… as I have text, route to the right tool, and then if I can infer from the text, the job to do in the tool, I might as well infer that and route as far down the user journey as I can.”
“The Bedrock team rebuilt your inference engine with six engineers in 76 days, and that would have taken 40 people a year”
♫
Said on The a16z Show
Automation6 engineers, 76 days, would have taken 40 people a yearShare this moment
“I’ve been champing at the bit for weeks… inference without GPUs… this opens an entire new world… Apple’s camera can tell you what’s going on without sending anything outside of itself”