“I didn't realize that even though the model is running locally, it can be slower than a round trip to a model out there.”

The InfoQ Podcast · The Future of Web Apps: Will AI Agents Generate UIs on the Fly?

Play from 19:44
19:44

Audio streams from the publisher’s own feed. Verbatim does not host or copy it.

What was described

asserted that local-first models are not always lower latency than remote APIs

Said nearby on the same subject

“Closing the first meeting with an assertive assumption rather than asking if the prospect has any questions increases the chances the prospect will schedule a second meeting”
CS
Chris Smith
Financial Advisor Success ·
How they work Share this moment
“We've distributed easily more than 30-40 million dollars worth of compute access and credits for different API models”
♫
Said on Money Moves Podcast: Meet the Future of Finance with Samantha Lewis, Ian Epstein, and David Sutter
Data play $30-40 million worth of compute credits Share this moment
“Latency is like 20 to 30 milliseconds for first read on hard disk… on SSD it gets down to closer to SSD latencies… 30 microseconds, 50 microseconds.”
PM
Peter Mattis
The Pragmatic Engineer ·
Hard number 20-30 ms on HDD, 30-50 µs on SSD (NVMe) Share this moment
“The call we had because he was remote. We did a Google Meet call. It lasted less than two minutes… I had a script… went through all the details… it was very quick.”
TM
Tyson Mutrix
Maximum Lawyer ·
How they work 2 minutes Share this moment

Moments like this, every Monday

The strongest new moments on your topics, in the speakers' own words. Free for three topics.

No card. No password. One-click unsubscribe.

Track a competitor?

Pro watches up to 25 names or phrases and emails you the morning after they are said.

See Pro, $29 a month

Verbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.