“It actually does caching of prompts so that if you do the prompt before, it doesn't test it again, saves you money”

The Data Stack Show · Re-Air: What Every Developer Needs to Know About Microservices with Mark Fussell, Founder & CEO at Diagrid

Play from 38:16
38:16

Audio streams from the publisher’s own feed. Verbatim does not host or copy it.

What was described

Introduce LLM Conversation API that caches prompts to avoid re-running them

Said nearby on the same subject

“It writes a prompt… submits that prompt to an LLM… the agent program just directly executes whatever it says… and then it loops”
♫
Said on Prof G Markets
Why it worked Share this moment
“We can go into the KV cache of the model and steer these positive or negative directions ... when you steer the zone with negative valence, the model is significantly more likely to avoid that zone”
♫
Said on "The Cognitive Revolution"
Why it worked Share this moment
“We just introduced Dreamforce on-device transcription… it will actually populate your visit based on that conversation and say, they wanted samples… Do you want to send it?”
DK
Devin Kunysz / Salesforce
The CPG Guys ·
“If you come back after one hour and one minute, that cache is gone… cache writes are expensive… leaving sessions idle for more than an hour is expensive.”
DS
Dave Slutzkin
The Startup Podcast ·
Automation 5 minutes; 1 hour Share this moment

Moments like this, every Monday

The strongest new moments on your topics, in the speakers' own words. Free for three topics.

No card. No password. One-click unsubscribe.

Track a competitor?

Pro watches up to 25 names or phrases and emails you the morning after they are said.

See Pro, $29 a month

Verbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.