“The model can write itself code ahead of time… it’s like, hey, I’m going to do these next five steps… makes a huge difference for speed and token efficiency”
Andrew Ambrosino · The Stack Overflow Podcast · Don't forget about the UI for AI
Play from 17:30
17:30
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
use a harness that lets the model write its own code ahead of time to batch multiple steps into a single inference
Said nearby on the same subject
“The trick is just externalize to a file system… force everything to run through code, you know, trust the model that can write code and it's going to write its own harnesses itself”
“You will write ahead logs in Postgres or binlog in MySQL… replay the changes into a new system when the new schema… essentially zero application code changes.”
SW
Sahil Walia
Software Engineering Radio - the podcast for professional software developers ·