“Agents have a distress button... I had to calibrate it because they would get just a little too eager to push it”

Alan Chan · "The Cognitive Revolution" · Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI

Play from 2:09:09
2:09:09

Audio streams from the publisher’s own feed. Verbatim does not host or copy it.

What was described

Calibrate agents' distress button so it only triggers on genuine ethical dilemmas, preventing over-eager activation

Said nearby on the same subject

“Solidus Labs started off as basically technology to prevent kind of bad actors on crypto exchanges… they apply their technology to prediction markets to prevent bad activity there.”
♫
Said on How I Invest with David Weisburd
Why it worked Share this moment
“We create what we call digital dilemmas. And these digital dilemmas allow students to just really critically think, would I use AI in this case or would I not and to what capacity?”
♫
Said on This Week with EdSurge
Why it worked Share this moment
“They are not perfectly calibrated, but they are way, way more calibrated than RLHF models are”
♫
Said on The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Hard number Share this moment
“‘a monitor actually observes in real time an action an agent is doing, it decides to block it… would have helped and prevented the actual incident’”
AC
Alan Chan GovAI
"The Cognitive Revolution" ·
How they work Share this moment

Moments like this, every Monday

The strongest new moments on your topics, in the speakers' own words. Free for three topics.

No card. No password. One-click unsubscribe.

Track a competitor?

Pro watches up to 25 names or phrases and emails you the morning after they are said.

See Pro, $29 a month

Verbatim quotes a short extract and links to the episode; the episode belongs to its publisher. Remove a show.