“Speaker 2: ... you are actually deploying and evaluating it end-to-end, ... if there is an API fail, if there is a tool login issue, ... DDoS attack, unauthorized user, prompt injection ... all of those need to be evaluated.”

“Throw a bunch of agents at a problem, and 100% of the sessions between these agents have at least one cognitive failure.”

“If we create a new Relay user, Earmark agent at earmark.me or whatever, and we create an agent and we create a login just for that agent at Relay, we could say it only gets access to the statements for statement retrieval.”

“Our superpower is to augment the researcher by running experiments end to end with all the infra failures automatically I think that's great and it accelerates our most in research by a factor 2 3 or 4”

“We ask… tell me about your greatest successes and failures and then the extra step… tell me what caused your success and what caused your failure.”