“Surewire allows you to actually do the testing of the AI ... using a system called LM as a judge ... it's a blue ocean.”
SaaS That App - Building Tech-Enabled Businesses · $20M, Zero VC in 20 Years: The Bootstrapped AI Playbook
Play from 38:36
38:36
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
launched a beta AI-assurance product called Surewire that uses a language model as a judge to generate dynamic test scenarios
Said nearby on the same subject
“I used Phoenix, which is a little evals library that you can set up there to run tests, immense real information, and used a large model as a judge to pick the best model that could do the job.”
“It was less common back then to manage to an equity beta or a range as more of a form of dynamic risk management. That's what TPA is bringing into the system.”
♫
Said on Capital Allocators - Inside the Institutional Investment Industry
“We’re mainly in Azure Shop using Azure OpenAI… testing in GCP with Vertex AI, also using their Anthropic models because we can use GCP as the broker, and we know that the data will stay with us”