“…scale of seven to eight billion … is a sweet spot where you can start to do good interpretability research… you can do zero-shot prompting and get interesting behavior.”

“A 26 billion parameter models can already orchestrate full replication chains ... able to create small $9 billion parameters models that established interface on eight of the nine hardware types”

“Haiku's price jumps 5x if you are sending really large prompts. So for those, you may want to choose a different model.”

“We will license core ai models from really deep resourced company and then train those models on client data ... without having to hire 300 people”

“Mistral Large 4, aka LeChunk, a trillion parameter model trained in two months on 3,800 GPUs in Mistral's own data centers”