“…when the transformer came out, it was literally, as soon as the paper came out, literally the next day… it was clear… that transformers address the limitations of recurrent neural networks of learning long-term dependencies.”

“When transformers came out, the paper… quickly converted it into PyTorch… they open source it and went viral on GitHub.”

“Under the hood, it’s a diffusion transformer… transformer ingests the most amount of data… diffusion helps you to refine the outcome to get to the smooth, precise, fine-grained motion”

“You put about 10 or 12 cloves of garlic in it ... after a few days, the microbes in the garlic transform the honey. They take away the hard taste of it.”

“We just like parameterized this with a small neural network and just searched over this decision-making network”