“Moving to looped transformers ... would be more problematic than achieving similar capability by increasing parameters”
The a16z Show · Nick Bostrom: How Should We Navigate Superintelligence?
Play from 17:00
17:00
Audio streams from the publisher’s own feed. Verbatim does not host or copy it.
What was described
warn that using looped transformer architectures (more serial depth per token) may increase capability but make interpretability harder compared to scaling parameter count
Said nearby on the same subject
“…we do that through this chain of thought and reasoning and traces… term… serial depth opacity… make sure we can actually still see what the model is thinking”