AI · Video
Looped Transformers Add Reasoning Depth Without Extra Model Weights
Prompt Engineering ·
At a glance
This video explains looped transformers, a design where a language model reruns part of itself several times for each token. Using Thinking Machines' open model Inkling 2 Mini as an example, it covers what the extra passes help with, what they don't help with, and their costs.
Key points
- A looped transformer reruns the middle layers of a model more than once for each token.
- Inkling 2 Mini has 24 layers, and its middle 12 run three times, giving 48 layer steps.
- The extra passes add reasoning steps without storing more model weights.
- The video covers what the loops improve, what they do not improve, and what they cost.
Why it matters
Readers curious about how AI models reason with the same parameters may find this useful.
Original source
Looped Transformers: 2x Deeper Without More Weights
Prompt Engineering · Nov 22, 23:00 KST
Watch on YouTube ↗AI summarized the video title and description. Check the original for accuracy. Copyright belongs to Prompt Engineering.
More from this field
- Senator Kennedy Mocks Jensen Huang Affair as Focus Shifts to Chip Export Loopholes
US Senator John Kennedy joked about former Nvidia CEO Jensen Huang's reported affair with a suspected Chinese spy. His main concern was how China seemed to get around US export controls on advanced AI chips.
The Guardian · Nov 23, 04:34 KST - Cartesia's H-Net Tries a Smarter Way to Split Text for AI Training
Cartesia's H-Net is a new way for AI models to handle text, moving past the traditional tokenization step. Instead of splitting text into fixed chunks, it learns to group raw bytes on its own. The video says this can train models much faster on selected tests and help them handle unfamiliar data better.
bycloud · Nov 23, 04:03 KST - OpenAI Proposes Shared AI Development Limits Tied to Safety Milestones
OpenAI's chief scientist Jakub Pachocki proposed that top AI developers agree on shared limits on frontier AI development, tied to measurable milestones. He warned that if pretraining progress speeds up as early as 2027, alignment and monitoring research may not keep pace. OpenAI also said AI has already helped with some model design work.
Unite.AI · Nov 23, 03:43 KST - Why Data Centers Are at the Center of the Big Tech Backlash
Journalist Paris Marx's new book, Hyperscale, examines how data centers and generative AI have become a political flashpoint. In an interview with TechCrunch, he described data center buildouts as unsustainable and said the opposition is part of a wider backlash against tech companies.
TechCrunch · Nov 23, 03:08 KST