AI · Video
Cartesia's H-Net Tries a Smarter Way to Split Text for AI Training
bycloud ·
At a glance
Cartesia's H-Net is a new way for AI models to handle text, moving past the traditional tokenization step. Instead of splitting text into fixed chunks, it learns to group raw bytes on its own. The video says this can train models much faster on selected tests and help them handle unfamiliar data better.
Key points
- H-Net learns to group raw bytes dynamically instead of using fixed tokens.
- On selected benchmarks, it reaches up to 10x greater training efficiency.
- It may also help language models generalize to data they have not seen before.
- The approach could extend to other data types beyond text.
Original source
The current tokenization is suboptimal, and we just got a solution.
bycloud · Nov 23, 04:03 KST
Watch on YouTube ↗AI summarized the video title and description. Check the original for accuracy. Copyright belongs to bycloud.
More from this field
- Senator Kennedy Mocks Jensen Huang Affair as Focus Shifts to Chip Export Loopholes
US Senator John Kennedy joked about former Nvidia CEO Jensen Huang's reported affair with a suspected Chinese spy. His main concern was how China seemed to get around US export controls on advanced AI chips.
The Guardian · Nov 23, 04:34 KST - OpenAI Proposes Shared AI Development Limits Tied to Safety Milestones
OpenAI's chief scientist Jakub Pachocki proposed that top AI developers agree on shared limits on frontier AI development, tied to measurable milestones. He warned that if pretraining progress speeds up as early as 2027, alignment and monitoring research may not keep pace. OpenAI also said AI has already helped with some model design work.
Unite.AI · Nov 23, 03:43 KST - Why Data Centers Are at the Center of the Big Tech Backlash
Journalist Paris Marx's new book, Hyperscale, examines how data centers and generative AI have become a political flashpoint. In an interview with TechCrunch, he described data center buildouts as unsustainable and said the opposition is part of a wider backlash against tech companies.
TechCrunch · Nov 23, 03:08 KST - Binding of Isaac Drops AI Music from Repentance Plus After Backlash
Binding of Isaac: Repentance Plus was review-bombed on Steam after a Reddit user found AI-generated music in a Grand Theft Auto parody section. Creator Edmund McMillen said he had no idea AI music was used. The studio then updated the game and removed the generated music files.
The Verge · Nov 23, 02:25 KST