AI · Article
llama-modchip Runs 284B DeepSeek Model on 64GB RAM Without a GPU
GeekNews ·
At a glance
llama-modchip is an open-source project that runs the DeepSeek-V4-Flash model on a computer with 64GB of memory and no graphics card (GPU). It compresses the model file and optimizes CPU calculations to make this possible. According to the project's own measurements, actual memory use in a 256K-token conversation was about 57.61 to 58.9GiB, and it did not use swap, which treats disk space as memory. These results are measured by the project itself.
Key points
- It reduced the model file from 76.87GiB to 56.56GiB so that it fits within 64GB of memory.
- On a Ryzen 9 9955HX CPU, input reading speed reached up to 44.36 tokens per second, and output writing speed was 9.06 to 9.95 tokens per second, based on the project's measurements.
- The Nemotron-3.5-30B model had a default response speed of 25.02 tokens per second on the same CPU.
- These models do not open in regular llama.cpp, Ollama, or LM Studio and require a dedicated runner.
Why it matters
It shows that large AI models may be run on personal computers without powerful GPUs. However, speed and answer quality vary depending on settings.
Original source
Llama-modchip: GPU 없이 64GB RAM에서 284B DeepSeek 실행
GeekNews · Nov 9, 00:32 KST
Read on GeekNews ↗AI summarized the original article. Check the original for accuracy. Copyright belongs to GeekNews.
DeepSeek · Ranking and more news
More from this field
- Designers Say Ssense's Website Shows Their Clothes Inaccurately
Designer Reese Cooper noticed his RCOS puffer coat looked distorted on the luxury retailer Ssense's website, with a chest hook drawn far too large. Ssense said some site images came from a "virtual try-on" program, though it would not say whether that program uses AI.
The New York Times · Nov 21, 06:16 KST - Korean Firms Build Three-Way AI Push: Anthropic Stake, HBM Chips, Claude Rollout
Korean companies are expanding their roles in the global AI market around Anthropic's IPO. SK Telecom earned large paper gains from an early investment, while Samsung Electronics and SK hynix are strengthening chip supply ties. Samsung SDS and LG CNS are bringing Anthropic's Claude model into the business AI market.
뉴스웨이 · Nov 21, 06:10 KST - MIT and IBM Team Probes Gaps in Multimodal AI Understanding
MIT and IBM researchers are studying where today's multimodal AI models fall short, even when they handle text, images, and sound fluently. Their team, co-led by MIT's Jim Glass, builds new benchmarks and training methods that need less labeled data. This Q&A walks through how the project started and how it has evolved.
MIT News · Nov 21, 06:00 KST - Leaky AI Supply Chain Exposes Secret User Data at Alarming Rate
A research test sent nearly two million prompts with fake but working secrets through a network of AI services to see where they ended up. Many of the secrets were stolen, and attackers used them to reach cloud accounts. The results point to weak protection of customer data along the chain.
Ars Technica · Nov 21, 05:59 KST