Back to news

AI · Video

Open AI Models Still Fall Short for Agentic Engineering, Despite Strong Benchmarks

Cole Medin ·

At a glance

This video from the Cole Medin channel examines whether open-weight models (AI models whose weights are publicly released) can be a practical choice for agentic engineering (where AI carries out multi-step work on its own). According to the introduction, their benchmark scores are similar to top commercial models, but they fail to follow instructions and break down on long tasks.

Key points

Why it matters

It gives people deciding whether to adopt AI models for work a realistic view of their limits.

Original source

I REALLY Want Open Models to Win, But They Just Don't Cut It (Yet)

Cole Medin · Nov 5, 10:00 KST

Watch on YouTube ↗

AI summarized the video title and description. Check the original for accuracy. Copyright belongs to Cole Medin.

AI