Google Launches MKA Benchmark for Visually Complex Knowledge Work

At a glance

Google DeepMind, working with Artificial Analysis, released MKA (Multimodal Knowledge Agent), a benchmark that tests how well AI agents (AI that carries out multi-step tasks on its own) handle work where visuals and layout matter, such as blueprints, schematics, reports, and spreadsheets. It includes 200 tasks written by domain experts across 10 fields and 83 occupations.

Key points

1. 1

   Each task is a multi-step job that takes a skilled professional three to eight hours and is graded with about 70 yes/no checks.

2. 2

   Scoring weights factual accuracy at 75% and visual design at 25%.

3. 3

   Grading is done by an AI from a non-Google company, which matched expert grades 96% of the time.

4. 4

   Ten public tasks are on Hugging Face, while the 200-task set stays private.

Why it matters

It gives a way to measure whether AI can handle real work that depends on images and document layout, not just text. Because models differ widely in how many steps they need, it also helps compare efficiency.