Summary
Google and Artificial Analysis have introduced MKA, a 200-task benchmark for AI agents working across visually complex files. Google says Gemini 4 Carbon ranked second in quality among the models tested and completed tasks at six times lower average cost than the top-scoring model.
Google and Artificial Analysis have launched the Multimodal Knowledge Agent (MKA) benchmark to assess how AI agents handle knowledge-work tasks involving visually complex files. The 200-task evaluation asks models to work from bundles of multimodal inputs and produce deliverables such as reports, spreadsheets or presentations. Google says Gemini 4 Carbon ranked second in quality among the models tested, at an average cost six times lower than the top-scoring model.
How MKA tests agent work
Google says the benchmark was developed with domain experts across 10 industry verticals and 83 occupations, including architecture, biology, underwriting and data science. Each task is designed as a multi-step project that a seasoned professional would take three to eight hours to complete. It pairs a realistic prompt with a collection of input files and an expert-written rubric of roughly 70 yes-or-no checks.
The rubric gives 75% of its weight to factual accuracy and 25% to visual design. That makes the evaluation about more than reading text or identifying objects in an image: an agent must use information across files and produce a finished output that meets the task criteria.
For the main evaluation, Artificial Analysis runs the models in Stirrup, an open-source agent harness. Google says each model receives the same tools, sandbox and turn budget. The 200 evaluation tasks are kept private to reduce the chance of their entering training data or being found through search. A separate set of 10 public tasks is available on Hugging Face for people who want to inspect the format or test agents.
Scoring and reported results
An automated agent grades each deliverable against the expert rubric. Google says the autorater, built on a non-Google model, opens files, renders pages and inspects spreadsheet formulas. When domain experts graded model outputs, their assessments agreed with the autorater 96% of the time.
Google’s announcement also reports substantial differences in how many agent turns models used: between 30 and 130 on average per task. It says the technical report examines the role of visual perception and how models perform when vision is removed.
On the MKA results, Google reports that Gemini 4 Carbon placed second for quality among tested models and completed tasks at six times lower average cost than the top scorer. The announcement does not give the underlying quality scores or dollar costs, so the result is best read as a company-reported comparison within this evaluation. Gemini 4 Carbon is available through the Gemini API in Google AI Studio.
What the benchmark adds
MKA focuses on work where information is distributed across visual documents, page layouts and other files, rather than tasks that can be answered from text alone. Its shared tools and execution setup are intended to make comparisons across models more consistent, while the private task set helps limit benchmark contamination.
The benchmark’s findings apply to its 200 tasks and scoring approach. Its combination of factual and visual criteria offers one way to measure agents on complete knowledge-work outputs, rather than only on isolated question-and-answer performance.