Google Research Brings Human Deliberation into Subjective AI Classifiers

At a glance

Google Research has released Agile Deliberation, a tool that helps refine AI systems that classify ambiguous topics together with people. Users review borderline examples to set criteria, and those criteria are then turned into instructions the AI can read. The open-source code and a demo notebook have also been published.

Key points

1. 1

   It handles topics where people have different standards, such as healthy food or clickbait ads.

2. 2

   It first separates what should be included from what should be excluded, then refines the criteria by reviewing confusing images.

3. 3

   In an experiment with 9 participants, it was more accurate and less burdensome than directly editing the criteria by hand.

4. 4

   The researchers noted that the number of participants was small, so the results are hard to generalize widely.

Why it matters

Standards for filtering content vary with human judgment, and this tool makes that judgment process visible.