AI · Video
How a Biopharma Document Indexing AI Agent Reached 99.95% Accuracy
AI Engineer ·
At a glance
This talk covers a case of applying an AI agent (a program that carries out multi-step tasks on its own) to indexing work for biopharma documents. The speaker is Matthew Jewkes, who built the system at Clarivate. The early approach reached only about 80 to 83% accuracy.
Key points
- Working with experts, the team reviewed 1,000 evaluation examples and sorted errors into three types.
- The team distinguished whether an error came from the model, from a problem with no agreed-upon correct answer inside the organization, or from a problem requiring new scientific judgment.
- Routine work is handled by strictly rule-based automation, while uncertain or unusual cases are designed to be passed to people.
Why it matters
It is helpful for anyone who wants to learn how to automate large volumes of repetitive work while focusing experts on tasks that require their judgment.
Original source
Engineering Agency out of the Happy Path — Matthew Jewkes, Standard Cybernetics
AI Engineer · Nov 9, 13:00 KST
Watch on YouTube ↗AI summarized the video title and description. Check the original for accuracy. Copyright belongs to AI Engineer.
More from this field
- OpenAI Opens ChatGPT Space to Plus Users for Pages, Slides, and Team Sharing
OpenAI opened ChatGPT Space to Plus users on the 20th, local time. Space works as a new home for pages, slides, images, files, and uploads. Users can write pages with ChatGPT, add interactive charts, turn ideas into slides, and invite others to edit together.
AI 매터스 · Nov 21, 11:26 KST - Anthropic Researcher Claims Claude Helped Prove Riemann Hypothesis, Experts Await Review
Wes Roth covers a claim by Anthropic researcher Levent Alpöge that he proved the Riemann Hypothesis with help from the AI model Claude. He looks at the huge Lean formal proof, reactions from mathematicians and AI safety researchers, and what this could mean for AI and math.
Wes Roth · Nov 21, 11:24 KST - Amazon CloudWatch Omni Adds Microsoft Azure Logs, Metrics, and Traces Support
Amazon CloudWatch Omni can now collect Azure resource logs, Azure Monitor platform metrics, virtual network flow logs, and Application Insights traces. Teams running workloads on both AWS and Azure can view all of this telemetry in one place, without separate tools or custom pipelines.
AWS What's New · Nov 21, 10:10 KST - AWS Quick Adds Chat-Built Flows, Updated Web App, and Teamwork Among Agents
AWS Quick added updates that let a team turn one person's work routine into a shared, repeatable process. Flows can now be built and changed entirely in chat, and they can run from the web, desktop, or mobile app. Agents working together can also give more complete answers.
Amazon AWS Machine Learning blog · Nov 21, 10:06 KST