AI · Official
Amazon Bedrock AgentCore Cuts Token Costs and Latency for Large JSON Workloads
Amazon AWS Machine Learning blog ·

At a glance
Flightscape, an airline operations platform, built an AI agent that checks crew training schedules, and it handled 20 MB+ JSON files slowly and at high token cost. The team tried three ways to shrink what gets sent to the model each time, and their tests showed token use down by up to 71% and average latency down by up to 80%. The strategies also aim to keep the agent's reasoning working across long chats.
Key points
- Flightscape's crew scheduling agent, built on Amazon Bedrock AgentCore, was slow with large JSON data, with average response times above 15 seconds.
- Each tool call sent the whole data file to the AI model, which raised token counts and costs and could fill the model's memory before finishing.
- The team used three strategies to limit the data passed to the model during each conversation turn.
- In tests on a 23 MB sample, token use fell by up to 71% and average latency fell by up to 80%.
Why it matters
Large data sent to AI agents makes them costly and slow, so these methods can make production AI tools cheaper and more responsive.
Original source
Reducing token costs and latency in Amazon Bedrock AgentCore JSON workloads
Amazon AWS Machine Learning blog · Nov 18, 07:44 KST
Read on Amazon AWS Machine Learning blog ↗AI summarized the original article. Check the original for accuracy. Copyright belongs to Amazon AWS Machine Learning blog.
More from this field
- Nvidia Says Revenue Will Keep Soaring Despite Jensen Huang's Exit
Nvidia reported third-quarter revenue of $115.8 billion, up 103% from a year earlier and above its own forecast, even after CEO Jensen Huang's departure. Interim CEO Colette Kress also said the company expects fourth-quarter revenue of about $130 billion and raised its full-year outlook.
TechCrunch · Nov 18, 09:16 KST - Ofqual Warns Students Face Lifelong Penalties for AI and Smart Glasses Cheating
England's exam regulator, Ofqual, warned that students caught using AI or smart glasses to cheat could lose entire qualifications such as GCSEs and A-levels. It also wants retailers barred from marketing wearable tech for exam cheating, and exam staff trained to spot such devices.
The Guardian · Nov 18, 09:01 KST - OpenAI Launches GPT-6.2 Astra and GPT-6.2 Sol with Lower Astra Pricing
OpenAI has released two new models, GPT-6.2 Astra and GPT-6.2 Sol, in ChatGPT Work, Codex, and the API. Astra is built for complex projects and Sol for everyday text, code, and multi-step tasks. OpenAI also lowered Astra's base price.
mobidevices.com · Nov 18, 09:00 KST