Teaching AI to control computers with Gretel Navigator on Amazon Bedrock
One of the most powerful capabilities of modern AI systems is their ability to connect natural language understanding with API calls and function execution. Training AI models to bridge this gap–to understand user requests and safely invoke the right APIs–is a critical challenge that could define the next generation of AI assistants.
Today, we're excited to announce that Gretel Navigator is available on Amazon Bedrock, making it easier than ever to generate the synthetic training data needed to connect language models with real-world tools and APIs.
Why is training data such a pain?
The biggest challenge in building AI systems for computer control isn't just the models - it's the training data. Here's what we typically see teams struggling with:
- Having to write thousands of example commands that users might ask (both valid and invalid) is slow and error-prone
- Real user data often lacks important edge cases and error scenarios
- Teaching models when to reject unsafe requests requires carefully crafted negative examples
- Maintaining consistency across large datasets of function calls is tedious and difficult
Gretel Navigator on Amazon Bedrock provides a powerful solution
Enterprise-Grade Infrastructure
- Seamless scalability through AWS's robust infrastructure
- High availability and reliability
- Enterprise-level security and compliance
Streamlined Workflow
- Native integration with AWS services
- Simplified deployment and management
- Reduced operational overhead
Cost-Effective Synthetic Data Generation
- Pay-as-you-go pricing
- Elimination of manual data collection costs
- Reduced time-to-market
Enhanced Data Quality
- Consistent, well-structured data generation
- Comprehensive coverage of edge cases
- Built-in quality validation
Let's look at a real example
To demonstrate how Gretel Navigator works on Bedrock, we've created a reference notebook that generates training data for teaching AI to control computer functions.Â
Inspired by Anthropic's work on computer control, our reference implementation shows how to generate high-quality training data for these tool-use scenarios. While our example focuses on computer control functions, the approach can be easily adapted for any API or tool-calling scenario, from drone navigation to business workflows.
Function definition
At its core, the notebook defines a comprehensive set of computer control functions. These range from basic operations like opening applications and managing files, to more complex tasks like adjusting system settings and handling web interactions.Â
Parameter generation
Each function is carefully specified with proper parameter types and constraints. For example, volume controls must stay within 0-100, and file operations require valid paths and permissions.
Natural language command synthesis
The real magic happens when Gretel Navigator transforms these function definitions into natural language interactions. It generates hundreds of varied ways humans might request these operations. A single function like `open_file(file_name: str, file_path: str)` becomes dozens of natural requests like "Could you open the quarterly report from the Documents folder?" or "I need to access last month's budget spreadsheet in the Finance directory."
Rejection scenario generation
Just as important as teaching an AI what it can do, is teaching it what it shouldn't do. The notebook includes a robust system for generating rejection scenarios. These are perfectly natural-sounding requests that should be declined: things like accessing unauthorized directories, executing unsafe commands, or making ambiguous requests that need clarification. This helps build AI assistants that know their boundaries and can explain them clearly to users.
Function coverage analysis
To ensure the quality of the generated dataset, we include comprehensive coverage analysis. This analysis tracks how often each function is called, the diversity of parameter values used, and the distribution of different types of requests. It helps identify any gaps in the training data – like rarely-used functions or underrepresented parameter combinations – ensuring your AI system will be prepared for the full range of real-world interactions it might encounter.
We handle all this through straightforward API calls to Gretel Navigator on Bedrock, making it easy to generate diverse, realistic examples while maintaining proper constraints and safety boundaries. More importantly, you can use this as a template for your own tools and commands.
What's next?
The training data you generate with Gretel Navigator can be used in multiple ways:
Adapt for your domain
- Build datasets for home automation and IoT devices
- Generate training data for robotics and drone control
- Create examples for custom business tools and APIs
- Scale your dataset generation from hours to minutes
Choose your approach
- Fine-tune models with domain-specific commands and tools
- Implement RAG workflows for dynamic decision-making
- Use few-shot prompting for quick prototyping
- Build hybrid solutions that combine multiple approaches
Expand your capabilities
- Add support for complex multi-tool workflows
- Generate nuanced rejection scenarios
- Cover edge cases specific to your domain
- Iterate based on real-world feedback
Ready to get started?
- Sign up for Gretel
- Visit the Amazon Bedrock Marketplace to enable Gretel Navigator
- Try our computer-control data generation notebook for yourself
The combination of Gretel Navigator and Amazon Bedrock provides a powerful solution for generating the high-quality training data needed for modern AI systems. Whether you're building function calling capabilities, tool use scenarios, or other AI applications, this integration offers a scalable, efficient path to better training data.
If you have questions or comments about Gretel Navigator or the Gretel Platform, join us and 1,600 other developers, engineers, data scientists, and privacy fans in the Synthetic Data Community on Discord.