High-quality and diverse training data are essential for various machine learning applications. Extracting meaningful question-answer pairs from documents has traditionally been a cumbersome task. To address this, we developed Document to Synthetic QA, an application leveraging Gretel Navigator, available on Hugging Face, to streamline this process. Our app not only simplifies extraction but also enhances the quality and diversity of training data, proving invaluable for model training, educational content creation, and customer support solutions.

Key Features

• Document to Text Conversion: Converts PDF, TXT, and Markdown documents into text, enabling seamless processing and analysis. 
• Text Chunking: Splits extracted text into manageable chunks, optimizing the size for question-answer generation.
• Synthetic Data Generation: Utilizes the Gretel Navigator API to generate synthetic question-answer pairs tailored to specific needs such as user expertise levels and topics of interest.
  

Potential Applications

• Model Training and Evaluation: Generate diverse and high-quality question-answer pairs to train and evaluate various machine learning models.
• Educational Tools: Create customized question-answer sets for different educational levels and subjects.
• Customer Support: Develop comprehensive FAQs and troubleshooting guides based on user manuals and product documentation.
• QA Pair Scoring: Rates generated text based on conformance, quality, toxicity, bias, and groundedness. 

How It Works

1. Upload or Use Example PDF: Start by uploading your documents or using an example document provided within the app.
2. Process Documents: Convert the documents to text and split them into chunks.
3. Generate QA Pairs: Input your Gretel API key and specify the number of records to generate. The app will produce a dataset of question-answer pairs.
4. Download Results: Review and download the generated synthetic records in CSV format for further use.

Example Use Case: RAG Model Evaluation

One potential application is in the evaluation of Retrieval-Augmented Generation (RAG) models. High-quality evaluation datasets are essential for assessing RAG models on metrics such as coherence, relevance, and fluency. Gretel Navigator offers a streamlined solution to create question-truth pairs, ensuring that datasets are comprehensive and aligned with real-world use cases. It allows the creation of synthetic datasets that challenge RAG models to fetch relevant information and craft precise answers, even in adversarial and ambiguous scenarios. For more details, see our earlier blog post on RAG Model Evaluation with Azure AI and Gretel Navigator. 

Get Started

To get started with follow these steps:

1. Obtain your Gretel API key from the Gretel console.
2. Prepare your domain-specific documents or use the provided example.
3. Launch the app and start generating question-answer pairs tailored to your needs.