We are excited to announce a series of updates that will be rolling out over the next few months to the Gretel GPT Preview. Gretel GPT is an API for creating synthetic natural language text using Large Language Models (LLMs), which can be used for generating labeled examples for training or testing downstream machine learning models. You can fine-tune the model on your own unique data, or provide a few examples for the model to learn to recreate. 

Over the months since launching Gretel GPT in preview, it has grown to be one of our most popular synthetic data models — last month accounting for nearly 8% of all jobs run in Gretel’s cloud service. Our blueprint example using Gretel GPT to create additional synthetic examples to train a financial call-center chatbot to better recognize user intents has risen to be in our top five blueprint examples (see notebook version here). Today, users are using Gretel GPT for anything from generating labeled examples to help ML models better detect abusive and toxic language, to synthesizing doctors’ notes and patient symptoms from electronic medical records to enable research and data sharing between hospitals while protecting patient privacy.

The initial Gretel GPT preview was a promising start and it showed great potential in several key areas, notably the ability to run in the Gretel cloud or scale-out in your own cloud via Kubernetes, the ability to easily combine Gretel GPT with Gretel’s tabular and time-series models, and simultaneous request handling capabilities through the batch inference API. These features made it simple for developers to quickly iterate and create hundreds or thousands of synthetic examples for training and testing their ML models.

We are thrilled to reveal the latest enhancements to the Gretel GPT preview, including the ability to fine-tune state-of-the-art LLMs on your enterprise data with the option of operating entirely within your cloud or VPC to meet the strictest security and compliance requirements.

Key upgrades:

• Gretel GPT now enables fine-tuning and inference of state-of-the-art LLMs. Supported models include MosaicML's mpt-7b, a robust model boasting 7 billion parameters and trained on more than 1 trillion tokens, TogetherComputer's RedPajama 3B model that is specifically tuned for instructions, and StabilityAI’s stable LM 3B model.
  
• Unlike foundation-level models served behind an API, Gretel GPT offers commercially viable models that can operate using Gretel’s managed service, or entirely within your cloud or VPC. This ensures your sensitive data remains within your environment, thereby enhancing data security.
  
• Gretel GPT models now come equipped with PEFT (Parameter-Efficient Fine-Tuning) and LORA (Low-Rank Adaptation of Large Language Models) for supported models. These technologies enable you to create adapted versions of LLMs with significantly reduced training times and costs, compared to fine-tuning all model parameters.
• Forget the guesswork involved in hyper-parameter settings. Whether you're training in the Gretel cloud or on your own, Gretel’s auto-parameters automatically recommend the best configurations for your dataset, based on training hundreds of thousands of models via Gretel’s cloud service.

Get started in minutes by running one of our blueprint examples using the model upgrades ‌ — such as generating additional labeled examples to train a financial chatbot, or training the model to generate song lyrics that sound like Taylor Swift. If you’re already using Gretel GPT, you can simply update your training configuration to use the new models.

Gretel GPT configuration:

# Use when training on natural language data such as reviews, tweets, and conversations
# If training data contains multiple columns, specify the column containing natural language text
# using the column_name parameter
schema_version: "1.0"
name: "natural-language-gpt"
models:
  - gpt_x:
    data_source: "__temp__"
    pretrained_model: "gretelai/mpt-7b"
    batch_size: 4
    epochs: 3
    weight_decay: 0.01
    warmup_steps: 100
    lr_scheduler: "linear"
    learning_rate: 0.0002
    column_name: null
    generate:
      num_records: 10
      maximum_text_length: 100

For optimal results, we recommend training on up to 2000 examples. Fine-tuning larger datasets with 2000+ examples using default parameters may potentially surpass the default maximum runtime limit of one hour per job on Gretel's developer tier. If you would like to increase your maximum runtime limit, please feel free to reach out to us at support at gretel.ai.

We're all ears for any thoughts, ideas, or questions you might have, and our community Discord is the perfect place for that. Keep an eye out because we've got some pretty cool updates on the horizon. We're working on a bunch of new features that we believe will make it easier for you to trust and feel secure about the privacy and accuracy of LLMs for your enterprise use cases. Can't wait for you to try them out!