Skip to content
Relari AI

Harden your
GenAI Systems
with Synthetic Data

Open-source evaluation framework and synthetic data generation pipeline

pip install


Get StartedView Docs

Built by experts in reliable AI production

MIT Logo
Harvard Logo
Nvidia Logo
Bank of America Logo Logo
Raytheon Technologies Logo

of Complex Systems

Programatically define your pipeline
and select metrics for each module
View Example Code

Generate Synthetic Data to Stress Test Your AI Systems

Not enough data? No problem. You can now generate large-scale synthetic datasets tailored to your application to test your models and applications at a fraction of the cost.

Open Source Metric Library

Get started for free with 30+ open source metrics and evaluators.
Covers text generation, code generation, retrieval, classification, agents , and more.

Close-to-human Evaluators

Leverage user feedback data to train custom ensemble evaluators that are 90%+ aligned with human evaluation.

Systematic Fine-Tuning

Leverage synthetic data and comprehensive metrics to take control of the fine-tuning process.

From Prototype to Production

Get support throughout the GenAI application development lifecycle

    Prompt IterationModel SelectionRegression TestingBenchmarkingCI/CD IntegrationFine-TuningMonitoringFunctional TestingOnline EvaluationOffline Evaluation

Product Walkthrough

Trusted by Innovative AI Teams

Yuhong Sun

Founder, Danswer AI

“Relari's custom generated synthetic dataset is the best real world representation we've seen! We use the data to stress test our enterprise search engine and guide key product decisions.”

Nick Bradford


"We iterate much faster on our coding agents thanks to the granular metrics Relari offers! Through high-quality synthetic datasets, we can benchmark and validate our agent performance with ease.”

Mike Sands

Senior Director of Product, Thoropass

“Relari has helped immensely by building a set of metrics and standards that we can use to quickly and automatically evaluate changes in our LLM pipeline."


Ways to get started.


Ideal for individual developers, researchers, and those experimenting with GenAI products.

Complete suite of GenAI / LLM metrics

Ensemble close-to-human evaluators

Simple synthetic data generation

Open source community support


Best for AI teams who want to deploy reliable LLM applications at scale.
Custom Pricing

Granular golden dataset curation

Experimentation toolkit

Advanced production analytics

Priority feature requests

Dedicated support

Test and Improve GenAI Systems with Synthetic Data

Terms   Privacy Policy   Security
© 2024 Relari, inc. All rights reserved.

Backed by YC