Jobs at Braintrust Data

San Francisco, CA

Braintrust offers an AI stack to simplify the process from evaluations to data management, ensuring seamless integration into any business. Braintrust simplifies the evaluation process by facilitating easy scoring, logging, and visualization of outputs. Users can investigate failures, monitor performance trends, and address queries like identifying regressions or assessing new models. The platform offers a Prompt Playground, allowing users to compare multiple prompts, benchmarks, and input/output pairs across runs, enabling both ephemeral tinkering and experiment evaluation on large datasets. In Continuous Integration, Braintrust seamlessly integrates into workflows, enabling progress tracking on the main branch and automatic comparison of new experiments with live versions before deployment. With a focus on Datasets, the platform enables the effortless capture of rated examples from staging and production, incorporating them into versioned ''golden'' datasets stored in the cloud. This ensures the evolution of datasets without jeopardizing evaluations dependent on them.