Braintrust - Evals + Protokollierung

Braintrust verwaltet Auswertungen, Protokollierung, Prompt-Playground und Datenmanagement für KI-Produkte.

Schnellstart

# pip install langfuse 
import litellm
import os

# set env 
os.environ["BRAINTRUST_API_KEY"] = "" 
os.environ['OPENAI_API_KEY']=""

# set braintrust as a callback, litellm will send the data to braintrust
litellm.callbacks = ["braintrust"] 
 
# openai call
response = litellm.completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ]
)

Verwendung des OpenAI-Proxys

1. Schlüssel zur Umgebungsvariable hinzufügen

BRAINTRUST_API_KEY="" 

2. Braintrust zu Rückrufen hinzufügen

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY


litellm_settings:
  callbacks: ["braintrust"]

3. Testen Sie es!

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-D '{
    "model": "groq-llama3",
    "messages": [
        { "role": "system", "content": "Use your tools smartly"},
        { "role": "user", "content": "What time is it now? Use your tool"}
    ]
}'

Fortgeschritten - Projekt-ID oder Namen übergeben

SDK

response = litellm.completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ], 
  metadata={
    "project_id": "1234",
    # passing project_name will try to find a project with that name, or create one if it doesn't exist
    # if both project_id and project_name are passed, project_id will be used
    # "project_name": "my-special-project" 
  }
)

PROXY

Curl

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-D '{
    "model": "groq-llama3",
    "messages": [
        { "role": "system", "content": "Use your tools smartly"},
        { "role": "user", "content": "What time is it now? Use your tool"}
    ],
    "metadata": {
        "project_id": "my-special-project"
    }
}'

OpenAI SDK

import openai
client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={ # pass in any provider-specific param, if not supported by openai, https://docs.litellm.de/docs/completion/input#provider-specific-params
        "metadata": { # 👈 use for logging additional params (e.g. to langfuse)
            "project_id": "my-special-project"
        }
    }
)

print(response)

Für weitere Beispiele, Klicken Sie hier

Vollständige API-Spezifikation

Hier sind alle Metadaten, die Sie für eine Braintrust-Anfrage übergeben können

braintrust_* - Jedes Metadatenfeld, das mit braintrust_ beginnt, wird als Metadaten für die Protokollierungsanfrage übergeben

project_id - Legt die Projekt-ID für einen Braintrust-Aufruf fest. Der Standardwert ist litellm.