Zum Hauptinhalt springen

Together AI

LiteLLM unterstĂĽtzt alle Modelle auf Together AI.

API-Schlüssel​

import os 
os.environ["TOGETHERAI_API_KEY"] = "your-api-key"

Beispielverwendung​

from litellm import completion 

os.environ["TOGETHERAI_API_KEY"] = "your-api-key"

messages = [{"role": "user", "content": "Write me a poem about the blue sky"}]

completion(model="together_ai/togethercomputer/Llama-2-7B-32K-Instruct", messages=messages)

Together AI Modelle​

liteLLM unterstĂĽtzt Non-Streaming- und Streaming-Anfragen an alle Modelle auf https://api.together.xyz/

Beispiel für die Verwendung von TogetherAI – Hinweis: liteLLM unterstützt alle auf TogetherAI bereitgestellten Modelle

Llama LLMs - Chat​

ModellnameFunktionsaufrufErforderliche OS-Variablen
togethercomputer/llama-2-70b-chatcompletion('together_ai/togethercomputer/llama-2-70b-chat', messages)os.environ['TOGETHERAI_API_KEY']

Llama LLMs - Sprache / Instruktion​

ModellnameFunktionsaufrufErforderliche OS-Variablen
togethercomputer/llama-2-70bcompletion('together_ai/togethercomputer/llama-2-70b', messages)os.environ['TOGETHERAI_API_KEY']
togethercomputer/LLaMA-2-7B-32Kcompletion('together_ai/togethercomputer/LLaMA-2-7B-32K', messages)os.environ['TOGETHERAI_API_KEY']
togethercomputer/Llama-2-7B-32K-Instructcompletion('together_ai/togethercomputer/Llama-2-7B-32K-Instruct', messages)os.environ['TOGETHERAI_API_KEY']
togethercomputer/llama-2-7bcompletion('together_ai/togethercomputer/llama-2-7b', messages)os.environ['TOGETHERAI_API_KEY']

Falcon LLMs​

ModellnameFunktionsaufrufErforderliche OS-Variablen
togethercomputer/falcon-40b-instructcompletion('together_ai/togethercomputer/falcon-40b-instruct', messages)os.environ['TOGETHERAI_API_KEY']
togethercomputer/falcon-7b-instructcompletion('together_ai/togethercomputer/falcon-7b-instruct', messages)os.environ['TOGETHERAI_API_KEY']

Alpaca LLMs​

ModellnameFunktionsaufrufErforderliche OS-Variablen
togethercomputer/alpaca-7bcompletion('together_ai/togethercomputer/alpaca-7b', messages)os.environ['TOGETHERAI_API_KEY']

Andere Chat LLMs​

ModellnameFunktionsaufrufErforderliche OS-Variablen
HuggingFaceH4/starchat-alphacompletion('together_ai/HuggingFaceH4/starchat-alpha', messages)os.environ['TOGETHERAI_API_KEY']

Code LLMs​

ModellnameFunktionsaufrufErforderliche OS-Variablen
togethercomputer/CodeLlama-34bcompletion('together_ai/togethercomputer/CodeLlama-34b', messages)os.environ['TOGETHERAI_API_KEY']
togethercomputer/CodeLlama-34b-Instructcompletion('together_ai/togethercomputer/CodeLlama-34b-Instruct', messages)os.environ['TOGETHERAI_API_KEY']
togethercomputer/CodeLlama-34b-Pythoncompletion('together_ai/togethercomputer/CodeLlama-34b-Python', messages)os.environ['TOGETHERAI_API_KEY']
defog/sqlcodercompletion('together_ai/defog/sqlcoder', messages)os.environ['TOGETHERAI_API_KEY']
NumbersStation/nsql-llama-2-7Bcompletion('together_ai/NumbersStation/nsql-llama-2-7B', messages)os.environ['TOGETHERAI_API_KEY']
WizardLM/WizardCoder-15B-V1.0completion('together_ai/WizardLM/WizardCoder-15B-V1.0', messages)os.environ['TOGETHERAI_API_KEY']
WizardLM/WizardCoder-Python-34B-V1.0completion('together_ai/WizardLM/WizardCoder-Python-34B-V1.0', messages)os.environ['TOGETHERAI_API_KEY']

Language LLMs​

ModellnameFunktionsaufrufErforderliche OS-Variablen
NousResearch/Nous-Hermes-Llama2-13bcompletion('together_ai/NousResearch/Nous-Hermes-Llama2-13b', messages)os.environ['TOGETHERAI_API_KEY']
Austism/chronos-hermes-13bcompletion('together_ai/Austism/chronos-hermes-13b', messages)os.environ['TOGETHERAI_API_KEY']
upstage/SOLAR-0-70b-16bitcompletion('together_ai/upstage/SOLAR-0-70b-16bit', messages)os.environ['TOGETHERAI_API_KEY']
WizardLM/WizardLM-70B-V1.0completion('together_ai/WizardLM/WizardLM-70B-V1.0', messages)os.environ['TOGETHERAI_API_KEY']

Prompt-Vorlagen​

Verwenden eines Chat-Modells auf Together AI mit seinem eigenen Prompt-Format?

Verwenden von Llama2 Instruct-Modellen​

Wenn Sie die Llama2-Varianten von Together AI verwenden (model=togethercomputer/llama-2..-instruct), kann LiteLLM automatisch zwischen dem OpenAI-Prompt-Format und dem Llama2-Format von TogetherAI ([INST]..[/INST]) ĂĽbersetzen.

from litellm import completion 

# set env variable 
os.environ["TOGETHERAI_API_KEY"] = ""

messages = [{"role": "user", "content": "Write me a poem about the blue sky"}]

completion(model="together_ai/togethercomputer/Llama-2-7B-32K-Instruct", messages=messages)

Verwenden eines anderen Modells​

Sie können eine benutzerdefinierte Prompt-Vorlage in LiteLLM erstellen (und wir begrüßen PRs, um sie zum Haupt-Repository hinzuzufügen 🤗)

Erstellen wir eine fĂĽr OpenAssistant/llama2-70b-oasst-sft-v10!

Das akzeptierte Vorlagenformat ist: Referenz

"""
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
"""

Registrieren wir unsere benutzerdefinierte Prompt-Vorlage: Implementierungscode

import litellm 

litellm.register_prompt_template(
        model="OpenAssistant/llama2-70b-oasst-sft-v10",
        roles={
            "system": {
                "pre_message": "[<|im_start|>system",
                "post_message": "\n"
            },
            "user": {
                "pre_message": "<|im_start|>user",
                "post_message": "\n"
            }, 
            "assistant": {
                "pre_message": "<|im_start|>assistant",
                "post_message": "\n"
            }
        }
    )

Lassen Sie es uns verwenden!

from litellm import completion 

# set env variable 
os.environ["TOGETHERAI_API_KEY"] = ""

messages=[{"role":"user", "content": "Write me a poem about the blue sky"}]

completion(model="together_ai/OpenAssistant/llama2-70b-oasst-sft-v10", messages=messages)

Vollständiger Code

import litellm 
from litellm import completion

# set env variable 
os.environ["TOGETHERAI_API_KEY"] = ""

litellm.register_prompt_template(
        model="OpenAssistant/llama2-70b-oasst-sft-v10",
        roles={
            "system": {
                "pre_message": "[<|im_start|>system",
                "post_message": "\n"
            },
            "user": {
                "pre_message": "<|im_start|>user",
                "post_message": "\n"
            }, 
            "assistant": {
                "pre_message": "<|im_start|>assistant",
                "post_message": "\n"
            }
        }
    )

messages=[{"role":"user", "content": "Write me a poem about the blue sky"}]

response = completion(model="together_ai/OpenAssistant/llama2-70b-oasst-sft-v10", messages=messages)

print(response)

Ausgabe

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": ".\n\nThe sky is a canvas of blue,\nWith clouds that drift and move,",
        "role": "assistant",
        "logprobs": null
      }
    }
  ],
  "created": 1693941410.482018,
  "model": "OpenAssistant/llama2-70b-oasst-sft-v10",
  "usage": {
    "prompt_tokens": 7,
    "completion_tokens": 16,
    "total_tokens": 23
  },
  "litellm_call_id": "f21315db-afd6-4c1e-b43a-0b5682de4b06"
}

Rerank​

Verwendung​

from litellm import rerank
import os

os.environ["TOGETHERAI_API_KEY"] = "sk-.."

query = "What is the capital of the United States?"
documents = [
    "Carson City is the capital city of the American state of Nevada.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
    "Washington, D.C. is the capital of the United States.",
    "Capital punishment has existed in the United States since before it was a country.",
]

response = rerank(
    model="together_ai/rerank-english-v3.0",
    query=query,
    documents=documents,
    top_n=3,
)
print(response)