Groq - Portkey Docs

Quick Start

Get started with Groq in under 2 minutes:

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @groq provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@groq/llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @groq provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@groq/llama-3.3-70b-versatile",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @groq provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@groq/llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @groq provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@groq/llama-3.3-70b-versatile",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

# 1. Add @groq provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@groq/llama-3.3-70b-versatile",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Add Provider in Model Catalog

Before making requests, add Groq to your Model Catalog:

Go to Model Catalog → Add Provider
Select Groq
Enter your Groq API key
Name your provider (e.g., groq)

Complete Setup Guide

See all setup options and detailed configuration instructions

Groq Capabilities

Tool Calling

Use Groq’s tool calling feature to trigger external functions:

Supported Models

View Groq models that support tool calling

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

tools = [{
    "type": "function",
    "function": {
        "name": "getWeather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

response = portkey.chat.completions.create(
    model="@groq/llama-3.3-70b-versatile",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the weather like in Delhi?"}
    ],
    tools=tools,
    tool_choice="auto"
)

print(response.choices[0].finish_reason)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY'
});

const tools = [{
    type: "function",
    function: {
        name: "getWeather",
        description: "Get the current weather",
        parameters: {
            type: "object",
            properties: {
                location: { type: "string", description: "City and state" },
                unit: { type: "string", enum: ["celsius", "fahrenheit"] }
            },
            required: ["location"]
        }
    }
}];

const response = await portkey.chat.completions.create({
    model: "@groq/llama-3.3-70b-versatile",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What's the weather like in Delhi?" }
    ],
    tools,
    tool_choice: "auto"
});

console.log(response.choices[0].finish_reason);

curl https://api.portkey.ai/v1/chat/completions \
     -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -H "x-portkey-provider: @groq" \
     -d '{
       "model": "llama-3.3-70b-versatile",
       "messages": [
         {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What'\''s the weather like in Delhi?"}
       ],
       "tools": [{
         "type": "function",
         "function": {
           "name": "getWeather",
           "description": "Get the current weather",
           "parameters": {
             "type": "object",
             "properties": {
               "location": {"type": "string", "description": "City and state"},
               "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
             },
             "required": ["location"]
           }
         }
       }],
       "tool_choice": "auto"
     }'

Speech to Text (Whisper)

Transcribe or translate audio using Groq’s Whisper model:

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@groq")

audio_file = open("/path/to/file.mp3", "rb")

# Transcription
transcription = portkey.audio.transcriptions.create(
  model="whisper-large-v3",
  file=audio_file
)
print(transcription.text)

# Translation
translation = portkey.audio.translations.create(
  model="whisper-large-v3",
  file=audio_file
)
print(translation.text)

import Portkey from 'portkey-ai';
import fs from 'fs';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@groq'
});

// Transcription
async function transcribe() {
  const transcription = await portkey.audio.transcriptions.create({
    file: fs.createReadStream("/path/to/file.mp3"),
        model: "whisper-large-v3"
  });
  console.log(transcription.text);
}
transcribe();

// Translation
async function translate() {
    const translation = await portkey.audio.translations.create({
        file: fs.createReadStream("/path/to/file.mp3"),
        model: "whisper-large-v3"
    });
    console.log(translation.text);
}
translate();

# Transcription
curl https://api.portkey.ai/v1/audio/transcriptions \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -H "x-portkey-provider: @groq" \
     -H "Content-Type: multipart/form-data" \
     -F "file=@/path/to/file.mp3" \
     -F "model=whisper-large-v3"

# Translation
curl https://api.portkey.ai/v1/audio/translations \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -H "x-portkey-provider: @groq" \
     -H "Content-Type: multipart/form-data" \
     -F "file=@/path/to/file.mp3" \
     -F "model=whisper-large-v3"

Text to Speech

Convert text to natural-sounding audio:

from portkey_ai import Portkey
from pathlib import Path

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@groq")

speech_file_path = Path(__file__).parent / "speech.mp3"
response = portkey.audio.speech.create(
  model="playai-tts",
  voice="Fritz-PlayAI",
  input="Today is a wonderful day to build something people love!"
)

with open(speech_file_path, "wb") as f:
    f.write(response.content)

import Portkey from 'portkey-ai';
import path from 'path';
import fs from 'fs';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@groq'
});

const speechFile = path.resolve("./speech.mp3");

async function main() {
    const mp3 = await portkey.audio.speech.create({
    model: "playai-tts",
    voice: "Fritz-PlayAI",
        input: "Today is a wonderful day to build something people love!"
  });
  const buffer = Buffer.from(await mp3.arrayBuffer());
  await fs.promises.writeFile(speechFile, buffer);
}

main();

curl https://api.portkey.ai/v1/audio/speech \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -H "x-portkey-provider: @groq" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "playai-tts",
       "voice": "Fritz-PlayAI",
       "input": "Today is a wonderful day to build something people love!"
     }' \
     --output speech.mp3

Supported Models

Groq Models

View the complete list of models available on Groq

Next Steps

Gateway Configs

Add fallbacks, load balancing, and more

Observability

Monitor and trace your Groq requests

Prompt Library

Manage and version your prompts

Metadata

Add custom metadata to requests

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation

​Quick Start

​Add Provider in Model Catalog

Complete Setup Guide

​Groq Capabilities

​Tool Calling

Supported Models

​Speech to Text (Whisper)

​Text to Speech

​Supported Models

Groq Models

​Next Steps

Gateway Configs

Observability

Prompt Library

Metadata

SDK Reference

Quick Start

Add Provider in Model Catalog

Groq Capabilities

Tool Calling

Speech to Text (Whisper)

Text to Speech

Supported Models

Next Steps