Documentation
Model Registry
Models you can pick when you deploy an agent.
Chat models for text and files. Use a Live model below if you need voice.
| Model | Input | Output |
|---|---|---|
| gemini-3.6-flash | Text, image, audio, video, document | Text |
| gemini-3.6-pro | Text, image, audio, video, document | Text |
| gemini-3.5-flash | Text, image, audio, video, document | Text |
| gemini-3.5-flash-lite | Text, image, audio, video, document | Text |
| gemini-3.5-pro | Text, image, audio, video, document | Text |
| gemini-2.5-flash | Text, image, audio, video, document | Text |
| gemini-2.5-flash-lite | Text, image, audio, video, document | Text |
| gemini-2.5-pro | Text, image, audio, video, document | Text |
Google Live
Voice agents. Use these when the agent should listen and speak in real time.
| Model | Name | Runs on | Input | Output |
|---|---|---|---|---|
| gemini-3.8-live | Gemini 3.8 Live | Gemini API | Text, audio, image, video | Text, audio |
| gemini-live-2.5-flash-native-audio | Gemini Live 2.5 Flash | Vertex | Text, audio, image, video | Text, audio |
| gemini-3.1-flash-live-preview | Gemini 3.1 Flash Live | Gemini API | Text, audio, image, video | Text, audio |
| gemini-2.5-flash-native-audio-preview-12-2025 | Gemini 2.5 Flash Live | Gemini API | Text, audio, image, video | Text, audio |
Background tools
On a Live voice agent, a slow tool can freeze the conversation — the model goes quiet until the tool returns. Background tools fix that: the agent keeps speaking while the tool runs, then folds the result in when it arrives.
You control this with two fields on each tool: response_scheduling (run in the background, and say when to react) and blocking (hold the agent still until the tool finishes). Non-Live models ignore both.
Defaults by model
Start from the model default, then override only the tools that need different behavior.
| Model | Default | What you usually do |
|---|---|---|
| gemini-3.8-live | Background (WHEN_IDLE) | Leave most tools alone. Set blocking: true only when the agent must wait (payments, confirmations). |
| Other Live models | Blocking | Set response_scheduling on tools that can run while the agent keeps talking. |
When the agent reacts
Pick a scheduling mode for each background tool. This only matters when the tool is not blocking.
| Mode | What happens | Good for |
|---|---|---|
| WHEN_IDLE | Waits for a natural pause, then brings the result up. Default when a tool runs in the background. | Lookups, chart pulls, most API calls |
| INTERRUPT | Breaks into whatever the agent is saying as soon as the result lands. | Urgent alerts, safety checks |
| SILENT | Takes the result in without mentioning it, for later in the conversation. | Prefetch, context enrichment |
In the console
- Open your agent and pick a Live model that supports background tools.
- On each tool, use Run in background to choose whether the agent keeps talking while it runs.
- For background tools, choose when the agent should react to the result (
WHEN_IDLE,INTERRUPT, orSILENT). - On
gemini-3.8-live, turn background off (blocking) only for tools that must finish before the next spoken reply.
In code (Python)
Decorate tools with response_scheduling or blocking. Do not set both on the same tool — a blocking tool has no result left to schedule.
from hikigai.agentsdk import tool
# Background: agent keeps talking; reacts at a natural pause (default on 3.8).
@tool(response_scheduling="WHEN_IDLE")
def fetch_record(patient_id: str) -> dict:
"""Pull a chart. Slow, and the agent can keep talking meanwhile."""
return lookup(patient_id)
# Urgent: speak as soon as the result arrives.
@tool(response_scheduling="INTERRUPT")
def check_allergy(patient_id: str, drug: str) -> dict:
"""Safety check — interrupt if something is wrong."""
return allergy_api(patient_id, drug)
# Prefetch: absorb the result quietly for later turns.
@tool(response_scheduling="SILENT")
def warm_context(patient_id: str) -> dict:
"""Load context the agent may need later."""
return prefetch(patient_id)
# Blocking: required on gemini-3.8-live when the agent must wait.
@tool(blocking=True)
def take_payment(amount: float) -> dict:
"""Must settle before the agent says anything else."""
return charge(amount)In code (Node)
Same idea in TypeScript — use responseScheduling or blocking on tool().
import { tool } from '@hikigai/agent-sdk'
const fetchRecord = tool({
responseScheduling: 'WHEN_IDLE',
description: 'Pull a chart while the agent keeps talking.',
})(async (patientId: string) => lookup(patientId))
const takePayment = tool({
blocking: true,
description: 'Must settle before the agent speaks again.',
})(async (amount: number) => charge(amount))Quick rules
- Prefer background for slow, non-critical work (lookups, EHR pulls, weather, notes).
- Prefer blocking for irreversible or gatekeeping steps (payments, identity confirmation, "are you sure?" actions).
- Start with
WHEN_IDLE; useINTERRUPTonly when the user needs to hear the result immediately. - Never set
blockingandresponse_schedulingtogether — the SDK rejects that combination.
Anthropic
| Model | Input | Output |
|---|---|---|
| claude-3.5-sonnet | Text, image, document | Text |
| claude-3-opus | Text, image | Text |
| claude-3-sonnet | Text, image | Text |
| claude-3-haiku | Text, image | Text |