AGENT SKILL

azure-ai-voicelive-dotnet

Azure AI Voice Live SDK for .NET. Build real-time voice AI applications with bidirectional WebSocket communication. Use for voice assistants, conversational AI, real-time speech-to-speech, and voice-enabled chatbots. Triggers: "voice live", "real-time voice", "VoiceLiveClient", "VoiceLiveSession", "voice assistant .NET", "bidirectional audio", "speech-to-speech".

License 许可
MIT
最近更新
2026-05-11
Stars
★ 3
Forks
1

01 前言 Front Matter

name
azure-ai-voicelive-dotnet
license
MIT
metadata
{ "author": "Microsoft", "package": "Azure.AI.VoiceLive", "version": "1.0.0" }
description
Azure AI Voice Live SDK for .NET. Build real-time voice AI applications with bidirectional WebSocket communication. Use for voice assistants, conversational AI, real-time speech-to-speech, and voice-enabled chatbots. Triggers: "voice live", "real-time voice", "VoiceLiveClient", "VoiceLiveSession", "voice assistant .NET", "bidirectional audio", "speech-to-speech".

03 SKILL.md 正文

Azure.AI.VoiceLive (.NET)

Real-time voice AI SDK for building bidirectional voice assistants with Azure AI.

Installation

dotnet add package Azure.AI.VoiceLive
dotnet add package Azure.Identity
dotnet add package NAudio                    # For audio capture/playback

Current Versions: Stable v1.0.0, Preview v1.1.0-beta.1

Environment Variables

AZURE_VOICELIVE_ENDPOINT=https://<resource>.services.ai.azure.com/  # Required: Voice Live endpoint
AZURE_VOICELIVE_MODEL=gpt-4o-realtime-preview  # Required: model deployment name
AZURE_VOICELIVE_VOICE=en-US-AvaNeural  # Optional: Voice Live voice name
AZURE_VOICELIVE_API_KEY=<your-api-key>  # Only required for AzureKeyCredential auth
AZURE_TOKEN_CREDENTIALS=prod  # Required only if DefaultAzureCredential is used in production

Authentication

Microsoft Entra Token Credential

using Azure.Identity;
using Azure.AI.VoiceLive;

Uri endpoint = new Uri("https://your-resource.cognitiveservices.azure.com");
// Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS=<specific_credential>
var credential = new DefaultAzureCredential(
    DefaultAzureCredential.DefaultEnvironmentVariableName
);
// Or use a specific credential directly in production:
// See https://learn.microsoft.com/dotnet/api/overview/azure/identity-readme?view=azure-dotnet#credential-classes
// var credential = new ManagedIdentityCredential();
VoiceLiveClient client = new VoiceLiveClient(endpoint, credential);

Required Role: Cognitive Services User (assign in Azure Portal → Access control)

API Key

Uri endpoint = new Uri("https://your-resource.cognitiveservices.azure.com");
AzureKeyCredential credential = new AzureKeyCredential("your-api-key");
VoiceLiveClient client = new VoiceLiveClient(endpoint, credential);

Client Hierarchy

VoiceLiveClient
└── VoiceLiveSession (WebSocket connection)
    ├── ConfigureSessionAsync()
    ├── GetUpdatesAsync() → SessionUpdate events
    ├── AddItemAsync() → UserMessageItem, FunctionCallOutputItem
    ├── SendAudioAsync()
    └── StartResponseAsync()

Core Workflow

1. Start Session and Configure

using Azure.Identity;
using Azure.AI.VoiceLive;

var endpoint = new Uri(Environment.GetEnvironmentVariable("AZURE_VOICELIVE_ENDPOINT"));
var client = new VoiceLiveClient(endpoint, new DefaultAzureCredential());

var model = "gpt-4o-mini-realtime-preview";

// Start session
using VoiceLiveSession session = await client.StartSessionAsync(model);

// Configure session
VoiceLiveSessionOptions sessionOptions = new()
{
    Model = model,
    Instructions = "You are a helpful AI assistant. Respond naturally.",
    Voice = new AzureStandardVoice("en-US-AvaNeural"),
    TurnDetection = new AzureSemanticVadTurnDetection()
    {
        Threshold = 0.5f,
        PrefixPadding = TimeSpan.FromMilliseconds(300),
        SilenceDuration = TimeSpan.FromMilliseconds(500)
    },
    InputAudioFormat = InputAudioFormat.Pcm16,
    OutputAudioFormat = OutputAudioFormat.Pcm16
};

// Set modalities (both text and audio for voice assistants)
sessionOptions.Modalities.Clear();
sessionOptions.Modalities.Add(InteractionModality.Text);
sessionOptions.Modalities.Add(InteractionModality.Audio);

await session.ConfigureSessionAsync(sessionOptions);

2. Process Events

await foreach (SessionUpdate serverEvent in session.GetUpdatesAsync())
{
    switch (serverEvent)
    {
        case SessionUpdateResponseAudioDelta audioDelta:
            byte[] audioData = audioDelta.Delta.ToArray();
            // Play audio via NAudio or other audio library
            break;
            
        case SessionUpdateResponseTextDelta textDelta:
            Console.Write(textDelta.Delta);
            break;
            
        case SessionUpdateResponseFunctionCallArgumentsDone functionCall:
            // Handle function call (see Function Calling section)
            break;
            
        case SessionUpdateError error:
            Console.WriteLine($"Error: {error.Error.Message}");
            break;
            
        case SessionUpdateResponseDone:
            Console.WriteLine("\n--- Response complete ---");
            break;
    }
}

3. Send User Message

await session.AddItemAsync(new UserMessageItem("Hello, can you help me?"));
await session.StartResponseAsync();

4. Function Calling

// Define function
var weatherFunction = new VoiceLiveFunctionDefinition("get_current_weather")
{
    Description = "Get the current weather for a given location",
    Parameters = BinaryData.FromString("""
        {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state or country"
                }
            },
            "required": ["location"]
        }
        """)
};

// Add to session options
sessionOptions.Tools.Add(weatherFunction);

// Handle function call in event loop
if (serverEvent is SessionUpdateResponseFunctionCallArgumentsDone functionCall)
{
    if (functionCall.Name == "get_current_weather")
    {
        var parameters = JsonSerializer.Deserialize<Dictionary<string, string>>(functionCall.Arguments);
        string location = parameters?["location"] ?? "";
        
        // Call external service
        string weatherInfo = $"The weather in {location} is sunny, 75°F.";
        
        // Send response
        await session.AddItemAsync(new FunctionCallOutputItem(functionCall.CallId, weatherInfo));
        await session.StartResponseAsync();
    }
}

Voice Options

Voice Type Class Example
Azure Standard AzureStandardVoice "en-US-AvaNeural"
Azure HD AzureStandardVoice "en-US-Ava:DragonHDLatestNeural"
Azure Custom AzureCustomVoice Custom voice with endpoint ID

Supported Models

Model Description
gpt-4o-realtime-preview GPT-4o with real-time audio
gpt-4o-mini-realtime-preview Lightweight, fast interactions
phi4-mm-realtime Cost-effective multimodal

Key Types Reference

Type Purpose
VoiceLiveClient Main client for creating sessions
VoiceLiveSession Active WebSocket session
VoiceLiveSessionOptions Session configuration
AzureStandardVoice Standard Azure voice provider
AzureSemanticVadTurnDetection Voice activity detection
VoiceLiveFunctionDefinition Function tool definition
UserMessageItem User text message
FunctionCallOutputItem Function call response
SessionUpdateResponseAudioDelta Audio chunk event
SessionUpdateResponseTextDelta Text chunk event

Best Practices

  1. Always set both modalities — Include Text and Audio for voice assistants
  2. Use AzureSemanticVadTurnDetection — Provides natural conversation flow
  3. Configure appropriate silence duration — 500ms typical to avoid premature cutoffs
  4. Use using statement — Ensures proper session disposal
  5. Handle all event types — Check for errors, audio, text, and function calls
  6. Use DefaultAzureCredential — Never hardcode API keys

Error Handling

if (serverEvent is SessionUpdateError error)
{
    if (error.Error.Message.Contains("Cancellation failed: no active response"))
    {
        // Benign error, can ignore
    }
    else
    {
        Console.WriteLine($"Error: {error.Error.Message}");
    }
}

Audio Configuration

  • Input Format: InputAudioFormat.Pcm16 (16-bit PCM)
  • Output Format: OutputAudioFormat.Pcm16
  • Sample Rate: 24kHz recommended
  • Channels: Mono

Related SDKs

SDK Purpose Install
Azure.AI.VoiceLive Real-time voice (this SDK) dotnet add package Azure.AI.VoiceLive
Microsoft.CognitiveServices.Speech Speech-to-text, text-to-speech dotnet add package Microsoft.CognitiveServices.Speech
NAudio Audio capture/playback dotnet add package NAudio

Reference Links

Resource URL
NuGet Package https://www.nuget.org/packages/Azure.AI.VoiceLive
API Reference https://learn.microsoft.com/dotnet/api/azure.ai.voicelive
GitHub Source https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/ai/Azure.AI.VoiceLive
Quickstart https://learn.microsoft.com/azure/ai-services/speech-service/voice-live-quickstart

04 同仓库更多 Skills

qhjqhj00
qhjqhj00/research-skills-pool

transformer-lens-interpretability

Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.

★ 3
qhjqhj00
qhjqhj00/research-skills-pool

wiki-vitepress

Packages generated wiki Markdown into a VitePress static site with dark theme, dark-mode Mermaid diagrams with click-to-zoom, and production build output. Use when the user wants to create a browsable website from generated wiki pages.

★ 3
qhjqhj00
qhjqhj00/research-skills-pool

phoenix-observability

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

★ 3
qhjqhj00
qhjqhj00/research-skills-pool

frontend-ui-dark-ts

Build dark-themed React applications using Tailwind CSS with custom theming, glassmorphism effects, and Framer Motion animations. Use when creating dashboards, admin panels, or data-rich interfaces with a refined dark aesthetic.

★ 3
qhjqhj00
qhjqhj00/research-skills-pool

tensorboard

Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit

★ 3
qhjqhj00
qhjqhj00/research-skills-pool

markdown-mermaid-writing

Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.

★ 3