Are you sure your app is really listening to users? Most apps can talk — few can hold a conversation. In this post, I’ll show you how to wire ChatGPT into your .NET apps the way production systems do it: typed clients, DI, streaming, tool calling, structured output, resilience, and testing. By the end, you’ll have copy‑pasteable snippets you can run today and a mental model for building secure, reliable AI features.

What you’ll build/learn

• A clean C# client setup (official OpenAI .NET SDK) with DI and configuration.
• Chat basics and streaming tokens to your UI.
• Function/Tool calling to let the model trigger your code safely.
• Structured outputs via JSON Schema (goodbye brittle regex parsing!).
• Responses API (stateful workflows) at a glance.
• Resilience: retries, timeouts, rate‑limit backoff.
• Safety & cost hygiene: system prompts, output limits, logging.
• Testing: mockable handlers and golden tests.

All code targets .NET 8 and uses modern C# features. Create a fresh dotnet new webapi or console app and follow along.

Project setup

Install packages

# Official OpenAI .NET SDK
 dotnet add package OpenAI
# (Optional) Polly for resilience
 dotnet add package Polly

Configure secrets

# macOS/Linux
export OPENAI_API_KEY="<your-key>"

# Windows (PowerShell)
$env:OPENAI_API_KEY="<your-key>"

Prefer environment variables or dotnet user-secrets in development; never commit keys.

Register clients (ASP.NET Core)

using OpenAI.Chat;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSingleton<ChatClient>(_ =>
{
    var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
    var model = "gpt-4o"; // pick your model snapshot
    return new ChatClient(model, apiKey);
});

var app = builder.Build();
app.MapGet("/health", () => Results.Ok("ok"));
app.Run();

This registers a thread‑safe ChatClient singleton (good for connection reuse).

First chat: minimal and fast

using OpenAI.Chat;

public static class HelloChat
{
    public static async Task DemoAsync(ChatClient client)
    {
        var completion = await client.CompleteChatAsync(
            [ new UserChatMessage("Write a haiku about C# async.") ]);

        Console.WriteLine(completion.Content[0].Text);
    }
}

Tips:

• Keep your system message explicit in production (tone, persona, safety rails).
• Bound output size with max tokens when you expect short answers.

var options = new ChatCompletionOptions { MaxOutputTokenCount = 300 };
await client.CompleteChatAsync([
    new SystemChatMessage("You are a precise assistant. Answers are concise."),
    new UserChatMessage("Summarize the SOLID principles in one paragraph."),
], options);

Real‑time UX: token streaming

Streaming reduces perceived latency dramatically. Print tokens as they arrive or push them over SignalR/WebSocket.

using OpenAI.Chat;

public static class StreamingDemo
{
    public static async Task RunAsync(ChatClient client)
    {
        await foreach (var update in client.CompleteChatStreamingAsync(
            [ new UserChatMessage("Explain Span<T> in 2 sentences.") ]))
        {
            if (update.ContentUpdate.Count > 0)
                Console.Write(update.ContentUpdate[0].Text);
        }
        Console.WriteLine();
    }
}

UI idea: Push each Text chunk to the browser via SignalR for a typing effect.

Let the model call your code (Tools / Function Calling)

Function calling lets the model request that your app run specific functions. You describe those functions with a JSON Schema, the model returns a tool invocation with arguments, and you decide whether to execute.

Example: a weather bot that may ask for the user’s location, then query weather.

using OpenAI.Chat;

static string GetCurrentLocation() => "Sofia";
static string GetCurrentWeather(string location, string unit = "celsius")
    => unit == "celsius" ? "23 celsius" : "73 fahrenheit";

static readonly ChatTool getLocationTool = ChatTool.CreateFunctionTool(
    functionName: nameof(GetCurrentLocation),
    functionDescription: "Get the user's current location");

static readonly ChatTool getWeatherTool = ChatTool.CreateFunctionTool(
    functionName: nameof(GetCurrentWeather),
    functionDescription: "Get the current weather in a given location",
    functionParameters: BinaryData.FromBytes(
        """
        {
          "type": "object",
          "properties": {
            "location": { "type": "string" },
            "unit": { "type": "string", "enum": ["celsius","fahrenheit"] }
          },
          "required": ["location"]
        }
        """u8.ToArray()));

public static async Task RunAsync(ChatClient client)
{
    var messages = new List<ChatMessage>
    {
        new SystemChatMessage("You are a weather assistant. Prefer celsius."),
        new UserChatMessage("What's the weather today?")
    };

    var options = new ChatCompletionOptions { Tools = { getLocationTool, getWeatherTool } };

    while (true)
    {
        var completion = await client.CompleteChatAsync(messages, options);

        if (completion.FinishReason == ChatFinishReason.ToolCalls)
        {
            // Add assistant message (with tool calls) to the conversation
            messages.Add(new AssistantChatMessage(completion));

            foreach (var call in completion.ToolCalls)
            {
                switch (call.FunctionName)
                {
                    case nameof(GetCurrentLocation):
                    {
                        var result = GetCurrentLocation();
                        messages.Add(new ChatRequestToolMessage(call, result));
                        break;
                    }
                    case nameof(GetCurrentWeather):
                    {
                        var args = System.Text.Json.JsonDocument.Parse(call.FunctionArguments);
                        var location = args.RootElement.GetProperty("location").GetString()!;
                        var unit = args.RootElement.TryGetProperty("unit", out var u) ? u.GetString()! : "celsius";
                        var result = GetCurrentWeather(location, unit);
                        messages.Add(new ChatRequestToolMessage(call, result));
                        break;
                    }
                }
            }
        }
        else
        {
            // Normal assistant message — we’re done
            messages.Add(new AssistantChatMessage(completion));
            Console.WriteLine(messages[^1].Content[0].Text);
            break;
        }
    }
}

Safety notes:

• Validate every argument coming from the model.
• Keep tools idempotent and side‑effect aware.
• Log tool calls; rate‑limit dangerous ones.

Structured outputs (JSON you can trust)

When you expect a specific shape, constrain the model with a JSON Schema. It’s far more robust than free‑form text parsing.

using OpenAI.Chat;

public record MathReasoning(List<Step> Steps, string FinalAnswer)
{
    public record Step(string Explanation, string? Expression);
}

public static async Task<MathReasoning?> SolveAsync(ChatClient client)
{
    var options = new ChatCompletionOptions
    {
        ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat(
            jsonSchemaFormatName: "math_reasoning",
            jsonSchema: BinaryData.FromString(
                """
                {
                  "type":"object",
                  "properties":{
                    "steps":{ "type":"array", "items":{
                      "type":"object",
                      "properties":{
                        "explanation":{"type":"string"},
                        "expression":{"type":["string","null"]}
                      },
                      "required":["explanation"]
                    }},
                    "finalAnswer":{"type":"string"}
                  },
                  "required":["steps","finalAnswer"]
                }
                """)
        )
    };

    var completion = await client.CompleteChatAsync([
        new SystemChatMessage("Return ONLY valid JSON for the schema."),
        new UserChatMessage("Solve 8x + 7 = -23; show steps then final answer.")
    ], options);

    return System.Text.Json.JsonSerializer.Deserialize<MathReasoning>(
        completion.Content[0].Text);
}

Tip: Fail fast if JSON parse fails; fall back to a simpler prompt or ask the model to self‑repair.

Responses API: stateful workflows in one call (overview)

For multi‑step tasks (search files, browse, reason, reply) you can use the Responses API and its typed client. It can stream and surface intermediate tool calls.

using OpenAI.Responses;

var responseClient = new OpenAIResponseClient(
    model: "gpt-4o-mini",
    apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY")!);

var response = await responseClient.CreateResponseAsync(
    userInputText: "Give me a happy news headline today.",
    new ResponseCreationOptions
    {
        Tools = { ResponseTool.CreateWebSearchTool() } // optional tool
    });

foreach (var item in response.OutputItems)
{
    if (item is MessageResponseItem msg)
        Console.WriteLine($"[{msg.Role}] {msg.Content?.FirstOrDefault()?.Text}");
}

Use this when you want tool orchestration without hand‑rolling the loop.

Resilience for production

Timeouts & retry/backoff

using Polly;
using Polly.Retry;

var retry = Policy
    .Handle<HttpRequestException>()
    .OrResult<HttpResponseMessage>(r => (int)r.StatusCode == 429 || (int)r.StatusCode >= 500)
    .WaitAndRetryAsync(5, (attempt, result, ctx) =>
    {
        // Respect Retry-After when available
        if (result.Result?.Headers.RetryAfter?.Delta is TimeSpan ra)
            return ra;
        return TimeSpan.FromSeconds(Math.Pow(2, attempt));
    });

If you wrap the SDK’s underlying HttpClient, configure a sensible overall timeout (e.g., 60-90s for long generations) and cancellation tokens for user‑initiated aborts.

Rate‑limit hygiene

• Queue user requests; apply per‑user quotas.
• Prefer streaming over huge max tokens.
• Cache deterministic prompts (e.g., static summaries) when possible.

Idempotency & observability

• Use idempotency keys for critical actions (retries without duplication).
• Log prompts (redacted), tokens, model, latency, and tool calls.

Architecture: where does ChatGPT fit?

+----------------------------+           +-------------------+
|        Your UI (Web)       |  SignalR  |  Streaming tokens |
+-------------+--------------+ <-------- +-------------------+
              | REST
              v
+-------------+--------------+      +-----------------------+
|  ASP.NET Core API         | ----> |  OpenAI Chat/Response |
|  (Controllers/Handlers)   |       |  Clients (Singleton)  |
+-------------+--------------+      +-----------+-----------+
              |                               |
              | Tools (function calls)        | Embeddings/Images/etc
              v                               v
     +--------+--------+            +---------+---------+
     | Domain Services |            | Storage/Vector DB |
     +-----------------+            +-------------------+

Guidelines: keep prompting at the edge, business logic in services, and tool effects explicit. Treat the model as a collaborator, not the boss.

End‑to‑end sample: Ask Docs (RAG‑lite)

A minimal endpoint that answers questions over a few in‑memory “docs”, with structured answers and a citation list.

using Microsoft.AspNetCore.Mvc;
using OpenAI.Chat;

[ApiController]
[Route("api/ask")]
public class AskController(ChatClient chat) : ControllerBase
{
    private static readonly string[] Docs =
    [
        "Logging: Use Serilog in production.",
        "Rate limits: Back off on 429 using Retry-After.",
        "Security: Never log raw secrets or PII.",
    ];

    [HttpPost]
    public async Task<IActionResult> Ask([FromBody] string question)
    {
        var system = "You answer strictly from the provided docs."
                   + " If unsure, say you don’t know. Return JSON with fields: answer, citations (array of strings).";

        var schema = BinaryData.FromString(
            """
            {"type":"object","properties":{"answer":{"type":"string"},"citations":{"type":"array","items":{"type":"string"}}},"required":["answer","citations"]}
            """
        );

        var options = new ChatCompletionOptions
        {
            ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat("qa", schema)
        };

        var content = string.Join("\n- ", Docs);

        var completion = await chat.CompleteChatAsync([
            new SystemChatMessage(system),
            new UserChatMessage($"Docs:\n- {content}\n\nQuestion: {question}")
        ], options);

        var json = completion.Content[0].Text;
        return Content(json, "application/json");
    }
}

This pattern (strict grounding + JSON schema) is a great default for internal tools and admin panels.

Testing strategies

• Golden tests: freeze prompts and assert stable shapes (not exact wording). Store a short expected JSON fixture.
• Mocking: wrap the SDK behind an interface and inject a fake implementation for unit tests.
• Contract tests: a small suite that hits the real API nightly with tiny prompts.

Example: wrapper interface

public interface IChat
{
    Task<string> AskAsync(string prompt, CancellationToken ct = default);
}

public sealed class OpenAiChat(ChatClient client) : IChat
{
    public async Task<string> AskAsync(string prompt, CancellationToken ct = default)
    {
        var r = await client.CompleteChatAsync([ new UserChatMessage(prompt) ], cancellationToken: ct);
        return r.Content[0].Text;
    }
}

In tests, swap OpenAiChat with a fake that returns canned data.

Cost, safety, and UX tips

• Right‑size models: start with a small, cheap model; escalate if needed.
• Bound everything: max tokens, max tool invocations, max recursion.
• Prompt discipline: keep system prompts short and versioned.
• User controls: a visible Stop generating button wired to a CancellationToken.
• Privacy: redact PII before logging; hash user IDs; prefer aggregated metrics.

FAQ: Common hurdles when integrating ChatGPT in C#

Conclusion: Ship conversations, not demos

You don’t need a research team to build a great AI UX. With a typed client, streaming, tools, and structured outputs, your .NET app can listen, reason, and act safely. Grab the snippets above and wire them into your next feature — and tell me in the comments what you’re building next!