Part 6 in the “Local AI with Ollama and .NET” series: Part 1 – Ollama and .NET | Part 2 – Local RAG | Part 3 – AI Agents | Part 3.5 – MCP Server | Part 4 – Microsoft Agent Framework | Part 5 – Aspire | Version française
Asking a local model to reply in JSON directly in the prompt works most of the time. The problem is the rest of the time: the parsing blows up, and rarely at a good moment. Here’s a simple trick to get reliable JSON with Ollama and .NET.
A complete, working example is available here: mongeon/code-examples · structured-outputs-ollama-dotnet.
The problem with “reply in JSON”
To extract data from some text, the reflex is to write a prompt like:
Extract the information from this invoice and reply only in JSON
with the fields "vendor", "amount" and "date". No other text.
Most of the time, the model returns valid JSON and JsonSerializer.Deserialize does the job. But once in a while, the response comes with text around it (“Here is the requested JSON:”), with ```json fences, with a renamed field or with one comma too many. So we end up writing cleanup code like this:
// Remove the code fences and the text around the JSON
var text = response.Trim();
if (text.StartsWith("```json"))
{
text = text.Replace("```json", "").Replace("```", "");
}
var start = text.IndexOf('{');
var end = text.LastIndexOf('}');
text = text.Substring(start, end - start + 1);
var invoice = JsonSerializer.Deserialize<Invoice>(text);
This code works until the next variation in the model’s response. The root problem: we ask the model to produce JSON, but nothing forces it to.
Ollama’s format field
Since version 0.5, Ollama’s API accepts a full JSON schema in the format field of /api/chat. An example with curl:
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "Extract the info: Invoice from Hydro-Quebec, $142.50, June 15 2026" }
],
"stream": false,
"format": {
"type": "object",
"properties": {
"vendor": { "type": "string" },
"amount": { "type": "number" },
"date": { "type": "string" }
},
"required": ["vendor", "amount", "date"]
}
}'
With this field, the schema is enforced at the decoding level: for each generated token, the inference engine eliminates the tokens that would produce JSON that’s invalid against the schema. The response is therefore always a conforming JSON object, with no surrounding text and no code fences.
Note that Ollama also accepts plain "format": "json". That mode guarantees valid JSON, but not its structure. With a full schema, the structure is guaranteed too.
The .NET version with Microsoft.Extensions.AI
Writing JSON schemas by hand gets painful quickly. Microsoft.Extensions.AI can generate them from a C# type. Install the packages and pull a model:
dotnet add package Microsoft.Extensions.AI
dotnet add package OllamaSharp
ollama pull llama3.2
As in the previous articles in this series, OllamaApiClient implements IChatClient, so everything plugs in directly. Define a record describing the expected result:
using System.ComponentModel;
using Microsoft.Extensions.AI;
using OllamaSharp;
// The Description attributes are included in the schema sent to the model
public record Invoice(
[property: Description("The vendor name")] string Vendor,
[property: Description("The total amount, taxes included")] decimal Amount,
[property: Description("The date in YYYY-MM-DD format")] string Date);
And use the generic version of GetResponseAsync:
IChatClient client = new OllamaApiClient(
new Uri("http://localhost:11434"), "llama3.2");
var response = await client.GetResponseAsync<Invoice>(
"Extract the info (date in YYYY-MM-DD format): " +
"Invoice from Hydro-Quebec, $142.50, June 15 2026",
new ChatOptions { Temperature = 0 }); // temperature at 0 for more stable results
Invoice invoice = response.Result;
Console.WriteLine($"{invoice.Vendor}: ${invoice.Amount}");
GetResponseAsync<T> generates the JSON schema from the Invoice type, sends it in the format field and deserializes the response into a C# object. No more cleanup code.
If the model still returns something unusable, TryGetResult avoids the exception:
if (response.TryGetResult(out var result))
{
// We're working with a typed, valid object
}
else
{
// Log the raw response and handle the failure
}
Picking the right approach
There are several ways to get structured data out of a model and the choice depends on who consumes the output.
If the response is meant for a human, the prompt alone is enough. No need to constrain a response that nobody parses.
JSON mode ("format": "json") guarantees valid JSON, but of varying shape. I don’t really use it anymore: if you know the expected shape, you might as well provide the full schema.
The JSON schema is the right choice as soon as the output is consumed by code: data extraction, classification, ticket routing. That’s what GetResponseAsync<T> does for you.
Function calling is for something else: it lets the model pick a tool to call with typed arguments. If the model has to decide what to do, that’s function calling, and I cover it in the agents article. If it only has to answer in a precise format, a structured output is enough.
The limits with local models
The schema guarantees the structure, but not the content. A small model like llama3.2 forced to fill a required field will fill it even if it doesn’t have the information, with an empty string or a made-up value. So you still have to validate the content on the C# side, like any external input.
Complex schemas also cause problems. The bigger the schema (nested objects, long enums), the harder it is for a small local model to produce a sensible result. If the extraction goes off the rails, simplify the schema before switching models. In my tests, splitting a big extraction into two calls with simple types gives better results than a single call with a big type.
For dates, specify the expected format in the Description and repeat it in the prompt, because a small model will happily ignore the attribute alone. Then parse them yourself, it’s more predictable than hoping for a clean DateTime. And streaming brings nothing with a typed output: a half-deserialized object is useless, so GetResponseAsync<T> waits for the complete response.
Happy extracting! You can now delete your JSON cleanup code, but keep your validation.
This post was written with AI assistance and edited by me.