.NET Aspire Community Toolkit Ollama integration
Includes: 
Hosting integration and
Client integration
Note
This integration is part of the .NET Aspire Community Toolkit and isn't officially supported by the .NET Aspire team.
In this article, you learn how to use the .NET Aspire Ollama hosting integration to host 📦 OllamaSharp client.
Hosting integration
To model the Ollama server, install the 📦 CommunityToolkit.Aspire.Hosting.Ollama NuGet package in the app host project.
dotnet add package CommunityToolkit.Aspire.Hosting.Ollama
For more information, see dotnet add package or Manage package dependencies in .NET applications.
Add Ollama resource
In the app host project, register and consume the Ollama integration using the AddOllama extension method to add the Ollama container to the application builder. You can then add models to the container, which downloads and run when the container starts, using the AddModel extension method.
var ollama = builder.AddOllama("ollama")
.AddModel("llama3");
When .NET Aspire adds a container image to the app host, as shown in the preceding example with the docker.io/ollama/ollama image, it creates a new Ollama instance on your local machine. For more information, see Container resource lifecycle.
Download the LLM
When the Ollama container for this integration first spins up, it downloads one or more configured LLMs. The progress of this download displays in the State column for this integration on the Aspire orchestration app.
Important
Keep the .NET Aspire orchestration app open until the download is complete, otherwise the download will be cancelled.
Cache the LLM
One or more LLMs are downloaded into the container which Ollama is running from, and by default this container is ephemeral. If you need to persist one or more LLMs across container restarts, you need to mount a volume into the container using the WithDataVolume method.
var ollama = builder.AddOllama("ollama")
.AddModel("llama3")
.WithDataVolume();
Use GPUs when available
One or more LLMs are downloaded into the container which Ollama is running from, and by default this container runs on CPU. If you need to run the container in GPU you need to pass a parameter to the container runtime args.
var ollama = builder.AddOllama("ollama")
.AddModel("llama3")
.WithContainerRuntimeArgs("--gpus=all");
For more information, see GPU support in Docker Desktop.
Open WebUI support
The Ollama integration also provided support for running Open WebUI and having it communicate with the Ollama container.
var ollama = builder.AddOllama("ollama")
.AddModel("llama3")
.WithOpenWebUI();
Client integration
To get started with the .NET Aspire OllamaSharp integration, install the CommunityToolkit.Aspire.OllamaSharp NuGet package in the client-consuming project, that is, the project for the application that uses the Ollama client.
dotnet add package CommunityToolkit.Aspire.OllamaSharp
Add Ollama client API
In the Program.cs file of your client-consuming project, call the AddOllamaClientApi extension to register an IOllamaClientApi for use via the dependency injection container.
builder.AddOllamaClientApi("ollama");
After adding IOllamaClientApi to the builder, you can get the IOllamaClientApi instance using dependency injection. For example, to retrieve your context object from service:
public class ExampleService(IOllamaClientApi ollama)
{
// Use ollama...
}
Access the Ollama server in other services
The Ollama hosting integration exposes the endpoint of the Ollama server as a connection string that can be accessed from other services in the application.
var connectionString = builder.Configuration.GetConnectionString("ollama");
See also
.NET Aspire