Edit

Share via

Ollama integration updates

  • Article

The .NET Aspire Community Toolkit 9.0 release of the Ollama integrations introduces significant changes. These updates enhance functionality and better integrate with the .NET Aspire API design.

Version introduced

.NET Aspire Community Toolkit 9.0 GA

Previous behavior

The Ollama integration includes both hosting and client integration updates, both are details in the following sections.

Ollama hosting

The Ollama hosting resource had to be provided as the reference to other resources and it would provide model information as a set of environment variables. The connection string was only the HTTP endpoint and not in a real "connection string" format.

OllamaSharp client

Supports v3 of the library and doesn't support the Microsoft.Extensions.AI (M.E.AI) interfaces.

New behavior

The new behavior includes the following updates.

Models as resources

In earlier versions, models were added to the Ollama resource, and you had to pass this resource as a reference. This approach required workarounds to set and discover the default model. In version 9.0, we introduced the OllamaModelResource. This resource can be passed as a reference and provides connection information to clients about which model to use.

New connection string format

Originally the "connection string" from an Ollama resource was just the HTTP endpoint, but to support the Model as resource feature better, the resources create a "real" connection string of Endpoint=<...>;Model=<...>. The Model part is only included if you're passing the OllamaModelResource.

OllamaSharp 4 and Microsoft.Extensions.AI

OllamaSharp updated to a new major version and now supports the interfaces from Microsoft.Extensions.AI. You can register the IOllamaApiClient (native OllamaSharp client), or from M.E.AI IChatClient and IEmbeddingGenerator<TInput,TEmbedding> (depending on your model type). Using these new interfaces makes code more portable across LLM/SLM options.

API deprecations and removals

With all the refactoring, some APIs are being deprecated or removed.

Type of breaking change

This change is a binary incompatible and behavioral change.

Reason for change

The changes aim to make the library more functional and better integrated with the .NET Aspire API design.

Upgrade to .NET Aspire 9.0.

Affected APIs