Using the Postgres Vector Store connector (Preview)

Warning

The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release.

Overview

The Postgres Vector Store connector can be used to access and manage data in Postgres. The connector has the following characteristics.

Feature Area Support
Collection maps to Postgres table
Supported key property types
  • short
  • int
  • long
  • string
  • Guid
Supported data property types
  • bool
  • short
  • int
  • long
  • float
  • double
  • decimal
  • string
  • DateTime
  • DateTimeOffset
  • Guid
  • byte[]
  • bool Enumerables
  • short Enumerables
  • int Enumerables
  • long Enumerables
  • float Enumerables
  • double Enumerables
  • decimal Enumerables
  • string Enumerables
  • DateTime Enumerables
  • DateTimeOffset Enumerables
  • Guid Enumerables
Supported vector property types ReadOnlyMemory<float>
Supported index types Hnsw
Supported distance functions
  • CosineDistance
  • CosineSimilarity
  • DotProductSimilarity
  • EuclideanDistance
  • ManhattanDistance
Supported filter clauses
  • AnyTagEqualTo
  • EqualTo
Supports multiple vectors in a record Yes
IsFilterable supported? No
IsFullTextSearchable supported? No
StoragePropertyName supported? Yes

Getting started

Add the Postgres Vector Store connector NuGet package to your project.

dotnet add package Microsoft.SemanticKernel.Connectors.Postgres --prerelease

You can add the vector store to the IServiceCollection dependency injection container using extension methods provided by Semantic Kernel.

In this case, an instance of the Npgsql.NpgsqlDataSource class, which has vector capabilities enabled, will also be registered with the container.

using Microsoft.SemanticKernel;

// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddPostgresVectorStore("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");

Extension methods that take no parameters are also provided. These require an instance of the Npgsql.NpgsqlDataSource class to be separately registered with the dependency injection container.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Npgsql;

// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton<NpgsqlDataSource>(sp => 
{
    NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");
    dataSourceBuilder.UseVector();
    return dataSourceBuilder.Build();
});

builder.Services.AddPostgresVectorStore();

You can construct a Postgres Vector Store instance directly.

using Microsoft.SemanticKernel.Connectors.Postgres;
using Npgsql;

NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");
dataSourceBuilder.UseVector();
var dataSource = dataSourceBuilder.Build();

var connection = new PostgresVectorStore(dataSource);

It is possible to construct a direct reference to a named collection.

using Microsoft.SemanticKernel.Connectors.Postgres;
using Npgsql;

NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");
dataSourceBuilder.UseVector();
var dataSource = dataSourceBuilder.Build();

var collection = new PostgresVectorStoreRecordCollection<string, Hotel>(dataSource, "skhotels");

Data mapping

The Postgres Vector Store connector provides a default mapper when mapping from the data model to storage. This mapper does a direct conversion of the list of properties on the data model to the columns in Postgres.

It's also possible to override the default mapper behavior by providing a custom mapper via the PostgresVectorStoreRecordCollectionOptions<TRecord>.DictionaryCustomMapper property.

Property name override

You can override property names to use in storage that is different to the property names on the data model. The property name override is done by setting the StoragePropertyName option via the data model property attributes or record definition.

Here is an example of a data model with StoragePropertyName set on its attributes and how that will be represented in Postgres query.

using Microsoft.Extensions.VectorData;

public class Hotel
{
    [VectorStoreRecordKey]
    public ulong HotelId { get; set; }

    [VectorStoreRecordData(StoragePropertyName = "hotel_name")]
    public string? HotelName { get; set; }

    [VectorStoreRecordData(StoragePropertyName = "hotel_description")]
    public string? Description { get; set; }

    [VectorStoreRecordVector(Dimensions: 4, DistanceFunction.CosineDistance)]
    public ReadOnlyMemory<float>? DescriptionEmbedding { get; set; }
}
CREATE TABLE public."Hotels" (
    "HotelId" INTEGER NOT NULL,
    "hotel_name" TEXT ,
    "hotel_description" TEXT ,
    "DescriptionEmbedding" VECTOR(4) ,
    PRIMARY KEY ("HotelId")
);

Coming soon

More info coming soon.

JDBC

The JDBC connector can be used to connect to Postgres.