Using the Couchbase connector (Preview)

Warning

The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release.

Overview

The Couchbase Vector Store connector can be used to access and manage data in Couchbase. The connector has the following characteristics.

Feature Area	Support
Collection maps to	Couchbase collection
Supported key property types	string
Supported data property types	All types that are supported by System.Text.Json (either built-in or by using a custom converter)
Supported vector property types	ReadOnlyMemory<float>
Supported index types	N/A
Supported distance functions	CosineSimilarity DotProductSimilarity EuclideanDistance
Supported filter clauses	AnyTagEqualTo EqualTo
Supports multiple vectors in a record	Yes
IsFilterable supported?	No
IsFullTextSearchable supported?	No
StoragePropertyName supported?	No, use JsonSerializerOptions and JsonPropertyNameAttribute instead. See here for more info.

Getting Started

Add the Couchbase Vector Store connector NuGet package to your project.

dotnet add package CouchbaseConnector.SemanticKernel --prerelease

You can add the vector store to the dependency injection container available on the KernelBuilder or to the IServiceCollection dependency injection container using extension methods provided by Semantic Kernel.

using Microsoft.SemanticKernel;

// Using Kernel Builder.
var kernelBuilder = Kernel.CreateBuilder()
    .AddCouchbaseVectorStore(
        connectionString: "couchbases://your-cluster-address",
        username: "username",
        password: "password",
        bucketName: "bucket-name",
        scopeName: "scope-name");

using Microsoft.Extensions.DependencyInjection;

// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddCouchbaseVectorStore(
    connectionString: "couchbases://your-cluster-address",
    username: "username",
    password: "password",
    bucketName: "bucket-name",
    scopeName: "scope-name");

Extension methods that take no parameters are also provided. These require an instance of the IScope class to be separately registered with the dependency injection container.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Couchbase;
using Couchbase.KeyValue;

// Using Kernel Builder.
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.Services.AddSingleton<ICluster>(sp =>
{
    var clusterOptions = new ClusterOptions
    {
        ConnectionString = "couchbases://your-cluster-address",
        UserName = "username",
        Password = "password"
    };

    return Cluster.ConnectAsync(clusterOptions).GetAwaiter().GetResult();
});

kernelBuilder.Services.AddSingleton<IScope>(sp =>
{
    var cluster = sp.GetRequiredService<ICluster>();
    var bucket = cluster.BucketAsync("bucket-name").GetAwaiter().GetResult();
    return bucket.Scope("scope-name");
});

// Add Couchbase Vector Store
kernelBuilder.Services.AddCouchbaseVectorStore();

using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Couchbase.KeyValue;
using Couchbase;

// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSingleton<ICluster>(sp =>
{
    var clusterOptions = new ClusterOptions
    {
        ConnectionString = "couchbases://your-cluster-address",
        UserName = "username",
        Password = "password"
    };

    return Cluster.ConnectAsync(clusterOptions).GetAwaiter().GetResult();
});

builder.Services.AddSingleton<IScope>(sp =>
{
    var cluster = sp.GetRequiredService<ICluster>();
    var bucket = cluster.BucketAsync("bucket-name").GetAwaiter().GetResult();
    return bucket.Scope("scope-name");
});

// Add Couchbase Vector Store
builder.Services.AddCouchbaseVectorStore();

You can construct a Couchbase Vector Store instance directly.

using Couchbase;
using Couchbase.KeyValue;
using Couchbase.SemanticKernel;

var clusterOptions = new ClusterOptions
{
    ConnectionString = "couchbases://your-cluster-address",
    UserName = "username",
    Password = "password"
};

var cluster = await Cluster.ConnectAsync(clusterOptions);

var bucket = await cluster.BucketAsync("bucket-name");
var scope = bucket.Scope("scope-name");

var vectorStore = new CouchbaseVectorStore(scope);

It is possible to construct a direct reference to a named collection.

using Couchbase;
using Couchbase.KeyValue;
using Couchbase.SemanticKernel;

var clusterOptions = new ClusterOptions
{
    ConnectionString = "couchbases://your-cluster-address",
    UserName = "username",
    Password = "password"
};

var cluster = await Cluster.ConnectAsync(clusterOptions);
var bucket = await cluster.BucketAsync("bucket-name");
var scope = bucket.Scope("scope-name");

var collection = new CouchbaseFtsVectorStoreRecordCollection<Hotel>(
    scope,
    "hotelCollection");

Data mapping

The Couchbase connector uses System.Text.Json.JsonSerializer for data mapping. Properties in the data model are serialized into a JSON object and mapped to Couchbase storage.

Use the JsonPropertyName attribute to map a property to a different name in Couchbase storage. Alternatively, you can configure JsonSerializerOptions for advanced customization.

using Couchbase.SemanticKernel;
using Couchbase.KeyValue;
using System.Text.Json;

var jsonSerializerOptions = new JsonSerializerOptions
{
    PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};

var options = new CouchbaseFtsVectorStoreRecordCollectionOptions<Hotel>
{
    JsonSerializerOptions = jsonSerializerOptions
};

var collection = new CouchbaseFtsVectorStoreRecordCollection<Hotel>(scope, "hotels", options);

Using the above custom JsonSerializerOptions which is using CamelCase, the following data model will be mapped to the below json.

using System.Text.Json.Serialization;
using Microsoft.Extensions.VectorData;

public class Hotel
{
    [JsonPropertyName("hotelId")]
    [VectorStoreRecordKey]
    public string HotelId { get; set; }

    [JsonPropertyName("hotelName")]
    [VectorStoreRecordData]
    public string HotelName { get; set; }

    [JsonPropertyName("description")]
    [VectorStoreRecordData]
    public string Description { get; set; }

    [JsonPropertyName("descriptionEmbedding")]
    [VectorStoreRecordVector(Dimensions: 4, DistanceFunction.DotProductSimilarity)]
    public ReadOnlyMemory<float> DescriptionEmbedding { get; set; }
}

{
  "hotelId": "h1",
  "hotelName": "Hotel Happy",
  "description": "A place where everyone can be happy",
  "descriptionEmbedding": [0.9, 0.1, 0.1, 0.1]
}

Not supported

Not supported.

Not supported

Not supported.