AzureOpenAITokenizerParameters Class

Definition

Namespace:

Azure.Search.Documents.Indexes.Models

Assembly:

Azure.Search.Documents.dll

Package:

Azure.Search.Documents v11.7.0-beta.2

Source:

AzureOpenAITokenizerParameters.cs

The AzureOpenAITokenizerParameters.

public class AzureOpenAITokenizerParameters

type AzureOpenAITokenizerParameters = class

Public Class AzureOpenAITokenizerParameters

Inheritance

Object

AzureOpenAITokenizerParameters

Constructors

AzureOpenAITokenizerParameters()

Initializes a new instance of AzureOpenAITokenizerParameters.

Properties

AllowedSpecialTokens	(Optional) Only applies if the unit is set to azureOpenAITokens. This parameter defines a collection of special tokens that are permitted within the tokenization process.
EncoderModelName	Only applies if the unit is set to azureOpenAITokens. Options include 'R50k_base', 'P50k_base', 'P50k_edit' and 'CL100k_base'. The default value is 'CL100k_base'.