AzureOpenAITokenizerParameters Class
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
The AzureOpenAITokenizerParameters.
public class AzureOpenAITokenizerParameterstype AzureOpenAITokenizerParameters = classPublic Class AzureOpenAITokenizerParameters- Inheritance
-
AzureOpenAITokenizerParameters
Constructors
| AzureOpenAITokenizerParameters() |
Initializes a new instance of AzureOpenAITokenizerParameters. |
Properties
| AllowedSpecialTokens |
(Optional) Only applies if the unit is set to azureOpenAITokens. This parameter defines a collection of special tokens that are permitted within the tokenization process. |
| EncoderModelName |
Only applies if the unit is set to azureOpenAITokens. Options include 'R50k_base', 'P50k_base', 'P50k_edit' and 'CL100k_base'. The default value is 'CL100k_base'. |
Applies to
Collaborate with us on GitHub
The source for this content can be found on GitHub, where you can also create and review issues and pull requests. For more information, see our contributor guide.
Azure SDK for .NET