Freigeben über

TokenizerExtensions Class

  • Reference

Definition

Namespace:
Microsoft.Extensions.AI.Evaluation
Assembly:
Microsoft.Extensions.AI.Evaluation.dll
Package:
Microsoft.Extensions.AI.Evaluation v9.3.0-preview.1.25126.9

Extension methods for Tokenizer.

public ref class TokenizerExtensions abstract sealed
public static class TokenizerExtensions
type TokenizerExtensions = class
Public Module TokenizerExtensions
Inheritance
Object
TokenizerExtensions

Methods

ToTokenCounter(Tokenizer, Int32)

Returns an IEvaluationTokenCounter given the Tokenizer and the inputTokenLimit for a particular AI model / deployment.

Applies to