TokenizerExtensions.ToTokenCounter(Tokenizer, Int32) Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Returns an IEvaluationTokenCounter given the Tokenizer and the
inputTokenLimit for a particular AI model / deployment.
public:
[System::Runtime::CompilerServices::Extension]
static Microsoft::Extensions::AI::Evaluation::IEvaluationTokenCounter ^ ToTokenCounter(Microsoft::ML::Tokenizers::Tokenizer ^ tokenizer, int inputTokenLimit);public static Microsoft.Extensions.AI.Evaluation.IEvaluationTokenCounter ToTokenCounter (this Microsoft.ML.Tokenizers.Tokenizer tokenizer, int inputTokenLimit);static member ToTokenCounter : Microsoft.ML.Tokenizers.Tokenizer * int -> Microsoft.Extensions.AI.Evaluation.IEvaluationTokenCounter<Extension()>
Public Function ToTokenCounter (tokenizer As Tokenizer, inputTokenLimit As Integer) As IEvaluationTokenCounterParameters
- inputTokenLimit
- Int32
The threshold of maximum allowed input tokens for a particular AI model / deployment.
Returns
An IEvaluationTokenCounter for a particular AI model / deployment.
Applies to
Samarbejd med os på GitHub
Kilden til dette indhold kan findes på GitHub, hvor du også kan oprette og gennemse problemer og pullanmodninger. Du kan få flere oplysninger i vores vejledning til bidragydere.
