TokenizerExtensions.ToTokenCounter(Tokenizer, Int32) Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Returns an IEvaluationTokenCounter given the Tokenizer and the
inputTokenLimit for a particular AI model / deployment.
public:
[System::Runtime::CompilerServices::Extension]
static Microsoft::Extensions::AI::Evaluation::IEvaluationTokenCounter ^ ToTokenCounter(Microsoft::ML::Tokenizers::Tokenizer ^ tokenizer, int inputTokenLimit);public static Microsoft.Extensions.AI.Evaluation.IEvaluationTokenCounter ToTokenCounter (this Microsoft.ML.Tokenizers.Tokenizer tokenizer, int inputTokenLimit);static member ToTokenCounter : Microsoft.ML.Tokenizers.Tokenizer * int -> Microsoft.Extensions.AI.Evaluation.IEvaluationTokenCounter<Extension()>
Public Function ToTokenCounter (tokenizer As Tokenizer, inputTokenLimit As Integer) As IEvaluationTokenCounterParameters
- inputTokenLimit
- Int32
The threshold of maximum allowed input tokens for a particular AI model / deployment.
Returns
An IEvaluationTokenCounter for a particular AI model / deployment.
Applies to
Spolupracujte s námi na GitHubu
Zdroj tohoto obsahu najdete na GitHubu, kde můžete také vytvářet a kontrolovat problémy a žádosti o přijetí změn. Další informace najdete v našem průvodci pro přispěvatele.
