TokenizerFunction Delegate
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Represents a callback method that can break a string into its component tokens.
public delegate System.Collections.Generic.List<Microsoft.Bot.Builder.Dialogs.Choices.Token> TokenizerFunction(string text, string locale = default);type TokenizerFunction = delegate of string * string -> List<Token>Public Delegate Function TokenizerFunction(text As String, Optional locale As String = Nothing) As List(Of Token) Parameters
- text
- String
The input text.
- locale
- String
Optional, identifies the locale of the input text.
Return Value
The list of the found Token objects.