TokenizerResult(String, String, IReadOnlyList<Split>, Boolean) Constructor
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Create a new object of the TokenizerResult object.
public TokenizerResult (string originalString, string normalizedString, System.Collections.Generic.IReadOnlyList<Microsoft.ML.Tokenizers.Split> splits, bool offsetsMappedToOriginalString);new Microsoft.ML.Tokenizers.TokenizerResult : string * string * System.Collections.Generic.IReadOnlyList<Microsoft.ML.Tokenizers.Split> * bool -> Microsoft.ML.Tokenizers.TokenizerResultPublic Sub New (originalString As String, normalizedString As String, splits As IReadOnlyList(Of Split), offsetsMappedToOriginalString As Boolean)Parameters
- originalString
- String
The list of tokens to merge.
- normalizedString
- String
The list of tokens to merge.
- splits
- IReadOnlyList<Split>
The list of tokens to merge.
- offsetsMappedToOriginalString
- Boolean
Indicate whether the offsets is mapped to the original string or the normalized string.