SelectableWordsSegmenter.Tokenize Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Calls the provided handler with two iterators that iterate through the selectable words prior to and following a given index into the provided text.
public:
virtual void Tokenize(Platform::String ^ text, unsigned int startIndex, SelectableWordSegmentsTokenizingHandler ^ handler) = Tokenize;void Tokenize(winrt::hstring const& text, uint32_t const& startIndex, SelectableWordSegmentsTokenizingHandler const& handler);public void Tokenize(string text, uint startIndex, SelectableWordSegmentsTokenizingHandler handler);function tokenize(text, startIndex, handler)Public Sub Tokenize (text As String, startIndex As UInteger, handler As SelectableWordSegmentsTokenizingHandler)Parameters
- text
-
String
Platform::String
winrt::hstring
Provided text containing words to be selected.
- startIndex
-
UInt32
unsigned int
uint32_t
A zero-based index into text. It must be less than the length of text.
The function that receives the iterators.
Remarks
The iterators in SelectableWordSegmentsTokenizingHandler are lazy and evaluate small chunks of text at a time.
The handler is called at most once per call to Tokenize. The handler is not called if there are no selectable words in text.