WordsSegmenter.Tokenize(String, UInt32, WordSegmentsTokenizingHandler) Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Calls the provided handler with two iterators that iterate through the words prior to and following a given index into the provided text.
public:
virtual void Tokenize(Platform::String ^ text, unsigned int startIndex, WordSegmentsTokenizingHandler ^ handler) = Tokenize;void Tokenize(winrt::hstring const& text, uint32_t const& startIndex, WordSegmentsTokenizingHandler const& handler);public void Tokenize(string text, uint startIndex, WordSegmentsTokenizingHandler handler);function tokenize(text, startIndex, handler)Public Sub Tokenize (text As String, startIndex As UInteger, handler As WordSegmentsTokenizingHandler)Parameters
- text
-
String
Platform::String
winrt::hstring
Provided text containing words to be returned.
- startIndex
-
UInt32
unsigned int
uint32_t
A zero-based index into text. It must be less than the length of text.
- handler
- WordSegmentsTokenizingHandler
The function that receives the iterators.
Remarks
The iterators in WordSegmentsTokenizingHandler are lazy and evaluate small chunks of text at a time.
The handler is called at most once per call to Tokenize. The handler is not called if there are no selectable words in text.