SpanExtensions.Tokenize<T>(Span<T>, T) Method

Definition

Namespace:

Microsoft.Toolkit.HighPerformance

Assembly:

Microsoft.Toolkit.HighPerformance.dll

Package:

Microsoft.Toolkit.HighPerformance v7.1.2

Tokenizes the values in the input Span<T> instance using a specified separator. This extension should be used directly within a foreach loop:

Span<char> text = "Hello, world!".ToCharArray();

foreach (var token in text.Tokenize(','))
{
    // Access the tokens here...
}

The compiler will take care of properly setting up the foreach loop with the type returned from this method.

public static Microsoft.Toolkit.HighPerformance.Enumerables.SpanTokenizer<T> Tokenize<T> (this Span<T> span, T separator) where T : IEquatable<T>;

static member Tokenize : Span<'T (requires 'T :> IEquatable<'T>)> * 'T -> Microsoft.Toolkit.HighPerformance.Enumerables.SpanTokenizer<'T (requires 'T :> IEquatable<'T>)> (requires 'T :> IEquatable<'T>)

<Extension()>
Public Function Tokenize(Of T As IEquatable(Of T)) (span As Span(Of T), separator As T) As SpanTokenizer(Of T)

Type Parameters

T

The type of items in the Span<T> to tokenize.

Parameters

span

Span<T>

The source Span<T> to tokenize.

separator

T

The separator T item to use.

Returns

SpanTokenizer<T>

A wrapper type that will handle the tokenization for span.

Remarks

The returned SpanTokenizer<T> value shouldn't be used directly: use this extension in a foreach loop.