TokenType Enum

Definition

Namespace:

Microsoft.VisualStudio.Package

Assembly:

Microsoft.VisualStudio.Package.LanguageService.15.0.dll

Package:

Microsoft.VisualStudio.Package.LanguageService.15.0 v17.12.40392

Specifies the different types of tokens that can be identified and returned from a language service scanner.

public enum class TokenType

public enum class TokenType

enum TokenType

public enum TokenType

type TokenType = 

Public Enum TokenType

Inheritance

Enum

TokenType

Fields

Name	Value	Description
Unknown	0	The token is an unknown type. This is typically used for any token not recognized by the parser and should be considered an error in the code being parsed.
Text	1	General text; any text not identified as a specified token type.
Keyword	2	A language keyword, an identifier that is reserved by the language. For example, in C#, do, while, foreach, if, and else, are all keywords.
Identifier	3	An identifier or name. For example, the name of a variable, method, or class. In XML, this could be the name of a tag or attribute.
String	4	A string. Typically defined as zero or more characters bounded by double quotes.
Literal	5	A literal value (a character or number). For example, in C# or C++, this is a character bounded by single quotes, or a decimal or hexadecimal number.
Operator	6	A punctuation character that has a specific meaning in a language. For example, in C#, arithmetic operators +, -, *, and /. In C++, pointer dereference operator ->, insertion operator >>, and extraction operation <<. In XML, assignment operator =.
Delimiter	7	A token that operates as a separator between two language elements. For example, in C#, the period "." between class name and member name. In XML, the angle brackets surrounding a tag, < and >.
WhiteSpace	8	A space, tab, or newline. Typically, a contiguous run of any whitespace is considered a single whitespace token. For example, the three spaces in "name this" would be treated as one whitespace token.
LineComment	9	A line comment (comment is terminated at the end of the line). For example, in C# or C++, a comment is preceded by a //. In Visual Basic, this is a single tick '.
Comment	10	A block comment. For example, in C# or C++, a comment is bounded by /* and */. In XML, the comment is bounded by <!-- and -->.

Remarks

This enumeration is used in the TokenType structure to identify the type of token parsed. The TokenType structure is used in the IScanner scanner as implemented in a language service.

The types specified in this enumeration cover tokens that can appear in all common computer languages. More importantly, these are the token types that the default managed package framework language service classes understand. You can add additional types that your scanner support but you should first adhere to the meanings of the types described here.