MicrosoftTokenizerLanguage Enum

Lists the languages supported by the Microsoft language tokenizer.

Fields

BANGLA

BULGARIAN

CATALAN

CHINESE_SIMPLIFIED

CHINESE_TRADITIONAL

CROATIAN

CZECH

DANISH

DUTCH

ENGLISH

FRENCH

GERMAN

GREEK

GUJARATI

HINDI

ICELANDIC

INDONESIAN

ITALIAN

JAPANESE

KANNADA

KOREAN

MALAY

MALAYALAM

MARATHI

NORWEGIAN_BOKMAAL

POLISH

PORTUGUESE

PORTUGUESE_BRAZILIAN

PUNJABI

ROMANIAN

RUSSIAN

SERBIAN_CYRILLIC

SERBIAN_LATIN

SLOVENIAN

SPANISH

SWEDISH

TAMIL

TELUGU

THAI

UKRAINIAN

URDU

VIETNAMESE