Jaa

Building Better Input Experience for East Asian Users with the IME API in IE11

  • Artikkeli

During the development of Internet Explorer 11, we proposed and implemented a new set of IME APIs to the Web Applications Working Group for a better input experience for East Asian users. These discussions resulted in changes to the Editor’s Draft of the W3C IME API. This API enables developers to build a tightly integrated experience between the IME that East Asian users rely on to type in their local languages and the UI that developers create to assist input, for example a search suggestion list that dynamically updates based on the keywords that the user has typed.

Avoid UI Occlusion

With IE11, developers can avoid the IME window occluding the part of the UI that is expected to be fully visible while the user is typing.

IME converts Roman alphabets into local characters. There are thousands of characters in East Asian languages. Often the same Roman input maps to several different East Asian words, in which case the user could choose from a list of candidates provided by the IME. The candidates UI is usually a floating window positioned close to the input box, and is not part of the DOM tree, thus JavaScript code doesn’t have a direct access to its size and position.

Candidate window in Chinese

Example of the IME candidate window when typing Simplified Chinese

Many Web sites show a custom UI below the input box as a form of auto-complete. For example, a list of city names that continues narrowing down based on what the user has typed.

Candidate window in English

Example of an input suggestion UI

Since the IME candidate window is also below the input box by default, the two UIs overlap with each other, making it difficult for the user to see or interact with them.

Candidate window in Chinese

Input suggestion UI occluded by the IME candidate window UI

Using the new IME API in IE11, developers can detect the opening of the IME candidate window by listening to MSCandidateWindowShow event, then call getCandidateWindowClientRect() function to find out where the candidate window is and position the suggestion UI away from it:

var context = document.getElementById("mySearchBox").msGetInputContext(); context.addEventListener("MSCandidateWindowShow", candidateWindowShowHandler); function candidateWindowShowHandler(e) {    var imeRect = context.getCandidateWindowClientRect();    var suggestionRect = document.getElementById("mySuggestionList").getBoundingClientRect();    // Check if the two rects intersect, and position them away from each other. }

When the IME candidate window changes position or closes, it fires MSCandidateWindowUpdate or MSCandidateWindowHide events. Developers could listen to them and shift the suggestion UI accordingly.

For a more integrated look, developers can apply the new -ms-ime-align: after CSS property on the textbox to tell the IME to adjust its width and dock below the textbox.

IME API demo on IE Test Drive

A demo where the IME candidate window doesn’t occlude the Web site’s suggestion UI

Earlier Suggestions

Before IE11, developers had to wait until the user finished composing an East Asian character inside the IME to know what the new character would be. In IE11, the list of candidates inside the IME is also exposed to developers while the user is composing, allowing the site to generate suggestions much earlier.

For example, to type the city name of Shanghai in Chinese (“上海”), the user has to type “shang” and hit the space key to get the first Chinese character “上” into the textbox. The suggestion logic of the site couldn’t begin until this sequence was complete. Using the new IME API in IE11, the site gets the list of IME candidates by calling the new getCompositionAlternatives() function as soon as the user has typed the first “s” character; the character “上” is returned among several other candidates. By matching the IME candidate list to the list of city names, the site can find out that the user may be looking for “上海,” and show that in the suggestion UI, so the user could finish the input by only typing “s” and tapping on the suggestion, saving many keystrokes.

var searchBox = document.getElementById("mySearchBox"); var context = searchBox.msGetInputContext(); var compositionAlternates = context.getCompositionAlternatives(); var searchStrings = new Array(); // Add the user’s raw input as one search string. searchStrings.push( searchBox.value.toLowerCase().trim() ); for (var i = 0; i < compositionAlternates.length; i ++) {     // Merge the alternate with the string that is already in the search box.     var searchString = searchBox.value.substr(0, context.compositionStartOffset);     searchString += compositionAlternates[i];     searchString += searchBox.value.substr(context.compositionEndOffset);     searchStrings.push(searchString); } // Find out which string matches the beginning of a city name, and put that city name into the suggestion UI. matchToCityNames(searchStrings);

Comparison with and without use of get composition alternatives.

Using getCompositionAlternatives() in IME API to provide earlier suggestions

Summary

With Internet Explorer 11, developers can enable an IME-aware input experience in their Web-based applications and sites where the IME collaborates tightly with the rest of the page and provides East Asian users a fast and fluid experience, especially in typing suggestion scenarios.

Please note that this API relies on some Windows 8.1 features of the Text Services Framework Interfaces so it is not fully functional with IE11 on Windows 7. It also requires support from the IME. At the time of writing this blog, only the in-box Chinese Simplified/Traditional IMEs in the desktop, and the in-box Chinese Simplified/Chinese Traditional/Japanese/Korean IMEs in the Modern UI environment have a full support of this API.

You can start using the new APIs in your sites to support these scenarios for East Asian users. We look forward to your feedback through Connect.

Jianfeng Lin
Program Manager, Internet Explorer

Comments

  • Anonymous
    March 31, 2014
    The comment has been removed

  • Anonymous
    March 31, 2014
    Thanks! Can you guys also implement V1,V2 and V3 source-map like Chrome / Firefox support? In transcompiler scenarios, this is highly imperative... LESS, CoffeeScript, SASS, Stylus, HAML, etc. Is it mandatory for IE to remain underdog; always late to the party after Chrome and Firefox? I wish if Microsoft spend all their resources for two weeks to Internet Explorer to make it super browser once forever..

  • Anonymous
    March 31, 2014
    Guys for the love of mankind, Web and technology as a whole, please fix the JS engine performance. Here are the dark highlights... bitter truth: IE11 performs extremely poorly in Branch prediction (458% slower than Chrome) connect.microsoft.com/.../performance-ie11-performs-exremely-poorly-in-branch-prediction-458-slower-than-chrome DOM manipulation Tests  (still slowest performance when compared to latest version of any other browsers) connect.microsoft.com/.../ie-performance-dom-manipulation-tests Layer movement for animation connect.microsoft.com/.../ie10-performance-benchmark-layer-movement-for-animation I love IE and I don't understand how come you guys don't feel bad about this JS engine performance situation. Please collaborate with VC team to learn new optimization tactics. Get in your A-game before its too late.

  • Anonymous
    April 01, 2014
    May I know that is it a April fool's joke, or an offical product? (no offense :-) ) Idea is good, yet it should not something relay on content providers. It should be solved on OS level !

  • Anonymous
    April 01, 2014
    Also, this would cause a nightmare if you are dealing with elements in random change scenarios; such as webgl games: for (var i = 0; i < compositionAlternates.length; i ++) unless we cache compositionAlternates.length, for IE6-11 only. The aforementioned issues suggest more scenarios where JS engine works poorly. @JS-Performance team, please don't ignore these issues related to fundamental constructs of language, when the web is evolving quite rapidly.

  • Anonymous
    April 01, 2014
    In IE12 please change IE such that typing a URL in the address bar that starts with "localhost" or "192.168." DOES NOT cause IE to have a his system fit but rather auto prefix with "http://" and load the darn page! This is probably the NUMBER ONE reason why developers HATE!!! Developing in IE! -- Every Developer

  • Anonymous
    April 01, 2014
    In IE12 please change IE such that typing a URL in the address bar that starts with "localhost" or "192.168." DOES NOT cause IE to have a his system fit but rather auto prefix with "http://" and load the darn page! This is probably the NUMBER ONE reason why developers HATE!!! Developing in IE! -- Every Developer

  • Anonymous
    April 01, 2014
    When I type an URL IE11 suggests sites. If i want tot go to a newssite it suggests: newssite.com/article_12345 newssite.com/article_45678 newssite.com/article_78901 IE11 however does not suggest newssite.com which is actually the site I want to go to and which I have already typed in several times. That is ridiculous

  • Anonymous
    April 01, 2014
    While I agree the URL root should have higher precedence, you can always try removing few results (delete them; when you hover over the suggestion from history, it shows cross at the end of the row).