Math Zone Navigation

The post Using MathML-Based Speech to Edit Math in Different Math Models discusses the need to navigate math zones in the edit space rather than in a MathML copy of the edit space. This need arises for editing since the navigation location must be synchronized with the edit selection. It’s also important for defining the selection bounding rectangles even if editing isn’t allowed. The present post compares the math-zone edit navigation in Microsoft Office apps to the structured child/parent/next/previous tree navigation provided by MathPlayer and other systems. Combining both approaches results in rich navigation and editing experiences for blind and sighted users alike.

Edit navigation

The OfficeMath math zone represents math as a “flattened” tree, in which nested math objects such as fractions and matrices have start and end delimiters. This allows character navigation to traverse every character in a math zone. Typing the → key at the start of a fraction moves into the numerator. Typing → at the end of the numerator moves into the denominator. Typing → at the end of the denominator moves out of the fraction. The → and ← keys let you “escape” from a math object (go to the next higher level in the tree) and to enter a math object (go down a level into the tree) as well as moving past characters.

Typing Ctrl+→ moves by siblings. So, typing Ctrl+→ at the start of a fraction moves past the fraction to whatever follows the fraction. A Ctrl+→ at the end of the denominator moves out of the fraction, so like the → key, Ctrl+→ can escape from down inside an object (go to the next higher level in the tree). But it cannot go down into a lower level of the tree. Both Ctrl+→ and → leave the selection as an insertion point (IP), ready for inserting more characters and math objects. Shift+Ctrl+→ and Shift+→ select the next object or character. The Ctrl+← and ← keys work the same ways but move toward the start of the math zone. (Note that Word hasn’t yet implemented this Ctrl+→ and Ctrl+←behavior, but OneNote and PowerPoint have).

All eight editing hot keys ([Shift+][Ctrl+]←/→) move out of the math zone when they come to the end of the math zone. The keys are usually geometric but are logical as well in that → moves from the end of a numerator to the start of the denominator. Such movement isn’t geometric since the motion is down and to the left rather than to the right. At any time, the user can enter and delete characters. If the selection is nondegenerate, entering a character replaces the selection and results in an insertion point (IP). If the selection is already degenerate, e.g., an IP following Shift-less key motion, the character is entered at the IP.

With OfficeMath speech, you can use this kind of navigation while entering and editing equations with a keyboard. You don’t need to see the screen. As far as I know, the Office apps Word, Outlook, PowerPoint, and OneNote are the only major apps that let blind users enter and edit equations using only speech and a keyboard.

Structured navigation

Contrast these edit movements with structured Parent/FirstChild/LastChild/Next/Previous tree navigation. This structured navigation always selects a character or object unless it ends up inside an empty argument. At the start of a fraction, FirstChild selects the numerator and LastChild selects the denominator. If the numerator is selected, Next selects the denominator. But unlike Ctrl+→, another Next does nothing since a fraction only has two arguments. To get out of the fraction, you enter Parent and then Next to go to the next sibling of the fraction.

If an argument is only partially selected, then Next goes to the next sibling in the argument. If the last sibling in an argument is selected, Next does nothing. Entering Parent selects the whole argument and another Parent selects the math object. The highest level in the tree is the whole math zone and structured navigation commands won’t move outside the math zone.

If a selection consists of multiple characters, e.g., the “sin” in the math function object “sin 𝑥”, FirstChild selects the ‘s’ and LastChild selects the ‘n’. Next/Previous select the next/previous character, respectively, unless the last/first character is selected (in which case Next/Previous do nothing). In this way, you can examine every character in the math zone using structured navigation.

Possible hot keys for Parent, FirstChild, LastChild, Next, and Previous are Alt+↑, Alt+↓(or Alt+Home), Alt+End, Alt+→, and Alt+←, respectively. Since these differ from [Shift+][Ctrl+]←/→, both kinds of navigation can be used together. It’s desirable to have corresponding navigation buttons on refreshable braille displays.

While structure navigation takes more keystrokes than edit navigation, it’s valuable for understanding complicated equations, particularly if you cannot see the equations easily and are navigating with the help of math speech.

Conclusions

Ideally for editors like Word, Outlook, PowerPoint, and OneNote, both kinds of navigation would be available. If using structured navigation, you want to enter a character at the end of the current selection, type → to collapse the selection to an IP at the end and type the character. If you want to enter a character at the start of the current selection, type ← to collapse the selection to an IP at the start and type the character. With these techniques, visually impaired users can have a rich navigation experience and all users can enter and edit equations easily.