Backwards compatibility of TrueType instructions with Microsoft ClearType

The release of Microsoft Windows XP with support for ClearType coincided with our second generation of the ClearType technology. The first generation shipped with the Microsoft Reader for Windows and the Microsoft PocketPC a year prior to the Windows XP release. The second generation of ClearType brought a much richer filtering algorithm as well as better support for the wide variety of existing TrueType™ fonts.

(A good description of this filtering can be found in Bétrisey C, Blinn JF, Dresevic B, Hill B, Hitchcock G, Keely B, et al. Displaced Filtering for Patterned Displays. Society for Information Display International Symposium Digest of Technical Papers 2000:1-4. Available for download at research.microsoft.com/apps/pubs/?id=68631.)

Although there is a clear architectural distinction between the TrueType rasterizer and the ClearType filtering in the Microsoft Windows implementation, there is significant interaction between these components, as there also is between TrueType and traditional anti-aliasing technology. The bulk of the interaction involves the accuracy of information coming from the TrueType rasterizer, especially as it relates to the asymmetric nature of ClearType.

One of the primary design goals for adding ClearType into Microsoft Windows was to maintain support for existing fonts without having to make modifications to these fonts. Our initial investigations revealed that there were three key issues that impact this compatibility issue. First, in a relatively few number of cases, hinted instructions at smaller sizes would cause what we referred to as “point explosions”—where a point would move off to a random distant location in the pixel grid. The second issue was the difference in advance widths between fonts hinted with ClearType and fonts hinted with the previous rendering technologies of bi-level or font smoothing. Finally, the third issue involved hints, mostly in fonts that were “super-hinted,” that in an attempt to clean up extraneous pixels, ended up causing aesthetic issues when displayed with ClearType.

The second generation of ClearType addresses these backwards compatibility issues. The first issue of “exploding glyphs” is resolved by a technique called asymmetric super-sampling, which will be discussed briefly a little later in this paper. The second issue of incompatible widths is addressed with a technique called compatible widths, which is not related to the issues being discussed in this paper.

The third issue and the goal of this white paper is to describe the techniques used in the interpreter of the Microsoft Windows TrueType rasterizer that improve the display of hinted glyphs when rasterized by Microsoft ClearType.

The target audience for this white paper is a font designer who would be proofing existing fonts being displayed in ClearType or font designers adding or updating instructions or hints to new or existing fonts. It is expected that the reader has some familiarity with the TrueType instruction set and its nomenclature. The paper is not targeted to implementers of font rasterizers.

A quick note on some terminology used in this document. Microsoft prefers to call the TrueType engine a rasterizer while Apple® uses the term scaler. I’ll use the rasterizer term in this document. Apple also generally refers to the concept of applying the TrueType instruction set as “instructing fonts” while Microsoft will often use the term “hinting fonts.” Both terms have the same meaning in this document. Also, since there are differences between the Apple Macintosh® TrueType implementation and the Microsoft TrueType implementation, unless otherwise mentioned, all references will be to the Microsoft Windows TrueType rasterizer.

TrueType Fonts and ClearType

The Microsoft Windows implementation of ClearType takes the output of the TrueType rasterizer in the form of overscaled bitmaps and applies filtering to create an image of alpha values. The TrueType rasterizer takes as input the TrueType font, which consists of contour outlines and a set of instructions as well as relevant information describing the size to be displayed, the resolution of the display, and additional miscellaneous input like character codes and scaling transforms.

The instructions provided by the TrueType interpreter are designed to allow designers to specify how character features should be rendered. Because of the richness of the instruction set, there can be significant interaction between the original outline provided in the font and the outputted bitmap from the TrueType rasterizer. This interaction cascades and impacts rendering methods like ClearType.

Bi-Level Rendering

The original design of TrueType corresponded to an era where the most common form of rendering was bi-level, or one pixel color for the background and one pixel color for the foreground. This is often called black and white rendering. There are several common techniques for applying instructions to improve the glyph shapes, especially at lower sizes, where the size of the glyph features is close to the size of a pixel. Most of these techniques, though, usually apply some level of “delta” hints to improve glyph shapes.

Delta Hints

Delta hints are a class of hinting techniques that get their name from the TrueType DELTAC or DELTAP instructions. Delta instructions move a point by a limited amount along the freedom vector in the case of DELTAP or adjust the value of a CVT by a limited amount in the case of DELTAC. The key thing about delta instructions is that they only work at a set of given sizes. Because the distance of a delta is limited and it is difficult to have a delta execute over a simple range of sizes, it is common to have TrueType functions that can move points under additional conditions.

Delta hints are typically implemented while proofing the quality of hints in a font. If at a given size or range of sizes, a particular pixel was either on or off when that was deemed incorrect, a point or points would be moved by an arbitrary amount in order to make the correct pixel pattern occur. Because there is a lot of slack in the relatively large pixels, there was not much accuracy in the values used for deltas.

ClearType Interaction with TrueType Instructions

The TrueType interpreter works very well with ClearType. Microsoft worked with Apple Computer on the original design to include support for non-square pixels, which were still common on the Microsoft Windows platform when TrueType was first released. The implementation of ClearType is also asymmetric, leading to non-square virtual pixels. Because the sub-pixel arrangement on an LCD panel is composed of vertically oriented colored elements, the resolution is at least three times higher along the x-axis while along the y-axis the resolution is the same as with no ClearType solution. In fact, the actual horizontal resolution for TrueType is sixteen times greater than the y-axis.

This asymmetry, though, leads to problems when implemented with the traditional overscaling techniques used for anti-aliasing. A typical problem involves the freedom and project vectors used by the hinting instructions.

In the example above, the diagonal stem of the lowercase ‘k’ is set up in the left image with the freedom vector and projection vector in the typical manner for controlling the weight of diagonal stems. In the right image, when the ‘k’ is asymmetrically overscaled by a large factor, such as sixteen along the x-axis, the freedom vector and projection vector become almost perpendicular. This leads to the “point explosions’ previously described. To address this issue, we moved from traditional overscaling to ClearType asymmetric super-sampling.

ClearType Asymmetric Super-sampling

For ClearType, you can imagine a grid with sixteen closely spaced grid lines per pixel in the x-direction, and one gridline per pixel in the y-direction. Rounding instructions (RTDG[], RDTG[], RUTG[], RTG[], RTHG[], and ROFF[]) now apply to this virtual grid as do MDAP[R], MIAP[R], MDRP[…R…], and ROUND[…]. Instructions that don’t round, such as SHP[], IP[], IUP[X], ALIGNRP[], ISECT[] work the same as without ClearType. Similarly, with ClearType in the x-direction, SCVTCI[] reduces CVT cut-in to one sixteenth of its actual value, matching the extra virtual resolution gained by rounding to one sixteenth of grid. SMD[] reduces minimum distance to ½ of its actual value, roughly matching the extra visual resolution gained by ClearType. (Theory says the visual gain is between 1.7 to 3 times, hence we used 2 for efficiency.)

Other benefits from this approach are the MPPEM[] and MD[] instructions measure meaningful PPEms (Pixels per Em, a device dependant way for measuring type size) and distances and the aspect ratio remains essentially square, eliminating problems where the projection vector and freedom vector become near perpendicular.

This behavior of some TrueType instructions, though, is asymmetric so when the vectors are set to the y-direction, the behavior is exactly how it was with bi-level—rounding is on the actual pixel grid and not the virtual pixel grid.

In all of these examples we refer to extra resolution along the x-direction (along the x-axis) but from the Microsoft Windows TrueType rasterizer point of view we call this the “ClearType Direction.” The “ClearType Direction” can be either the x-direction or the y-direction. If it were the case that the extra subpixel resolution was in the y-direction, the virtual grid would change directions and the extra precision would only take place along the y-direction.

Backwards Compatible ClearType

In general this technique of asymmetric super-sampling works very well, but some legacy fonts hinted for bi-level rendering can create some issues. Specifically, fonts that were “super hinted” for bi-level with many delta-like instructions and highly constrained hints are most likely to be impacted with ClearType.

One example of a problem area involves legacy delta instructions. With the extra resolution along the x-axis, delta instructions that were executed when the freedom vector is set to the ClearType direction have the potential for much higher accuracy, but previous usage with bi-level rendering was relatively sloppy leading to extreme exaggeration of delta like instructions.

To address this situation, we needed to create a backward compatible mode for ClearType when dealing with instructions using the ClearType direction for the freedom vector. Here is how we handle various instructions in the backward compatible mode for ClearType. It should be noted that these are the compatibility techniques addressed at the time of this document, additional compatibility techniques could be added in the future.

DELTAP and SHPIX

Many fonts use DELTAPs or SHPIX to fix unfortunate pixel patterns. Although problems with delta-like instructions have shown up in font smoothing anti-aliasing, they are even more problematic for ClearType. Therefore, all DELTAPs are skipped, except DELTAPs on previously touched points in the non-ClearType direction and DELTACs. Deltas are also skipped in composite glyphs if they are in the ClearType direction for the proper centering and vertical positioning of diacritical marks. Inline deltas are sometimes used to adjust the position of horizontal strokes, and CVT deltas do not always have the vectors set properly, hence they are kept to avoid un-correcting weights in the y-direction (weights in x are taken care of by tightened CVT cut-ins.)

(An inline delta is a delta that occurs before the IUP instruction on a previously touched point. A post-IUP delta occurs after the IUP instruction, while a pre-IUP delta occurs prior to the IUP instruction.)

In the TrueType rasterizer, we assume that if we are doing a SHPIX instruction, we are in the context of a function that implements delta instructions for a range of PPEM sizes. If such a delta occurs on an untouched point, regardless of whether it is in the ClearType direction or not, it creates a dent in the outline. While for bi-level this is intended to flip one or more pixels, it distorts the stroke in ClearType. If such a delta occurs on a touched point, it moves along the entire outline, such as to place strokes differently. If this happens in the ClearType direction, this distorts the natural spacing of the stroke(s). Therefore we keep only deltas on touched points in the non-ClearType direction.

As mentioned earlier, for composites, the touched/untouched rule does not apply the same way. A point that is flagged as untouched may have been previously touched while executing the code for the respective component. Yet a subsequent SHPIX or possibly DELTAP applied to that point will not create another dent in the outline, but move the entire outline instead. This is used in bi-level to reposition diacritics to ensure a minimal distance of one pixel between the base character and the diacritic. Hence we also keep deltas in composites.

Some deltas are implemented as functions in various fonts and they do not always have the same function number across fonts. To detect this situation, we look for signature code at the beginning of the function. There are two common signatures we are looking for to indicate inline deltas.

MPPEM[]  
GTEQ[]
SWAP[]
MPPEM[]
LTEQ[]
AND[]
IF[]
SHPIX[]
ELSE[]
POP[]
POP[]
EIF[]
ENDF[]

MPPEM[]
EQ[]
IF[]
SHPIX[]
ELSE[]
POP[]
POP[]
EIF[]
ENDF[]

TypeMan Talk DiagEndCtrl

Some fonts use the old TypeMan Talk DiagEndCtrl (Diagonal End Control) command, which in turn calls a support function that calculates an inline delta in x-direction to undo the side-effects of “cross-over” strokes. Therefore, as an exception to the exception, these inline deltas in the ClearType direction are not skipped.

(TypeMan Talk is a high level TrueType hinting language designed for the tool TypeMan. This tool was originally written by Sampo Kaasila of Type Solutions, Inc., now a wholly-owned subsidiary of Bitstream® Inc. Microsoft has updated the TypeMan Talk language in the Visual TrueType Product.)

RDTG

In some fonts, points were aligned using MDRP with RDTG preceded by SPVTL (not SDPVTL.) For this to align properly, RDTG rounds down to the physical grid in these cases.

Historically, if a point to be aligned was away by 1.5° or more from the line to which to align, the TypeMan Talk Align has translated to SPVTL, followed by RDTG and MDRP. This has a major problem: The MDRP instruction uses the set projection vector to measure the original distance on the uninstructed outline. This gives us an “original” distance which is more or less wrong, depending on how far the reference point has moved already. (Notice that it is most likely that the reference point has moved somewhat in x and in y before the alignee gets to be aligned.) Subsequently, when MDRP gets to move the point, it uses the same projection vector again, but this time around on the instructed outline, in order to determine how far the point has to be moved in the end. Luckily, this is used in connection with RDTG, which is applied to the “original” distance, and thereby often reduces the “original” distance to zero, which often is closer to the correct original distance than the one obtained by using the project vector on the uninstructed outline.

In the ClearType direction, however, these erroneous “original” distances would be rounded down to the nearest sixteenth of a pixel, and with that become highly visible (a distance of just under one pixel, which is what gets rounded down to zero in bi-level, is too large a distance to be ignored in ClearType. Therefore, we have to make an exception here.

ROUND in PreProgram

In some East Asian fonts, numbers were calculated and ROUNDed in the pre-program, without properly setting the vectors, which vertically misplaced horizontal strokes (off-grid, resulting in drop-outs.) Therefore all rounding is done to the physical grid while in the pre-program.

Unrounded MIRP and CVT Cut-In

In some fonts fractional stroke weights were calculated and used in MIRP[…r…]. While this provides some extra granularity in font smoothing, it unnecessarily quantizes stroke weights in ClearType, because un-rounded MIRPs now respect CVT cut-in.

Our goal is to avoid fractional font-smoothing targeted CVTs and unrounded MIRP (little r) which bypasses CVT cut-in. Here we may honor a CVT cut-in even though the round-off flag would require not doing so. We assume that the context is a stroke weight that has been tweaked for use with font smoothing, and the only way to force MIRP to use a particular distance is to round the CVT “by hand” and then turn off the rounding flag, which is quite what we don't want to do in ClearType, i.e. to enforce an unnatural weight.

To summarize, with ClearType on, do CVT cut-in always.

MSIRP and CVT Cut-In

Some fonts pre-calculate stroke weights and subsequently use MSIRP[.], which involves neither rounding nor CVT cut-ins. Therefore MSIRP[.] now respects the CVT cut-in.

Similar to the unrounded MIRP problem previously discussed, for this issue we honor a CVT cut-in in case there is a non-trivial outline distance between the parent (reference) and the child (target) point involved, in which case we assume the context is a stroke weight, else we assume the context is an accent placement function, in which case we use the actual distance as before.

TypeMan Talk DStroke and IStroke

Some legacy fonts use the old TypeMan Talk DStroke and IStroke commands, which apparently were not meant to work with non-square aspect ratios. Therefore, while in the respective support functions, the rasterizer pretends to be in non-square aspect ratio, even though in reality it is not.

We need to avoid collapsing strokes in (partly) auto-hinted fonts using the TypeMan Talk DStroke and IStroke commands. Here we are assuming that FDEF functions 64 through 66 are support functions for the TypeMan Talk DStroke and IStroke commands, which may make strokes collapse in ClearType. To determine whether we have actually run into the context of these functions, we look at the first couple of bytes only, as the exact implementation of these functions may have changed over time, but hopefully, the preamble (to look at storage location 22) has not. The signature for the function is:

#PUSH 22
RS[]
IF[]

The easiest way to by-pass these functions is to flag storage location 22 with zero, indicating not to use DStroke or IStroke, otherwise we would have to clean up the stack. The reason that DStroke does not work, is the following. DStroke brings pairs of points to the same y (or x), MIRP[…r…] the floating point, and then moves it back to the old y (or x.) Now that we re-interpret no round on the MIRP, CVT cut-in actually gets to cut in, the original distance gets to apply, just that the original distance is taken prior to bringing points to the same y (or x), meaning it is likely to be way off. As a result, the diagonal stroke weight (in x) starts to depend heavily on the distance (in y) of the control points involved, which is not what we want. Not that for non-square aspect ratios, the DStroke is a NOP, therefore was a NOP with sixteen times the resolution of ClearType, and hence we are not doing any major harm here by skipping it in ClearType.

Spacing Functions

Similar to the DStroke and IStroke, there are some spacing functions which also are not meant to work with non-square aspect ratios. Therefore the rasterizer pretends to be in non-square aspect ratio in these cases as well.

To avoid bloated characters, we are avoiding some specific functions for spacing. We are assuming that functions 0, 1, 2, 4, 7, and 8 are support functions for this spacing code, which may squish characters or pull them left or right or in both directions at the same time. This has to do amongst other things with using the twilight zone. The spacing algorithm is meant to be turned off above a certain size, such as 30 PPEm, hence we may argue that with the extra visual resolution we should turn it off at a considerably smaller PPEm size, or altogether. Storage location 24 is used to bypass the function. The signature for the function is one of the following:

SVTCA[X]
#PUSH 24
RS[]
IF[]

SVTCA[X]
RTG[]
#PUSH 24
RS[]
IF[]

Diagonal Stroke Function

Some fonts use a diagonal stroke method which made some italic fonts upright or back-slanted under ClearType. Therefore, the interpreter identifies the respective support function and pretends an alternative CVT cut-in value.

We are assuming that function 58 handles the diagonal stroke. If the signature matches the following code we will change the CVT cut-in value.

DUP[]
DUP[]
#PUSH[], 1
ADD[]
GC[N]
#PUSH[], 64
SWAP[]
WS[]

TypeMan Talk VacuFormRound

Some fonts used the old TypeMan Talk VacuFormRound command, which may work on certain bi-level pixel patterns, but which invariably creates diamonds in ClearType. Therefore, the interpreter attempts to skip all related code.

We are expecting function 0 to contain the signature:

RCVT[]
SWAP[]
GC[N]
ADD[]
DUP[]
#PUSH, 38

With this signature, we confirm that that IUP[X] and IUP[Y] have already been executed. Storage location 8 will return zero to stop most VacuFormRounds. In the case of a Type 2 VacuFormRound, the MD instruction, while in the VacuFormRound, will slightly increment the projection vector by the smallest possible amount (1) to throw off Type 2 VacuForms which are keyed off a distance being exactly 1 pixel.

Using TrueType Instructions with ClearType

As previously mentioned, the precision of ClearType rasterization can integrate quite naturally with the TrueType interpreter. When fonts are hinted with ClearType in mind, they can be proofed to verify that the hints work appropriately with ClearType and other rasterization techniques. There is additional support in the TrueType instruction set to help fonts optimize for ClearType and to help prevent other types of problems as we have discussed with backwards compatibility.

GETINFO

The GETINFO[] instruction, among other things, can be used to query what type of rasterization is being used for the current glyph. The instruction is used by setting one or more selector bits and the return value on the stack is a bit or group of bits based on which selector bits were set. These selector bits and return value bits are fully documented in the TrueType instruction specification, but it is worthwhile describing the ClearType related values.

ClearType (Selector bit 6, Return bit 13)

This flag is requested by setting selector bit six. If the return bit thirteen is set it indicates that the glyphs for this size will be rendered with ClearType rendering. If the return bit thirteen is cleared, one can only assume that it is not ClearType being rendered; it doesn’t imply which type of rendering is being performed.

Compatible Width ClearType (Selector bit 7, Return bit 14)

This flag is requested by setting selector bit seven. If the return bit fourteen is set, this indicates that the glyphs for this font size will be adjusted post hinting in order to return advance widths that are exactly the same as bi-level rendering. If the return bit fourteen is not set, then the advanced widths will be either returned as integers rounded to the nearest pixel or rounded to the nearest sixteenth of a pixel. The former is referred to as natural widths ClearType and the later as sub-pixel positioned ClearType.

Symmetrically Smoothed ClearType (Selector bit 8, Return bit 15)

At larger sizes ClearType performs better with a symmetrical rendering solution. Standard ClearType which is used exclusively by some graphics systems and at lower sizes in others uses a 6×1 filtering technique—with six times the resolution along the x-axis and no resolution change along the y-axis. With symmetrically smoothed ClearType, a 6×5 filtering technique is used—with six times the resolution along the x-axis and five times the resolution along the y-axis. This later solution tends to create more blur at lower sizes but prevents aliasing at higher sizes.

This flag is requested by setting selector bit eight. If the return bit fifteen is set, this indicates that symmetrically smoothed ClearType is enabled for this font size. If the return bit fifteen is not set, then the symmetrically smoothed ClearType is not enabled and the 6×1 filtering technique is being used.

BGR ClearType (Selector bit 9, Return bit 16)

The primary environment for ClearType is with displays that are similar to Liquid Crystal Displays (LCD) that implement color with vertical stripes. In most cases these displays have the red color filter, followed by green and then blue from left to right. In some cases it is possible for this ordering to be reversed. This can be due to the creation of the display filter or by having the display text inverted on the screen.

The BGR status can be requested by setting selector bit nine. If the return bit sixteen is set, this indicates that ClearType is processing the LCD stripes in BGR (Blue, Green, Red) order. If it is clear then ClearType is processing the LCD stripes in RGB (Red, Blue, Green) order.

GETINFO Usage with ClearType

The ClearType information returned by the GETINFO instruction is useful for both existing fonts that wish to be made compatible with ClearType and new fonts that need to use different hinting techniques for ClearType and other methods of rendering.

Before using the GETINFO instruction with ClearType, one should query the rasterizer version, which is also done with the GETINFO instruction by setting selector bit one and examining the result in bits 0 through 7 (first byte) of the return result. The ClearType selector can be used if the version number is 35 or greater. The rest of the ClearType selectors discussed above can be used if the version number is 37 or greater.

If information from GETINFO will be used often in the instructions, it is quite common to save the status of the flag in a storage location.

For instructions where compatibility is needed for existing bi-level hints, the code that has impact on ClearType (e.g. delta like instructions or functions) can be surrounded by a conditional IF that is only executed if ClearType is not enabled. Likewise, additional instructions that may improve the rendering of ClearType may also be surrounded by a conditional IF that is executed only if ClearType is enabled. These techniques can be used both in the CVT program (pre-program) and glyph programs and if some of the hinting techniques are implemented in functions, the conditional code can be implemented in that function.

INSTCTRL

The INSTCTRL instruction with selector flag three and a value of TRUE (4) should be used in new fonts and existing instructed fonts that have been verified to work correctly in the ClearType environment. Typically this instruction would be placed at the beginning of the CVT program, with a selector of three and a value of four on the stack. Although the other selector flags associated with INSTCTRL are limited to the CVT Program, selector three can be used in either the CVT Program or the glyph program.

(This is contrary to the current OpenType specification but intended in the design of this selector. To make sure that one glyph does not change the settings for another glyph, if the instruction selector is set at the beginning of a glyph, it should be cleared at the end of the glyph. This is not required if the instruction selector is used exclusively in the CVT program.)

If INSTRCTRL with selector flag two and a value of TRUE (2) is used in the CVT Program, it will disable the effects of selector flag three and each glyph will have to control this independently.

As most of this document describes, significant effort has been made to allow ClearType to work with existing fonts. In a few cases existing instruction behavior, as documented in the TrueType instruction specification, had to be modified to maintain backwards compatibility. When the INSTCTRL ClearType instruction is used it returns instructions to the behavior as described in the TrueType instruction specification.

Conclusion Significant effort has been taken to make ClearType perform optimally for Microsoft Windows. Continued support from font hinters is necessary in order that fonts will continue to be the most readable and legible on many types of displays and many types of rendering environments. This paper has described the techniques used to make existing, unmodified fonts work correctly with ClearType, but more importantly it provides the information to developers of new fonts on how to assure the consistent behavior of the TrueType interpreter for fonts by using the INSTCTRL instruction.

Microsoft recommends that new fonts which have code to instruct glyphs include the following sequence at the beginning of the CVT Program:

#PUSH 4,3
INSTCTRL[]

With this code, as always, quality assurance should be verified with ClearType, traditional font smoothing, and bi-level rendering.