RichEdit Property Sets
RichEdit has many character-format properties, most of which are documented for ITextFont2 and CHARFORMAT2. Nevertheless, the OpenType specification defines many more character-format properties called OpenType features consisting of a 32-bit identifier (id) and a 32-bit value. For example, the Gabriola font has stylistic set 6, which displays “Gabriola is graceful” as
Variable fonts are the latest addition to the OpenType specification and the variable-font axis coordinates are also specified by an id-value pair. For example, the experimental HoloFont font has three axes, ‘wght’, ‘wdth’, and ‘opsz’, the first two of which are illustrated in
HoloFont was designed by John Hudson and Ross Mills of Tiro Typeworks Ltd.
You can try out variable fonts by checking out this site and you can find myriad variable-font articles and talks here. Variable fonts present a user-interface (UI) challenge. One technique is to use a slide bar to choose an axis coordinate. AI might provide good default values. If the traditional font drop downs are used, you can be confronted with a zillion choices. HoloFont has 9 weights × 5 widths × 6 optical sizes = 270 entries which all appear in the current Word drop-down font list! And that’s tiny compared to the continua of possible axis coordinate values. To illustrate this quandary, here are the first few entries in the HoloFont font drop-down list
| Narrow ThinNarrow ExtraLightmmmNarrow LightNarrow SemiLightNarrowNarrow SemiBoldNarrow BoldNarrow ExtraBoldNarrow Black | SemiNarrow ThinSemiNarrow ExtraLightmmmSemiNarrow LightSemiNarrow SemiLightSemiNarrowSemiNarrow SemiBoldSemiNarrow BoldSemiNarrow ExtraBoldSemiNarrow Black | ThinExtraLightmmmLightSemiLightRegularSemiBoldBoldExtraBoldBlack | SemiWide ThinSemiWide ExtraLightSemiWide LightSemiWide SemiLightSemiWideSemiWide SemiBoldSemiWide BoldSemiWide ExtraBoldSemiWide Black |
Clearly such detailed font drop-down lists are impractical, so maybe we should use slide bars or drag selected text handles.
OpenType properties that are used in shaping complex scripts like Arabic are invoked automatically by DirectWrite and Uniscribe. But many other OpenType properties including these examples are discretionary and must be present in the backing store to work. In addition, it’s desirable to be able to add other kinds of properties. The CHARFORMAT2::dwCookie allows a client to attach one 32-bit value to a text run, but there’s need to attach multiple properties such spelling, grammar, and other proofing-error annotations along with other client properties.
To handle all these properties, the latest Office 365 RichEdit implements property sets as described in the remainder of this post. The D2D/DirectWrite RichEdit mode (but not the GDI/Uniscribe mode) displays the OpenType properties as illustrated in the figures above. The following, admittedly technical, discussion describes the property-set object model, the RTF and binary file format additions for property sets, how to display variable-font and other OpenType features using DirectWrite, and the OpenType variable-font (fvar) table.
Kinds of Properties
The kinds of RichEdit character format properties are summarized in the table
| ID Range | Usage |
| 0..0xFFFF | Properties not in property sets |
| 0x10000..0x1FFFF | RichEdit temporary properties such as proofing errors |
| 0x20000..0x2FFFF | Client temporary properties |
| 0x30000..0x3FFFF | RichEdit persisted properties |
| 0x40000..0x2020201F | Reserved; returns E_INVALIDARG if used |
| 0x20202020..0x7E7E7E7Emmm | OpenType features/axis (if 0x80808080 mask = 0; else invalid) |
| 0x7E7E7E7F..0xFFFFFFFF | Reserved; returns E_INVALIDARG if used |
There are no persisted client properties since they are client-specific and could be misinterpreted if read by a different client.
Property Set Object Model
The client APIs for setting and getting properties are ITextFont2::SetProperty (id, value) and ITextFont2::GetProperty (id, pvalue). The id’s for these methods are given by xxxx, where xxxx is an OpenType feature tag, an OpenType variable-font axis tag (see MakeTag() below) or an annotation id defined in the table at the end of the preceding section. Since OpenType x’s belong to a limited set of ASCII characters in the U+0020..U+007E range, there’s plenty of room in the 32-bit id space to define other properties. Common properties like font weight are already represented as CCharFormat::_wWeight and in principle don’t need to be members of a property set. Since by default there are no properties in a property set, calling ITextFont2::SetProperty(id, tomDefault) deletes the property id if it exists. Note that id values < 0x10000 are reserved for other purposes, such as tomFontStretch (0x33E) to define a font’s stretch value. These values are well below the first possible OpenType id 0x20202020 (4 spaces). The largest OpenType tag is 0x7E7E7E7E, which gives 944 = 78,074,896 tags, although most of them will never be used or are used for other purposes such as ‘MATH’ for the math table. This leaves 2564 − 944 = 4,294,967,296 − 78,074,896 = 4,216,892,400 IDs for other purposes.
OpenType tags are constructed in the order given by the macro
#define MakeTag(a, b, c, d) (((d)<<24) | ((c)<<16) | ((b)<<8) | a)
For example, the variable-font weight axis tag ‘wght’ has the value 0x74686777.
Internally it’s useful to mark OpenType feature tags with a bit (tomOpenTypeFeature—0x00800000) to distinguish them from variable-font axis tags. This bit cannot be confused with annotation id’s which have values of 0x3FFFF or less. The feature tags are defined by the DWRITE_FONT_FEATURE_TAG enum defined in dwrite.h. The variable-font axis tags are defined by the font’s fvar table discussed below and in principle can be any combination of ASCII letters. So, if a tag isn’t a feature tag, we assume that it’s a variable-font axis tag and let DirectWrite accept or reject it.
Property Set RTF
In RTF, property sets are encoded similarly to the {\colortbl…} for colors and have the form
{\*\propsets id value…; …}
Here the id and value are 32-bit values that are encoded for all properties in a property set. Each property set is ended by a semicolon. This format is repeated for all property sets used in the text. If an id starts with an ASCII letter and consists of 4 ASCII letters, it is written as a character string. For example, the id ‘wdth’ is written as such for the 32-bit id value 0x68746477. If any byte in the id isn’t an ASCII letter, the id is written as a 32-bit integer. These choices make it easier to read property IDs. A value with no fractional part is written as an integer. A value with a fractional part is written as a decimal fixed-point number, e.g., 123.545. Any other combination is invalid and ends reading the RTF stream. The property set table {\*\propsets …} is stored in the RTF header following {\fonttbl …} and {\colortbl …} (if they are present).
An example with two property sets containing variable-font id’s is
{\*\propsets wght 800 wdth 104;wght 400;}
This syntax is a slightly simplified version of the variable-font CSS syntax used in web applications.
In the RTF body, a reference to the Nth property set in the \propsets table is given by \psN (like \crN for choosing the Nth color in the \colortbl). Here N is 0-based, that is, \ps0 refers to the property set immediately following \propsets.
Property Set Binary Format
The property id-value pair is written in the binary format as opyidProperty (0x8A), optProperty (opt8Bytes) followed by the 32-bit id and value. CPropertySet is written as opyidPropertySet (0x89), optPropSet (optArray) followed by the set’s opyidProperty’s. The array of property sets CPropertySets is written as opyidPropertySets (0x88), optPropertySets (optArray) followed by the opyidPropertySet’s. These constants are defined in rebinary.h.
Rendering Variable-Fonts and OpenType Features
In addition to backing-store enhancements, the display routines need to pass active variable-font axis coordinates and OpenType features to DirectWrite. See OpenType Variable Fonts for information about the DirectWrite APIs for this. To create a font specified in part by axis coordinates, RichEdit gets an IDWriteFontFace5 (see dwrite_3.h) with the desired axis coordinates in place of the usual IDWriteFontFace. It does this by calling IDWriteFontFace::QueryInterface() to get an IDWriteFontFace5 interface, calling IDWriteFontFace5::GetFontResource() to get an IDWriteFontResource interface, releasing the IDWriteFontFace5 and calling IDWriteFontResource::CreateFontFace() to get a new IDWriteFontFace5 with the desired axis coordinates. Then it uses this IDWriteFontFace5 instead of the original IDWriteFontFace.
To pass OpenType features to DirectWrite, copy them into a std::vector<DWRITE_TYPOGRAPHIC_FEATURES> and pass them to IDWriteTextAnalyzer1::GetGlyphs() and IDWriteTextAnalyzer1::GetGlyphPlacements(). Some font features, such as Gabriola’s stylistic set 6 ‘ss06’ introduce glyphs with ascents and/or descents that exceed the standard typo ascents and descents as discussed in High Fonts and Math Fonts. To display such large glyphs with no clipping, the rendering software needs to calculate the line ascent and descent from the glyph ink, rather than from the usual font values. This is the approach used with the LineServices math handler.
OpenType Variable Font Axes
The variable font axes are defined in the OpenType fvar table, which has the header
struct FvarHeader // Variable font fvar table header
{
OTUint16 majorVersion; // Major version of fvar table (1)
OTUint16 minorVersion; // Minor version of fvar table (0)
OTUint16 axesArrayOffset; // Byte offset from table start to first VariationAxisRecord
OTUint16 reserved; // Permanently reserved (2)
OTUint16 axisCount; // Count of VariationAxisRecord's
OTUint16 axisSize; // BYTE count of VariationAxisRecord (20 for this version)
OTUint16 instanceCount; // Count of InstanceRecord's
OTUint16 instanceSize; // BYTE count of InstanceRecord
}; // (axisCount*sizeof(DWORD) + (4 or 6))
Types like OTUint16 that begin with OT describe 4-byte, big-endian quantities that need reverse ordering to work with our little-endian machine architecture. The header is followed by axisCount VariationAxisRecord’s defined by
struct VariationAxisRecord
{
OTUint32 axisTag; // Tag identifying axis design variation
OTFixed minValue; // Minimum coordinate value (16.16 format)
OTFixed defaultValue; // Default coordinate value
OTFixed maxValue; // Maximum coordinate value
OTUint16 flags; // Axis qualifiers (hidden if 1)
OTUint16 axisNameID; // ID for 'name' table entry that provides axis display name
};
The axisTag’s have the same MakeTag() form as the regular OpenType tags. Since they are accessed via the OpenType fvar table, they are in a different namespace from the regular OpenType tags. We don’t know of any tag conflicts between the two name spaces, so it’s probably okay not to mark the axis tags differently. But internally we mark OpenType feature tags by setting the high bit of byte 2 (OR in tomOpenTypeFeature), since the tags consist of ASCII symbols in the range 0x20..0x7E. This marking avoids sending OpenType tags to the wrong DirectWrite APIs.
The VariationAxisRecord’s are followed, in turn, by the InstanceRecord’s defined by
struct InstanceRecord
{
OTUint16 subfamilyNameID; // ID for 'name' table entry giving subfamily name
OTUint16 flags; // Reserved for future use (0)
OTFixed coordinates[axisCount]; // instanceSize coordinates
OTUint16 postScriptNameID; // Optional. ID for 'name' table entry giving PostScript name
};
At some point, it might be worth dealing with the InstanceRecord’s, but it’s certainly easier to use axis coordinates than handle myriad localizable font names (see Holofont discussion in the introduction). RichEdit could export a facility for translating between the two, but probably such a facility should be delegated to the font picker. The localizable font names are designed to help end users recognize the nature of a variable font instance, but they aren’t efficient at the RichEdit level. They also aren’t usable for variable-font animations, since such animations vary axis coordinates continuously.
4-28 Decimal Floating-Point Format
The OpenType “fvar” table described in the previous section defines the min, max, and default variable-font axis coordinate values using the OpenType 16.16 numeric format. The integer part of the value is given by shifting right 16 bits, i.e., dividing by 65536. If the fractional part is nonzero, store the value in a floating-point variable and divide by 65536. In applications, coordinates are easier to read when the fractional part is 0 if only the integer part is displayed. Since purely fractional coordinates (values < 1) are useless, if the absolute value is less than 65536, the value can be understood to be an integer without a fractional part.
The OpenType 16.16 format is a binary fixed-point format that may encounter roundoff when converted to decimal, e.g., 800.1 → 800.100006. This roundoff is ugly in RTF, CSS, and dialog boxes. So we need a decimal floating-point format that doesn’t have such roundoff. The IEEE 754-2008 decimal floating-point encoding defines decimal32 with 20 bits of precision, a sign bit and the large exponent range of 10192. OpenType variable-font axis coordinates need at most four decimal places. The sign bit is used for the slant (slnt) standard axis and can be used for custom axes.
If the value has no fractional part, we store it as a standard 2’s complement integer rather than in the high word of 16.16 for readability in RTF, CSS and dialog boxes. To convert it to the 16.16 format, multiply by 65536. But if the value has a fractional part, we use the following signed 4-28 decimal floating-point format
| s | n | significand | |||||||||||||||||||||||||||||
| 31 | 30 | 29 | 28 | 27 | 26 | 25 | 24 | 23 | 22 | 21 | 20 | 19 | 18 | 17 | 16 | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
If the number is negative, the sign bit 31 is 1. Bits 0..27 are the significand. The decimal divide value n is defined by
| n | divide significand by |
| 000mm | (not floating point) |
| 001 | 10 |
| 010 | 100 |
| 011 | 1000 |
| 100 | 10000 |
| 101 | 100000 |
| 110 | 1000000 |
| 111 | (not floating point) |
n must have at least one 0 bit to distinguish the format from a negative 2’s complement integer and at least one 1 bit to distinguish it from a positive integer.
This gives 28 bits of precision with a maximum value of (1028 – 1)/10 = 26843545.5 with one decimal place and a minimum value of 0.000001 with six decimal places. These limits are beyond the values used for OpenType variable font-axis coordinates, which typically range between 1 and 1000. The 4-28 decimal floating-point format is easy to use and displays the original fixed-point values with no round-off error. To convert it to the 16.16 format, store the 28-bit significand field in a double variable, divide by the number corresponding to n, multiply by 65536 and round to the nearest integer. For the DWrite APIs, store the 28-bit significand field in a double, divide by the number corresponding to n and cast the result to a FLOAT.
In C, the 4-28 decimal floating-point format of the value x is recognized by the function IsDecimalFloat(x) defined by
#define IsDecimalFloat(x) IN_RANGE(3, (x >> 28) & 7, 6)
where IN_RANGE() is defined by
#define IN_RANGE(n1, b, n2) ((unsigned)((b) - (n1)) <= unsigned((n2) - (n1)))
The divide factor in the n table is given by pow(10, (x >> 28) & 3) or (x >> 28) & 3 can be used as a table index.