MSDN Library and our "Coverage" Standards
I want to preface this article that the processes described are not generic to Microsoft and are specific to my team at the time that I was writing SDK documentation. Please take this information with a grain of salt as processes on other teams (under different management, for example) could be entirely different and the processes on the same teams that I participated on could have changed entirely. That said, hopefully this article helps you understand one of the ways that content teams at Microsoft make decisions for developer documentation.
An Introduction to OGF in Documentation
In a post I wrote earlier on the Redmond RDF, a number of people commented on the documentation some indicating that it was excellent and others saying that there is a low quality bar. We have adopted a system in the Windows Developer content space that addresses what we consider API "coverage" and this system is called the OGF system. OGF stands for outstanding, good, and fair and sets standards for how comprehensive an API reference page is documented. Outstanding is the most comprehensive, good is the next most covered, and fair is the least covered. API reference pages, such as the WM_TOUCH message, the IInertiaProcessor:::Reset method, and the IManipulationProcessorPivotPointX property all have associated OGF target and OGF actual values with them that indicate the planned completeness and the actual coverage of the documentation.
When we are planning documentation for yet to be released SDKs and APIs, we might start with the scenario planning for the SDK/API and look at APIs and usage scenarios and then will set planned OGF values based on whether an API is used in high priority scenarios or not. This information is captured in documentation plans and in XML metadata within the documentation itself so that we can plan and track our coverage. The general goal of setting target OGF values is to ensure that the documentation that will be used the most will be the most comprehensive. It's not uncommon for a programming writer to be supporting hundreds if not thousands of API reference pages, so not all pages can have an outstanding OGF value. Additionally, if more time is spent by programmer/writers documenting APIs to high levels of completeness, the developer guidance documentation (this is code examples, samples, and high level overviews) will not get as much attention. Another consideration when setting target OGF values is the complexity of an API: I would oftentimes change my targets if an API was complex enough that it would benefit from getting more coverage.
How do we define OGF?
The specifics have changed since I was a writer but as best I can describe it, the following standards are how we define the 3 levels of coverage (plus stub, which really means uncovered):
Stub - The documented API contains a syntax block and basic API description, but no authored descriptions. When I was an author, this was generated using a script that parsed MIDL files. There was a plan to create this using specifications from program managers, but I'm not sure we have implemented this yet.
Fair (F) - The documented API has the completeness of a stub and additionally has descriptions for the API, its parameters, and return values. At this point, the documentation is pretty much still boilerplate.
Good (G) - The documentation meets the requirements for Fair, specifies default values, and provides authored descriptions for autogenerated exceptions, and has appropriate remarks.
Outstanding (O) - The documentation meets the requirements for Good and additionally documents possible values as needed, provides a Remarks section about usage, a Code Example, and lists See Also links relevant to conceptual, sample, and reference pages.
Why are we doing this?
There are really two principle goals that we have when using OGF for planning:
- Ensuring that the effort that goes into reference documentation is focused to the right APIs.
- Ensuring that our documentation is compliant.
I have in some ways covered the first goal in my introduction but the second one is a little more complicated. Microsoft is subject to a consent decree which mandates that we adequately document protocols and APIs that are exposed and reused in our products. The "F" level of completeness is the bare minimum for compliance to the consent decree and ensures that we protect ourselves from legal action. In other words, we create at a minimum "F" level documentation for every API we ship in every SDK to ensure that we don't get sued which is one of the highest priorities for programmer/writers. What's also interesting is that we structure our documentation in XML and have scripts that automate and validate our documentation against our OGF standards to ensure that we are keeping accurate Target and Actual values in our documentation metadata.
How does OGF / compliance affect documentation quality?
Regarding OGF, I feel that it helps to improve the quality of documentation overall. By prioritizing our work to what will be the most heavily used APIs, we ensure that we ship our documentation on time with a high bar for coverage where most developers will be using the API references. The downside is that there are still APIs out there with a coverage level of F and additionally, there are many documentation sets that have been documented for completeness (some that I authored in fact) that are really documented to a bare minimum. In many cases, the documentation was created just to ensure minimum coverage for compliance.
When I was working on compliance documentaiton for Microsoft's DRM system, I wrote a blog post as a response to criticisms I read on Slashdot and I lamented the challenges that programmer/writers face when making trade-offs between compliance and quality and also bringing up the issue of documenting APIs that are changing, a common challenge for PWs:
The challenges that Microsoft faces in producing compliance documentation for software are varied. For starters, what is sufficient documentation, and for whom? For an architect, a specification that shows the system from a high level and that specifies the protocols for a particular technology may be sufficient to communicate a means for implementing a custom or third party application that uses or inter-operates with a technology. For SOMEdevelopers (and some interfaces for that matter), code demonstrating how an API is used is sufficient to use that API. For others, a broad explanation of the particulars of that interface and its usage are necessary. What if our audience is NOT a developer? What is sufficient for those people? Do we need to explain what a computer is, what networking is, and so on? Where do we, as writers draw the line? How do we identify the right areas to focus our attention. The even larger problem is that we are constantly working with a moving target. The documentation MUST ship for compliance at the same time that its related product is shipping. If halfway through the product's development life cycle, the interfaces change, all of the work that we, as writers, have produced can immediately become useless. Another problem is that if a particular design will not work, the architecture can change, eliminating much of the writing work that was performed to explain the architecture. These are the problems that face writers every day.
What does this mean for developers?
This really may not mean very much for you but hopefully it helps you understand the reasons behind there being some of the differences between one reference topic and another. In many cases, if you are on an API page and the documentation seems really scarce, it may be like that because the API is deprecated or going away, or it may be due to there being a better, more completely documented solution that we're passively trying to steer you to. No matter what, if you see a page with scarce documentation that is critical to a scenario that you are trying to address, please send the writer feedback on the page so that we can be aware that customers are trying to use that API. In many cases, the right resolution to issues with a particular reference page is to include links to the right conceptual overview, a code example, or perhaps even another SDK.