Planning for a new level of verifications in automation

I'm getting ready to check in a new level of verification for our automated tests. The verification is to ensure we follow our schema. It's pretty basic, and for almost all cases the verification passes. In some cases, though, it fails, but that is expected since we automation tests that inject errors. And if we are deliberately injecting errors, then verifications need to detect that OneNote caught the error state. So for those cases, this verification will be off.

But there is a little bit more to it than this. Right now, the code that checks our schema only operates on all opened notebooks. It will not limit itself to the current notebook, section group, section or page. This is not large hurdle that we need to overcome any time soon. Since it is automated, I really would rather check all notebooks that just a single section - might as well verify it all since run time in a lab is relatively cheap. To verify a small notebook - roughly the size of the OneNote Guide notebook, takes just a few seconds. But that runtime, while cheap, is not free, and this verification can potentially take minutes if the total size of the notebooks gets huge.

But for now, I'm trying to decide on implementing this verification as a boolean (either run it or don't) or as a list of possible levels to verify (all_notebooks, current_notebook, current_section, etc…). This may seem like a trivial decision, but we have a requirement that if we check code into our test library, it has to be used in a test. This keeps us from accumulating a bunch of unreachable code that is hard to maintain and can potentially cause problems when we build. So I need to decide on something like this:

enum VerificationLevel
{
  AllNotebooks = 0,
  CurrentNotebook,
  CurrentSectionGroup,
  CurrentSection,
  CurrentPage
};

with only AllNotebooks being enabled, and returning a "Not Yet Implemented" error if someone tries to use one of the others,

or

bool verifyEntireNotebook = true;

Obviously, the second is much simpler and less likely to cause problems down the line, at least until the new levels of verification get added. But the first method lays the groundwork for future changes. I'll talk this over with our automation driver and she what she has to say and then go down her recommended path.

Questions, comments, concerns and criticisms always welcome,

John