Share via

Tablet PC: Getting started with InkAnalysis – intro to parsing

  • Article

Last time:

https://blogs.msdn.com/gavingear/archive/2006/09/11/750005.aspx

We took a look at asynchronous analysis and handwriting recognition using the new InkAnalysis API (Part of the Windows Vista SDK for RC1). This time, we’ll extend that code to include parsing (layout analysis of ink) results.

Q: What is parsing?

A: Parsing can be broken down into a couple basic categories: 1. Writing/Drawing classification of ink –and- 2. Categorization and relation of ink (Ex: These strokes make up a drawing that enclose those strokes that make up a word)

Example of a drawing containing a word:

 

Here’s a breakdown of what we’ll do to modify the asynchronous analysis application to incorporate parsing results: (In the ResultsUpdated event handler)

-          Call InkAnalyzer.FindNodesOfType() to obtain a ContextNodeCollection of InkWord objects

-          Call InkAnalyzer.FindNodesOfType() to obtain a ContextNodeCollection of InkDrawing objects

-          Build up a string that outputs the recognition for each of the words and each of the drawings, and display that string to the user

Example Code:

//

// Basic Ink enabled Windows Forms application with

// handwriting recognition using InkAnalyzer

// Gavin Gear - https://blogs.msdn.com/gavingear

// 09/2006

//

using System;

using System.Collections.Generic;

using System.ComponentModel;

using System.Data;

using System.Drawing;

using System.Text;

using System.Windows.Forms;

using Microsoft.Ink; // The managed Tablet PC API

namespace BasicInkApplication

{

    public partial class BasicInkApplication : Form

    {

        // The InkOverlay that we'll attach to our Form

        private InkOverlay inkOverlay;

        private InkAnalyzer inkAnalyzer;

        public BasicInkApplication()

        {

            InitializeComponent();

            // Create an InkOverlay object that's attached to the Form

            this.inkOverlay = new InkOverlay(this);

            // Enable the InkOverlay (default is Enabled == false)

            this.inkOverlay.Enabled = true;

            this.inkOverlay.Stroke += new InkCollectorStrokeEventHandler(inkOverlay_Stroke);

            // Create a new InkAnalyzer

            // - Associate with the InkOverlay's Ink object

            // - Send the Form "this" as the synchronizing object

            this.inkAnalyzer = new InkAnalyzer(this.inkOverlay.Ink, this);

            // Hook up to the InkAnalyzer.ResultsUpdated event

            // in order to be notified when BackgroundAnalyze() finishes

            this.inkAnalyzer.ResultsUpdated += new

                         ResultsUpdatedEventHandler(inkAnalyzer_ResultsUpdated);

            // The InkOverlay needs to be disposed due to unmanaged resources

            // used by the InkOverlay

            this.FormClosing += new FormClosingEventHandler(BasicInkApplication_FormClosing);

        }

        void inkOverlay_Stroke(object sender, InkCollectorStrokeEventArgs e)

     {

            // We have a new stroke, add it to the InkAnalyzer

            this.inkAnalyzer.AddStroke(e.Stroke);

        }

        void BasicInkApplication_FormClosing(object sender, FormClosingEventArgs e)

        {

            this.inkOverlay.Dispose();

            this.inkAnalyzer.Dispose(); // Free the unmanaged resources

        }

        private void buttonRecognize_Click(object sender, EventArgs e)

        {

            // Kick off asynchronous analysis

            this.inkAnalyzer.BackgroundAnalyze();

        }

        void inkAnalyzer_ResultsUpdated(object sender, ResultsUpdatedEventArgs e)

        {

            // Build up a summary of the parsing structure

            // this is essentially a hierarchical collection

      // of elements called ContextNodes which can represent things

            // like Drawings, Paragraphs, Lines, Words, and even relationships

            // (ContextLinks) like connectors, containers, etc

            // In this case, we'll keep things simple by just showing the user drawings

            // and words

            ContextNodeCollection words =

                         this.inkAnalyzer.FindNodesOfType(ContextNodeType.InkWord);

            ContextNodeCollection drawings =

                         this.inkAnalyzer.FindNodesOfType(ContextNodeType.InkDrawing);

            string summary = "Words:" + Environment.NewLine;

            foreach (InkWordNode node in words)

            {

                summary += "\t" + node.GetRecognizedString() + Environment.NewLine;

            }

            summary += Environment.NewLine + "Drawings:" + Environment.NewLine;

            foreach (InkDrawingNode node in drawings)

            {

                summary += "\t" + node.GetShapeName() + Environment.NewLine;

            }

            MessageBox.Show(summary, "Parsing Results");

        }

    }

}

 

So that’s a basic into to parsing. This is a simple example, but gives you a basic idea of what parsing is, and how to get started with it. We’ll look at parsing in greater detail in subsequent posts.

See Ya,

Gavin