What does the POINTER_INFO frameId really mean or guarantee?

Jim Bean 0

I'm having trouble understanding what the frameId really means.

Are all devices generating simultaneous pointers combined into one frame with the same frame id?

I am having trouble understanding this description:

"An identifier common to multiple pointers for which the source device reported an update in a single input frame. For example, a parallel-mode multi-touch digitizer may report the positions of multiple touch contacts in a single update to the system. Note that frame identifier is assigned as input is reported to the system for all pointers across all devices. Therefore, this field may not contain strictly sequential values in a single series of messages that a window receives. However, this field will contain the same numerical value for all input updates that were reported in the same input frame by a single device."

Does this mean that the frame id associated with a pointer window message may be older than the latest frame id? Does it mean that devices have their own separate id's? Confusing...

I need a better understanding.

Jeanine Zhang-MSFT 10,046 Reputation points Microsoft Vendor

2024-12-04T07:24:19.54+00:00

Are all devices generating simultaneous pointers combined into one frame with the same frame id?

However, this field will contain the same numerical value for all input updates that were reported in the same input frame by a single device.

In my opinion, for a single device, the input of all pointers in the same input frame have the same frameId. Frame identifiers are different on different devices

Does this mean that the frame id associated with a pointer window message may be older than the latest frame id? Does it mean that devices have their own separate id's?

Could you please tell us what do you mean "the frame id associated with a pointer window message may be older than the latest frame id"?
Jim Bean 0 Reputation points

2024-12-04T07:58:43.5666667+00:00
@Jeanine Zhang-MSFT

Could you please tell us what do you mean "the frame id associated with a pointer window message may be older than the latest frame id"?

Say my window receives WM_UPDATE messages from touch screen A, and then afterwards WM_UPDATE messages from touch screen B.

Which is possible?

pointerInfo_A->frameId < pointerInfo_B->frameId

pointerInfo_A->frameId > pointerInfo_B->frameId

pointerInfo_A->frameId <= pointerInfo_B->frameId

pointerInfo_A->frameId >= pointerInfo_B->frameId
Darran Rowe 1,236 Reputation points

2024-12-04T09:42:06.9333333+00:00

I would have to double check with a sample, but I believe that the answer is all of the above.

First of all, the frame id is an unsigned 32 bit value. So this means that there will be approximately 4 billion frames before all of the ids are exhausted, and for input this happens fairly quickly. The id itself should only be there to distinguish itself from other recent input frames though, so it isn't really that much of an issue if it just rolls over.

The big thing is that the pointer messages should send individual messages for each pointer. To give an example of what I mean, consider a situation where 4 fingers are in contact with a screen over a single window. If all 4 fingers are moved, then the window will receive 4 separate WM_POINTERUPDATE messages, one for each contact. The frame id would be used to indicate which updates are grouped together.

But I think the point that is really being made here is that because of how Windows handles input, if two sets of input are read for the same window then there is no guarantee that pointerInfo_B->frameId == pointerInfo_A->FrameId + 1.
Roy Li - MSFT 33,496 Reputation points Microsoft Vendor

2024-12-13T06:07:21.6133333+00:00

Hope you've been well. Any updates about this issue?
Jim Bean 0 Reputation points

2024-12-16T00:46:21.4066667+00:00

@Darran Rowe I did some testing, and I enabled the mouse for generating POINTER_INFO through the API functions. The result is that devices can indeed have their own separate frame ID's, and there is no guarantee about the numerical order of frame ID's extractable from POINTER_INFO. However, because frames can only be generated from a device sequentially, the frame ID from the same device should be increasing unless (as you pointed out) the uint cycles. And (don't take this as a fact) because it appears messages are generated at the same time a frame is generated, it does not seem possible for a different device to generate its own frame and messages (chronologically) in the middle of the messages associated with the other devices frame.

This information is important because it could allow a developer to improve performance by only processing a message with a unique frame, rather than every message with the same frame, WITHOUT indexing the devices to keep track of what last frameId is associated with them, and also without having to worry about the uint cycling back to zero.

For example, the messages come in bursts:

[WM_UPDATE] frame = 1, device = 1;

[WM_POINTER_DOWN] frame = 1, device = 1;

[WM_POINTER_DOWN] frame = 1, device = 1;

[WM_UDPATE] frame = 2, device = 2;

[WM_POINTER_DOWN] frame = 2, device = 2;

And so the developer should only have to process the first message associated with each change in frame id, and ignore the duplicate frame id's coming after. However, the processing code should still filter based on pointers, so that if the behavior changes the code will not logically break but only see a slight reduction in performance.
Darran Rowe 1,236 Reputation points

2024-12-16T21:26:43.71+00:00
You also have to be careful assuming that two subsequent frames read from the WM_POINTERxxx messages will produce sequential frame IDs, even taking multiple devices into account. While I'm not saying that this is what you are currently doing, @Jim Bean , I'm just getting this written here so that if anyone else finds this post in the future, they are able to come to the correct conclusion.

While Windows currently produces sequential IDs from all of the devices in the system, I believe that it is best to see this as an implementation detail more than anything. The only important information that should be seen in the frame ID is that the ID indicates if it was read along with other input in parallel from the same device. But in general, if there are two sequential WM_POINTERxxx messages, there is still no guarantee that the frame ID obtained will be equal to the previous ID, or if it will be the previous ID + 1.

First of all, if a pointer stays purely in the non client area, then all of the pointer messages will be non client messages, the client area will likely just see a WM_POINTERENTER followed by a WM_POINTERLEAVE a little while later. The frame IDs here don't have to be anywhere close either. Secondly, if the message pump is slower in obtaining the pointer messages than the device is in producing input, then Windows is documented to coalesce input in this case. In other words, Windows will obtain all pending input at once. The POINTER_INFO obtained from GetPointerInfo will be the latest input information meaning there will be a gap in the frame IDs. This isn't something that happens on a slow device either, the pen input on my surface regularly has coalesced input but the sample application that I was testing with is actually blocking on GetMessage.

But one final important point to note is that if the desire is to process the entire input frame as a unit, then GetPointerFrameInfo, GetPointerFrameInfoHistory and SkipPointerFrameMessages make things a lot easier. The biggest thing to be aware of is that the WM_POINTERENTER and WM_POINTERDOWN are part of the same frame and WM_POINTERUP and WM_POINTERLEAVE are also part of the same frame.

Pointer Enter Pointer ID: 316 Frame ID: 48709 Source Device: 0x10076 Target Window: 0x40766 Pointer Down Pointer ID: 316 Frame ID: 48709 Source Device: 0x10076 Target Window: 0x40766 Pointer Update Pointer ID: 316 Frame ID: 48710 Source Device: 0x10076 Target Window: 0x40766

Pointer Update Pointer ID: 316 Frame ID: 48761 Source Device: 0x10076 Target Window: 0x40766 Pointer Up Pointer ID: 316 Frame ID: 48762 Source Device: 0x10076 Target Window: 0x40766 Pointer Leave Pointer ID: 316 Frame ID: 48762 Source Device: 0x10076 Target Window: 0x40766

Pointer Up Pointer ID: 318 Frame ID: 49246 Source Device: 0x10076 Target Window: 0x40766 Pointer Up Pointer ID: 317 Frame ID: 49246 Source Device: 0x10076 Target Window: 0x40766 Pointer Leave Pointer ID: 318 Frame ID: 49246 Source Device: 0x10076 Target Window: 0x40766 Pointer Leave Pointer ID: 317 Frame ID: 49246 Source Device: 0x10076 Target Window: 0x40766

There is also no guaranteed ordering in the pointer IDs inside a frame.

Pointer Update Pointer ID: 318 Frame ID: 49223 Source Device: 0x10076 Target Window: 0x40766 Pointer Update Pointer ID: 321 Frame ID: 49223 Source Device: 0x10076 Target Window: 0x40766 Pointer Update Pointer ID: 317 Frame ID: 49223 Source Device: 0x10076 Target Window: 0x40766

This was logged from an application taking touch input.
Jim Bean 0 Reputation points

2024-12-17T08:00:27.0833333+00:00

@Darran Rowe Yes, that is the behavior I was also experiencing. The one thing I think you left out is that WM messages are clustered around the same frame. That is, they are all posted or sent at the same time because windows determines what messages to create based off the frame itself. For example, if your fingers break contact all in the same frame, then the behavior I observe is a WM_POINTERUPDATE followed by multiple WM_POINTERUP with no messages from other devices in-between.

And as I stated before, this is just observed behavior, so there are no guarantees it will stay that way. Windows could easily have multiple threads creating the messages. The nice part is this should be spectacularly rare, and one can design their app using exactly the API functions you listed in your comment, which is what I am currently doing. Contiguous messages with the same frame ID routed to other processing, such as deferred heavier processing from pervious frames. And if the behavior has changed and some other device message breaks the contiguous frame ID of the original device, then a stored frameID is actually checked. In my case, I manage a control object pool which contain the IC's. It seems like a pretty performant way to do it.
Darran Rowe 1,236 Reputation points

2024-12-17T17:44:59.6633333+00:00

For example, if your fingers break contact all in the same frame, then the behavior I observe is a WM_POINTERUPDATE followed by multiple WM_POINTERUP with no messages from other devices in-between.

The WM_POINTERUPDATE message(s) is/are not part of the same frame as the WM_POINTERUP/WM_POINTERLEAVE messages. I was doing some messing around using both pen and touch input on the same window, and the pen input could certainly get in between the last WM_POINTERUPDATE message(s) for the touch input and the WM_POINTERUP/WM_POINTERLEAVE messages for the touch input. The pen I was using had a higher polling rate than touch.
Roy Li - MSFT 33,496 Reputation points Microsoft Vendor

2024-12-20T02:38:16.4066667+00:00

Hope you've been well. Any updates about this issue?

Share via

What does the POINTER_INFO frameId really mean or guarantee?

Your answer