Transfering a pointer across processes
I seem to be "Riffing on Raymond" more and more these days, I'm not sure why, but..
Raymond Chen's post today on the type model for Win64 got me to thinking about one comment he made in particular:
Notice that in these inter-process communication scenarios, we don't have to worry as much about the effect of a changed pointer size. Nobody in their right mind would transfer a pointer across processes: Separate address spaces mean that the pointer value is useless in any process other than the one that generated it, so why share it?
Actually, there IS a really good reason for sharing handles across processes. And the Win64 team realized that and built it into the product (both the base team and the RPC team). Sometimes you want to allocate a handle in one process, but use that handle in another. The most common case where this occurs is inheritance - when you allocate an inheritable handle in one process, then spawn a child process, that handle is created in the child process as well. So if a WIn64 process spawns a Win32 process, all the inheritable handles in the Win64 process will be duplicated into the Win32 process.
In addition, there are sometimes reasons why you'd want to duplicate a handle from your process into another process. This is why the DuplicateHandle API has an hTargetProcessHandle parameter. One example of this is if you want to use a shared memory region between two processes. One way of doing this would be to use a named shared memory region, and have the client open it. But another is to have one process open the shared memory region, duplicate the handle to the shared memory region into the other process, then tell the other process about the new handle.
In both of these cases (inheritable handles and DuplicateHandle), if the source process is a 64bit process and the target process is a 32bit process, then the resulting handle is appropriately sized to work in the 32bit process (the reverse also holds, of course)
So we've established that there might be a reason to move a handle from one process to another. And now, the RPC team's part of the solution comes into play.
RPC (and by proxy DCOM) defines a data type call __int32644. An int3264 is functionally equivalent to the Win32 DWORD_PTR (and, in fact, the DWORD_PTR type is declared as an __int3264 when compiled for MIDL).
An __int3264 value is an integer that's large enough to hold a pointer on the current platform. For Win32, it's a 32 bit value, for Win64, it's a 64 bit value. When you pass an __int3264 value from one process to another it either gets truncated or extended (either signed or unsigned)..
__int3264 values are passed on the wire as 32bit quantities (for backwards compatibility reasons).
So you can allocate a block of shared memory in one process, force dup the handle into another process, and return that new handle to the client in an RPC call. And it all happens automagically.
Btw, one caveat: In the current platform SDK, the HANDLE_PTR type is NOT RPC'able across byte sizes - it's a 32bit value on 32bit platforms and a 64bit value on 64bit platforms, and it does NOT change size (like DWORD_PTR values do). The SDK documentation on process interoperability is mostly correct, but somewhat misleading in this aspect. It says "The 64-bit HANDLE_PTR is 64 bytes on the wire (not truncated) and thus does not need mapping" - I'm not going to discuss the "64 bytes on the wire" part, but most importantly it doesn't indicate that the 32-bit HANDLE_PTR is 32 bits on the wire.
Edit: Removed HTML error that was disabling comments...
Comments
- Anonymous
February 01, 2005
Larry,
what I definitely don't understand, is how RPC context handles are supposed to work if you have a 64-bit RPC Server and a 32-bit RPC client. Normally the server allocates some data and passes the pointer (64 bits address) to it as a context handle to the client. If across the wire this is truncated to 32 bits how can the RPC client pass it back in subsequent calls to the server and the server cast it back to the original 64-bit address appropriately if half of the ptr information gets lost?
Am I missing something? - Anonymous
February 01, 2005
You're talking about passing handles to shared memory accessable by API right?
Like HGLOBAL's ect... GlobalLock()
If you pass the actual paged address, it won't be the same when the other process accesses that location even if the memory is not page protected for that process because of the virtual table right?
This is unless you're talking about passing around pointers to unpaged memory for drivers right? - Anonymous
February 01, 2005
Stefan,
Do you truely mean context handles? As in the RPC [context_handle] type? In that case, the actual value passed on the wire is unrelated to the actual value on the server - the value on the wire is a "handle" to the 64bit value, it's not the actual contents.
If you don't want to use RPC context_handle's to pass values from the client to the server, then you need to use a LONGLONG for the value if you might be interoperating with a 64bit server.
But you need to be REALLY careful if you do that - you can't trust the value of that pointer if it comes from the caller - you need to treat as if it was an integer value and look its value up in a table on the server (otherwise the client has a great way of forcing arbitrary data into your server). - Anonymous
February 01, 2005
Beer29: Yup, I'm talking about handles to shared memory.
Using shared memory is a two part process in Win32. First, you create a "section", which gives you a handle to a piece of shared memory. You then "map a view" of that section into your processes address space.
The section handle can be shared between processes, the view of that section cannot. - Anonymous
February 01, 2005
Oh, and as for pointers to unpaged memory, user mode code can't access nonpaged pool (or paged pool). - Anonymous
February 01, 2005
Sorry Larry, just trying to post a comment.
-Scott - Anonymous
February 01, 2005
No problem Scott - it was a problem with the content of the post... - Anonymous
February 01, 2005
Yes, I meant real context handles. Up to just now I actually though the client would see the exact same value as the context handle as the server, but a minute of debugging showed to me you are right - the client sees handles, not the same pointer value as the server allocated. Thanks for clarifying this.
And no, I don't think of having state passed from the server to the client as something else than a context handle (the LONGLONG you mentioned), because is not a good idea for yet another reason: You won't have the luxury of the rundown method if your client goes away unexpectedly. - Anonymous
February 01, 2005
Stefan, Absolutely. But context handles have their own overhead - they act as binding handles and thus keep the RPC connection alive, sometimes that's not a good idea. - Anonymous
February 01, 2005
Isn't Raymond talking about pointers? You're talking about handles here. They're two different things. - Anonymous
February 01, 2005
Greg, I was waiting for someone to ask that :)
Actually handles and pointers aren't really different things. A HANDLE is a VOID *, which is a pointer. Logically (and by implementation) a handle is the same size as a pointer, and should be treated interchangably - in particular, a handle can have the top 32 bits set to a non zero value on a 64bit platform. - Anonymous
February 01, 2005
Last time I checked, in the current implementation, exhandles only have 24 bits of indexing information. The low two bits are tag bits, and the top eight aren't used by the handle table manager (though the top bit is used by the object manager for it's handles as a flag to identify a kernel mode only handle). - Anonymous
February 01, 2005
Isn't it a bit short-sighted calling it "__int3264" though? I mean, when 128-bit Windows comes out, they'll need a new type "__int3264128" won't they? - Anonymous
February 01, 2005
There is a difference between sharing handles across processes and transfering pointers across processes. I think there are valid cases for transfering pointers across processes but they're a lot rarer than for sharing handles.
(Calling all editors:
> I'm not going to discuss the "64 bytes on
> the wire" part, but most importantly it
> doesn't indicate that the 32-bit HANDLE_PTR
> is 32 bits on the wire.
For consistency that should be 32 bytes on the wire ^` Fortunately 644 bits can handle it ~^) - Anonymous
June 06, 2005
Not quite "Riffing on Raymond" but he just wrote about this, and it reminded me of a story that was related...