How To: Debug a Custom Debug Engine
A project type launches the debug engine (DE) from the DebugLaunch method. This means that the DE is launched under the control of the instance of Visual Studio controlling the project type. However, that instance of Visual Studio cannot debug the DE. What follows are the steps to allow you to debug your custom DE.
Note
: In the "Debugging a Custom Debug Engine" procedure, you must wait for the DE to start before you can attach to it. If you place a message box near the beginning of your DE that appears when the DE starts, you can attach at that point and then clear the message box to continue. That way, you can catch all DE events.
Warning
You must have remote debugging installed before you attempt the following procedures. See Remote Debugging and Diagnostics for details.
Debugging a Custom Debug Engine
Start msvsmon.exe, the Remote Debug Monitor.
From the Tools menu in msvsmon.exe, select Options to open the Options dialog box.
Select the "no authentication" option and click OK.
Start an instance of Visual Studio and open your custom DE project.
Start a second instance of Visual Studio and open your custom project that launches the DE (for development, this is typically in the experimental registry hive that is set up when VSIP is installed).
In this second instance of Visual Studio, load a source file from your custom project and start the program to be debugged. Wait a few moments to allow the DE to load, or wait until a breakpoint is hit.
In the first instance of Visual Studio (with your DE project), select Attach to Process from the Debug menu.
In the Attach to Process dialog box, change the Transport to Remote (Native only with no authentication).
Change the Qualifier to the name of your machine (note: there is a history of entries, so you need to type in this name only once).
In the Available Processes list, select the instance of your DE that is running and click the Attach button.
After the symbols have loaded in your DE, place breakpoints in your DE code.
Every time you stop and then restart the debugging process, repeat steps 6 through 10.
Debugging a Custom Project Type
Start Visual Studio in the normal registry hive and load your project type project (this is, the source to your project type, not an instantiation of your project type).
Open the Project properties and go to the Debug page. For the Command, type the path to the Visual Studio IDE (by default, this is [drive]\Program Files\Microsoft Visual Studio 8\Common7\IDE\devenv.exe).
For the Command Arguments, type /rootsuffix exp for the experimental registry hive (created when VSIP was installed).
Click OK to accept the changes.
Start your project type by pressing F5. This will launch a second instance of Visual Studio.
At this point, you can place breakpoints in your project type source code.
In the second instance of Visual Studio, load or create a new instance of your project type. During the load or creation, your breakpoints may be hit.
Debug your project type.
If you choose to debug the process of launching a DE, you can perform the steps in the "Debugging a Custom Debug Engine" procedure to attach to your DE after it is launched. This will give you three instances of Visual Studio running: one for your project type source, a second for your instantiated project type, and a third attached to your DE.