Chapter 19. Deeper Into PInvoke and Useful Examples
In This Chapter
• Callbacks
• Passing Structures
• Handling Variable-Length Structures and Signatures
• Using C# Unsafe Code
• Guarding Against Premature Garbage Collection
• Choosing the DLL Location or Name Dynamically
• Example: Responding Immediately to Console Input
• Example: Clearing the Console Screen
• Example: Using CoCreateInstanceEx to Activate Remote COM Objects
In this chapter, we cover a variety of advanced topics, but focus mainly on two essential topics that weren’t covered in the previous chapter: callbacks and structure marshaling. In examining these topics, we cover useful examples that you might want to copy and paste into your own applications. You can never have too many PInvoke examples!
Many of the examples in this chapter demonstrate Win32 console APIs, because System.Console exposes only a small fraction of everything that Win32 APIs enable you to accomplish. The most useful examples in this chapter show you how to:
• Receive progress notifications while copying a file (Listing 19.1)
• Trap Ctrl+C, Ctrl+Break, or other Windows termination signals in a console application (Listing 19.2)
• Write colored text to the console (Listing 19.4)
• Respond immediately to console input without waiting for the user to press the Enter key (Listing 19.13)
• Clear the console window (Listing 19.14)
• Use CoCreateInstanceEx to activate remote COM objects (Listing 19.15)
Callbacks
Often, static entry points such as Win32 functions require callback behavior. The most common mechanism for this is function pointers. For example, the CopyFileEx API in KERNEL32.DLL copies a file, enabling the caller to pass a callback function to receive progress updates. This function, declared in winbase.h, is documented with the following signature:
The LPPROGRESS_ROUTINE type represents a pointer to a function with the following signature:
For .NET applications that wish to define managed callback functions used by unmanaged code, the Interop Marshaler provides support for bridging the unmanaged concept of function pointers with the managed concept of delegates. For the other direction of passing an unmanaged function pointer to a .NET component for callbacks, there is no built-in support. This section examines what can be done in both of these directions.
Using Delegates as Function Pointers
To take advantage of the function pointer callback mechanism in managed code, we can define a delegate with the desired signature. For the CopyFileEx example, the delegate would look as follows in C#:
delegate uint CopyProgressDelegate(long TotalFileSize,
long TotalBytesTransferred, long StreamSize,
long StreamBytesTransferred, uint dwStreamNumber,
uint dwCallbackReason, IntPtr hSourceFile,
IntPtr hDestinationFile, IntPtr lpData);
This delegate type can be used in a CopyFileEx PInvoke signature where a function pointer is expected:
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
static extern bool CopyFileEx(string lpExistingFileName,
string lpNewFileName, CopyProgressDelegate lpProgressRoutine,
IntPtr lpData, [In] ref bool pbCancel, uint dwCopyFlags);
Listing 19.1 demonstrates the use of the CopyFileEx function and its CopyProgressDelegate delegate. It uses the ErrorHelper.GetErrorMessage function we defined in the previous chapter, so it should be compiled with ErrorHelper.cs from Listing 18.9.
Listing 19.1. The CopyFileEx Function Calls Back into Managed Code
Lines 6–8 define some Win32 constants to be used inside the callback function, found in winbase.h. Lines 10–14 define the delegate, and Lines 16–19 define the PInvoke signature. CopyFileEx is actually exported as a pair of methods—CopyFileExA and CopyFileExW—so Line 17 sets the function’s character set to Auto. SetLastError is set to true because CopyFileEx uses SetLastError to return an error code upon failure. This PInvoke signature would look as follows in Visual Basic .NET:
Declare Auto Function CopyFileEx Lib "kernel32.dll" _
(ByVal lpExistingFileName As String, ByVal lpNewFileName As String, _
ByVal lpProgressRoutine As CopyProgressDelegate, _
ByVal lpData As IntPtr, <[In]> ByRef pbCancel As Boolean, _
ByVal dwCopyFlags As Integer) As Boolean
Line 21 defines a static CopyProgressDelegate field to store a reference to the delegate we create in Line 26. Storing a static reference isn’t necessary for this example, because the callbacks only occur during the CopyFileEx function call. However, you generally need to take explicit action to prevent delegates from being garbage collected prematurely when unmanaged code holds onto a corresponding function pointer for later callbacks. The garbage collector has no way of knowing that unmanaged code plans to call back on your delegates, so if no managed code references the object, it is considered eligible for garbage collection.
Caution
The number one mistake made when using delegates as unmanaged function pointers is to allow them to be garbage collected before unmanaged code is finished using them. Premature collection of delegates can result in several different symptoms, depending on the implementation of the unmanaged code using the function pointers. For example, it could cause an access violation, or it could cause your callback methods to never be called while the rest of the program runs normally. An easy way to prevent collection is to assign the delegate reference to a static field, as done in Listing 19.1.
When the delegate is created in Line 26, we specify the OnProgress function defined later in the listing, which has the appropriate signature. Lines 30–31 call the CopyFileEx function, passing the names of the source file and destination file, the delegate instance, and a few miscellaneous parameters.
If the call fails (indicated by a false return value), Line 35 calls Marshal.GetLastWin32Error and Line 36 uses the ErrorHelper.GetErrorMessage method we defined in the previous chapter to retrieve an error message if errorCode is recognized by the operating system.
The callback method is defined in Lines 50–72, and prints some of the passed-in information to the console. Line 71 returns the PROGRESS_CONTINUE value because the Win32 documentation for the callback method states that it must return this value in order for the file copying to continue.
When this program runs, it prints out information like the following (as long as you have a c:file1.txt file ready to be copied):
Copy file...
*************** New Stream ***************
************* Chunk Finished *************
File Bytes: 65536 of 89430
Stream Bytes: 65536 of 89430
************* Chunk Finished *************
File Bytes: 89430 of 89430
Stream Bytes: 89430 of 89430
...Finished
For another example of calling a Win32 function that calls back into managed code, Listing 19.2 contains Visual Basic .NET code that uses the SetConsoleCtrlHandler function in KERNEL32.DLL, which has the following unmanaged definition:
BOOL SetConsoleCtrlHandler(PHANDLER_ROUTINE HandlerRoutine, BOOL Add);
PHANDLER_ROUTINE represents a pointer to a function with the following signature:
BOOL WINAPI HandlerRoutine(DWORD dwCtrlType);
The SetConsoleCtrlHandler function enables a console application to handle control signals that ordinarily end the process. These signals are provoked when:
• The user presses Ctrl+C
• The user presses Ctrl+Break
• The user closes the console window (by selecting either Close from its menu or End Task from Task Manager)
• The user logs off
• The computer is being shut down (for services only)
An application can be notified of these events to perform cleanup upon closing or to prevent closing, although the user can still force the program to close in the latter three cases.
Caution
In Version 1.0 of the .NET Framework, delegates are always marshaled as function pointers with the StdCall calling convention (also known as Winapi). If you require passing a function pointer with a different calling convention to unmanaged code, you can’t use a delegate. Your best bet is to use an unmanaged function pointer via an IntPtr type. Fortunately, Win32 APIs use StdCall for all callback function pointers.
Listing 19.2. The SetConsoleCtrlHandler Function Enables a Console Application to Trap Control Signals
Lines 3–9 contain an enumeration of all the values that can be passed to the callback function. Lines 13–15 contain the PInvoke signature for SetConsoleCtrlHandler and Lines 17–18 contain the delegate definition used for callbacks.
Line 24 creates an instance of the delegate (using VB .NET’s shortcut syntax of AddressOf FunctionName) and assigns it to the shared (static) field defined on Line 20. This is critical for this example because all the callbacks occur after the call to SetConsoleCtrlHandler in Line 27 yet there’s no evidence to the garbage collector that unmanaged code is effectively holding onto a reference to the delegate. After calling the method with the delegate instance and true (to indicate that we’re adding a handler rather than removing one), Lines 30–31 contain a while loop that keeps the program waiting until the user presses Q followed by the Enter key to exit the program. Line 34 unhooks our handler to end the program.
The OnSignal callback method is defined in Lines 37–52, and handles three of the signal types by printing a message at the console. Returning true indicates that the signal has been handled, so the process does not end as usual.
Tip
When a delegate is marshaled as an unmanaged function pointer, its parameters and return type have PInvoke’s default marshaling behavior rather than COM Interoperability’s default marshaling behavior. This is true regardless of whether the function pointer is passed to a static entry point or to a COM member. For example, the Boolean return value for ConsoleCtrlDelegate in Listing 19.2 is correctly marshaled as BOOL rather than VARIANT_BOOL.
Invoking Unmanaged Function Pointers in Managed Code
Version 1.0 of the .NET Framework has a limitation that function pointer/delegate marshaling is supported only in one direction. Delegates can be marshaled to unmanaged function pointers, but in the reverse direction, unmanaged function pointers cannot be marshaled to managed code as delegates (except for function pointers that originated as delegates). This means that unmanaged code can naturally use a managed callback function, but managed code cannot naturally use an unmanaged callback function.
This section demonstrates one way to get around this limitation and still invoke unmanaged functions whose pointer is somehow passed into managed code. The first step is to treat the unmanaged function pointer being passed to managed code as an IntPtr type. This is necessary in order to prevent the Interop Marshaler from recognizing it as a type that it can’t marshal. For example, the GetProcAddress API in KERNEL32.DLL returns a function pointer, but can be defined in C# as follows:
[DllImport("kernel32.dll")]
static extern IntPtr GetProcAddress(IntPtr hModule, string lpProcName);
Had you attempted to define the return type as a delegate, calling this method would cause an ArgumentException with the message, “Function pointer was not created by a Delegate.”
Although unmanaged function pointers can’t be treated as .NET delegates, they can be directly invoked in managed code using the MSIL calli instruction. Such a call can be emitted by the Visual C++ .NET compiler when casting an IntPtr instance to an unmanaged function pointer and subsequently calling it. Such a call can also be done in IL Assembler syntax, but there is no built-in language construct in C# or Visual Basic .NET to enable such an invocation on an IntPtr type.
If you’re programming in C# or VB .NET, you could create a helper assembly using C++ or raw IL Assembler to be used by your .NET components. This helper assembly could contain a method that invokes on an IntPtr instance as a function pointer with the appropriate signature. However, thanks to Reflection Emit technology, any .NET language can dynamically create an assembly with a helper method that performs the necessary calli instruction, then simply invoke that. Listing 19.3 demonstrates how this advanced technique can be done in C#.
Listing 19.3. Invoking an Unmanaged Function Pointer via a Dynamic Method Containing Hand-Crafted MSIL
Reflection Emit is beyond the scope of this book, but we’ll examine the highlights of this listing. Lines 19–20 call the PInvoke method that returns a function pointer as an IntPtr type. This example calls GetProcAddress for CountClipboardFormats in USER32.DLL, so the function pointer returned matches the signature of the CountClipboardFormats API. This function has no parameters and returns a 32-bit integer. Lines 24–25 call our InvokeFunctionWithNoParams helper method, which is designed to handle any function with no parameters and a non-void return type. The returned integer, printed at the console in Line 29, is the value returned from the function being pointed to.
InvokeFunctionWithNoParams is defined in Lines 35–73. Lines 39–45 perform the standard action of creating a dynamic assembly containing a dynamic module. In this example, we throw away the dynamic assembly after invoking its method, but if we wanted to save it, then Line 43 would need to use AssemblyBuilderAccess.RunAndSave rather than AssemblyBuilderAccess.Run. Rather than defining a dynamic type to contain our dynamic method, Lines 48–50 define a static method directly on the module called DoTheDirtyWork.
This method will call the function pointer and return whatever value it returns. Therefore, the input returntype parameter is passed to DefineGlobalMethod to ensure that the return type of the dynamic method matches the return type that corresponds to the function pointer.
Lines 53–65 generate fill the dynamic method with the necessary MSIL instructions to perform the calli operation. By checking the static IntPtr.Size property, Lines 55–60 ensure that the generated code works on 32-bit and 64-bit platforms. Once the dynamic method is completed (Line 68), Lines 71–72 invoke the method and return the result.
Of course, this listing demonstrates an unrealistic use of invoking on a function pointer. If you want to call the Win32 CountClipboardFormats API in C#, you should use the standard PInvoke mechanism to call it directly rather than calling LoadLibrary and GetProcAddress yourself.
Passing Structures
Signatures of static entry points frequently have structure parameters, so using PInvoke often involves not just defining a function signature, but defining several structs as well. For the first example of using a PInvoke signature with a structure parameter, let’s look at the GetConsoleScreenBufferInfo Win32 function, which provides information about the console’s current settings (such as its background and foreground colors). It has the following unmanaged signature:
BOOL GetConsoleScreenBufferInfo(
HANDLE hConsoleOutput, // screen buffer handle
PCONSOLE_SCREEN_BUFFER_INFO lpConsoleScreenBufferInfo // screen buffer info
);
PCONSOLE_SCREEN_BUFFER_INFO represents a pointer to a CONSOLE_SCREEN_BUFFER_INFO structure, defined as follows:
typedef struct _CONSOLE_SCREEN_BUFFER_INFO {
COORD dwSize;
COORD dwCursorPosition;
WORD wAttributes;
SMALL_RECT srWindow;
COORD dwMaximumWindowSize;
} CONSOLE_SCREEN_BUFFER_INFO;
This structure has two fields that are structures themselves—SMALL_RECT:
typedef struct _SMALL_RECT {
SHORT Left;
SHORT Top;
SHORT Right;
SHORT Bottom;
} SMALL_RECT;
typedef struct _COORD {
SHORT X;
SHORT Y;
} COORD;
Listing 19.4 defines a PInvoke signature for GetConsoleScreenBufferInfo and value types representing these three structures, as part of my favorite example in the entire book. Using PInvoke, this C# listing defines a class for printing text with different foreground and background colors. Rather than just using Win32 APIs as the previous examples do, this listing defines a reusable class that can be easily used by other .NET applications.
Listing 19.4. The ColorfulConsole Class Wraps Win32 APIs for Printing in Color
Lines 6–27 define the CONSOLE_SCREEN_BUFFER_INFO, COORD, and SMALL_RECT value types, which are straightforward translations of their corresponding unmanaged structs. The CONSOLE_SCREEN_BUFFER_INFO type could have been defined without requiring the definitions of additional structs by expanding the fields of the sub-structs; for example:
internal struct CONSOLE_SCREEN_BUFFER_INFO
{
// Fields of COORD:
internal short dwSizeX;
internal short dwSizeY;
// Fields of COORD:
internal short dwCursorPositionX;
internal short dwCursorPositionY;
// A simple unsigned 16-bit integer:
internal ushort wAttributes;
// Fields of SMALL_RECT:
internal short srWindowLeft;
internal short srWindowTop;
internal short srWindowRight;
internal short srWindowBottom;
// Fields of COORD:
internal short dwMaximumWindowSizeX;
internal short dwMaximumWindowSizeY;
}
Either way, the memory layout is the same. Furthermore, because wAttributes is the only field of CONSOLE_SCREEN_BUFFER_INFO used by this listing, the struct could be further simplified as follows:
internal struct CONSOLE_SCREEN_BUFFER_INFO
{
internal long fourUnusedShortFields;
internal ushort wAttributes;
internal long fourMoreUnusedShortFields;
internal int twoUnusedShortFields;
}
As long as wAttributes is at the same memory offset from the beginning of the structure, using it works the same way in Listing 19.4. Still, it’s best to stay as faithful to the unmanaged struct definitions as possible.
Lines 30–51 define a Constants class containing relevant Win32 constants, marked internal so it’s not exposed to users. Lines 58–69 define three PInvoke methods: GetConsoleScreenBufferInfo discussed earlier, GetStdHandle to get the standard output handle, and SetConsoleTextAttribute to set the colors of text that we’ll output. ColorfulConsole exposes two public static WriteLine methods. The first one can be called with a string, a foreground color and a background color. The second one can be called with a string and a foreground color, and uses the console’s current background color for the text’s background.
These WriteLine methods use the public ForegroundColors and BackgroundColors enumerations we define in Lines 123–174. The colors in a console can be set to 16 different colors, but Windows defines just four color constants that need to be combined as bit flags in order to get the 16 colors. (There are separate values for foreground colors and background colors because both get bitwise-ORed in a 16-bit value.) Because these are sometimes unintuitive to use, our enumerations expose all the combinations with descriptive names like Cyan, Brown, LightGreen, and so forth. These enumeration values are not meant to be combined, so we don’t mark the enums with System.FlagsAttribute.
In Lines 77–87, the first overload of WriteLine calls GetStdHandle to get the standard output handle, then calls GetConsoleScreenBufferInfo to retrieve the existing console settings. Lines 90–99 call SetConsoleTextAttribute to set the console’s colors, Console.WriteLine to write the text, then SetConsoleTextAttribute with the structure passed back from GetConsoleScreenBufferInfo to restore the console to its old state. Notice that in Line 91, the two enumeration values are bitwise-ORed to obtain the single 16-bit value required by SetConsoleTextAttribute. This way, all of the bit flags treatment is hidden from the user.
The second overload of WriteLine (Lines 104–120) figures out the current console background color, then calls the first overload of WriteLine using this color. After calling GetConsoleScreenBufferInfo in Line 112, Line 116 extracts the background color from the CONSOLE_SCREEN_BUFFER_INFO structure’s wAttributes field by bitwise-ANDing it with a mask of 0xF0, and then casting the value to an instance of the BackgroundColors enumeration.
A .NET application can use this ColorfulConsole class as follows:
Tip
Listing 19.4 compiles with the following warnings:
Yet attempting to assign initial values to these struct members to silence the warnings results in compilation errors explaining that structs cannot have instance field initializers.
This is a common problem when using PInvoke in C# because it’s common to have non-public structs with fields that appear to be unused, but are actually read and written to by unmanaged code. There are a few things you could do to prevent these warnings:
• Use these fields somewhere in the class in a trivial way, simply to silence the warnings.
• Define structs as formatted classes instead, so you can initialize the field values. This can’t always be done, however, as discussed in the “Using Formatted Classes” section later in this chapter.
• Change the structs and their fields to be public, if you don’t mind exposing them to others.
• Define only the fields you use in managed code, but use StructLayoutAttribute and FieldOffsetAttribute to align them appropriately, discussed in the “Using FieldOffsetAttribute” section later in this chapter.
Do not make a struct’s fields static in order to initialize them. Static fields are not part of the instance passed to a PInvoke method. Passing the wrong-sized structure to unmanaged code can cause your application to crash pretty quickly.
Customizing Structures with Custom Attributes
The structures used in Listing 19.4 were simple enough to not require any custom attributes, but often customizations are required to make .NET type definitions match the expected memory layout of the unmanaged structures they need to represent. Struct customization can be done with three pseudo-custom attributes:
• StructLayoutAttribute
• FieldOffsetAttribute
• MarshalAsAttribute
Using StructLayoutAttribute
StructLayoutAttribute requires that you specify a member of the LayoutKind enumeration. This can be one of three values:
• Auto—The CLR chooses how to arrange the fields in the structure, and could rearrange them at any time. This should never be used on a type that is marshaled to unmanaged code as a structure.
• Sequential—The fields are arranged sequentially, in the order they appear in source code. This is typically the desired setting for types marshaled to unmanaged code as structures.
• Explicit—The fields are arranged using byte offsets specified by the user on each field using a second custom attribute: FieldOffsetAttribute. This gives you complete control over the layout of a struct, and even makes it possible to define a union (by giving every field an offset of zero).
Although LayoutKind.Auto is the CLR’s default for all types not marked with the StructLayoutAttribute pseudo-custom attribute, the Visual C# .NET, Visual Basic .NET, and Visual C++ .NET compilers all mark value types with LayoutKind.Sequential by default. This is why the value types in Listing 19.4 did not require StructLayoutAttribute with the LayoutKind.Sequential attribute; the compiler implicitly marks it as such.
StructLayoutAttribute has the following three optional named parameters:
• CharSet—Controls the default character set for string and character fields. CharSet can be set to a value of the same CharSet enumeration used with DllImportAttribute. With CharSet.Ansi, all string and character fields that don’t explicitly choose a character set (with MarshalAsAttribute) are marshaled as ANSI. With CharSet.Unicode, all string and character fields that don’t explicitly choose a character set are marshaled as Unicode. With CharSet.Auto, all string and character fields that don’t explicitly choose a character set are marshaled as either ANSI or Unicode, depending on the operating system. The Visual C# .NET, Visual Basic .NET, and Visual C++ .NET compilers all use CharSet.Ansi by default.
• Pack—Controls the struct’s packing alignment when using LayoutKind.Sequential. Pack can be set to one of the following values (representing a number of bytes): 0, 1, 2, 4, 8, 16, 32, 64, or 128. The default is 8, and 0 indicates that the default packing alignment for the current operating system should be used.
• Size—Controls the total size of a value type. Size can be set to any number of bytes greater than or equal to the size that the value type would be without any Size customization. Therefore, Size can be used to increase the size of a structure.
The packing of a struct is often a source of confusion. Without packing in the equation, fields align on their natural boundary. A natural boundary is a byte offset that is a multiple of the field’s size. So byte fields can begin anywhere, 2-byte short fields can begin on even byte boundaries, 4-byte integer fields can begin on boundaries that are multiples of 4, and so on. This means that if you had a structure with a byte field followed by an integer field, the byte would begin at offset 0, and the integer would begin at offset 4 (leaving empty space at bytes 1, 2, and 3), so the total size of the structure would be 8 bytes.
With packing in the equation, fields align on either their natural boundary or to the pack size—whichever results in the smaller offset. This is pictured in Figure 19.1.
Figure 19.1. The memory layout of structures can be controlled with the Pack named parameter.
This figure contains two different structures, each set to three different packing alignments. Struct A corresponds to the following (in C#), where n could be 2, 4, or 8:
[StructLayout(LayoutKind.Sequential, Pack=n)]
public struct A
{
public int One;
public double Two;
public int Three;
}
Struct B corresponds to the following (pictured in VB .NET), where n could be 2, 4, or 8:
<StructLayout(LayoutKind.Sequential, Pack:=n)> _
Public Structure B
Public One As Short
Public Two As Integer
Public Three As Short
Public Four As Integer
End Structure
Whether struct A has a packing alignment of 2 or 4 bytes, the memory layout is the same. In both cases, the end of every field touches a packing boundary (indicated with longer vertical lines), so each field resides immediately after the previous field. When it has a packing alignment of 8 bytes, it creates a 4-byte “hole” between the first and second field. That’s because the closest natural boundary for the double field resides at 8 bytes, which happens to coincide with the next packing boundary. Because all three fields reside at their natural boundaries in the 8-byte packing case, increasing the packing size would have no effect on the memory layout.
The memory layout of struct B, on the other hand, changes when the packing alignment changes from 2 bytes to 4 bytes. In the Pack = 4 case, the end of the first field stops short of the natural boundary for the second field, which also coincides with the packing boundary. The same thing occurs for the third and fourth fields. Because all fields reside at their natural boundaries when the packing is 4 bytes, changing the packing to 8 bytes changes nothing.
Tip
Ensure that any value types you define to use with PInvoke signatures have the appropriate memory layout to match the unmanaged structure definitions expected by the unmanaged functions. Fortunately, structs are usually arranged such that fields align to their natural boundaries without any gaps, so changing their packing alignment size has no effect.
The packing alignment affects the unmanaged layout of a value type, not necessarily the managed layout. If the value type is blittable, then StructLayoutAttribute happens to affect the managed layout, but all that StructLayoutAttribute is guaranteed to do is control the unmanaged representation of a type.
Using FieldOffsetAttribute
As mentioned in the previous section, FieldOffsetAttribute can and must be used with LayoutKind.Explicit in order to choose your own memory offsets for each field. It could be applied to the previously defined A and B structs as follows:
C#:
[StructLayout(LayoutKind.Explicit)]
public struct A
{
[FieldOffset(1)] public int One;
[FieldOffset(7)] public double Two;
[FieldOffset(15)] public int Three;
}
<StructLayout(LayoutKind.Explicit)> _
Public Structure B
<FieldOffset(0)> Public One As Short
<FieldOffset(0)> Public Two As Integer
<FieldOffset(0)> Public Three As Short
<FieldOffset(0)> Public Four As Integer
End Structure
The memory layout for these two structs is illustrated in Figure 19.2.
Figure 19.2. The memory layout of fields with explicit offsets.
Struct A uses unconventional offsets just for demonstration, and struct B is a more typical scenario: defining a union by overlapping fields at the same memory location. Only blittable fields are allowed to have overlapping offsets. Every field of a struct with explicit layout must be marked with FieldOffsetAttribute. Furthermore, the Pack named parameter should not be used with explicit layout because packing alignment doesn’t matter when you’re specifying exactly where you want each field to appear.
Tip
If you are creating a .NET definition of a structure with an embedded union, you should define the union as a separate structure and define a field of that union type in the original structure. That way, you can mark every union field with FieldOffset(0) and not bother with marking the fields of the main structure (unless it has unconventional memory layout that requires it). If you define it all as one big structure, then you must determine the proper memory offset of every field, which is error prone.
Using MarshalAsAttribute
The use of MarshalAsAttribute was covered in Chapter 12, “Customizing COM’s View of .NET Components,” including using it on structure fields. Here, we’ll summarize the few UnmanagedType values that are commonly used with MarshalAsAttribute on struct fields passed to PInvoke signatures:
• ByValArray—This is the only supported way to pass an array to unmanaged code as a struct field, rather than LPArray or SafeArray. This must be used with MarshalAsAttribute’s SizeConst parameter to indicate how many elements are in the array. If the array instance’s length is smaller than SizeConst, an exception is thrown. If it is larger, only SizeConst elements are copied.
• ByValTStr—Used on a string field to make it a by-value buffer (such as CHAR[128]). Used with MarshalAsAttribute’s SizeConst parameter to indicate how many elements are in the character array. If a .NET string has at least SizeConst characters, only SizeConst-1 characters are copied followed by a null character.
• Interface—Used on an object or class field to make it marshal as an interface. This is often required because the default void* marshaling (UnmanagedType.AsAny) for class types causes an exception to be thrown.
• LPStr, LPTStr, LPWStr—Used to customize the character set of string fields that are not passed as by-value buffers.
• U1, U2—Used to customize the character set of a System.Char field.
By default, strings and characters (System.String and System.Char) inherit their character set from the structure’s CharSet marking in StructLayoutAttribute, so the default marshaling is ANSI in C#, VB .NET, and C++. The ByValTStr and LPTStr values behave a little differently than you might expect, because these settings do not mean that the character set is determined at run time. Instead, it means that the struct’s character set is used for such fields. Therefore, fields marked with ByValTStr or LPTStr only have a platform-dependent character set when the struct is marked with CharSet.Auto. Because the UnmanagedType enumeration has no Char or WChar values, U1 can be used for a 1-byte ANSI character, and U2 can be used for a 2-byte Unicode character.
Special attention must be paid to by-value buffers passed as fields. Unlike the case for parameters, the Interop Marshaler does not enable passing StringBuilder types as structure fields! Instead, a string buffer field can be defined one of two ways:
• System.String with MarshalAs(UnmanagedType.ByValTStr, SizeConst=bufferSize+1)
• A character array with MarshalAs(UnmanagedType.ByValArray, SizeConst=bufferSize)
In both cases, the character set must be controlled by the containing structure. In the System.String case, the string instance must be initialized to a large enough string before being passed to unmanaged code (similar to using System.String for a buffer in VB .NET, discussed in the previous chapter). Internally, the CLR makes a copy of the string, passes it to unmanaged code, and replaces the original string reference with the new string potentially altered by unmanaged code. Therefore, this does not violate the rule that .NET strings are immutable; the string reference is silently swapped to a new string object.
Caution
When the Interop Marshaler passes a System.String field as a by-value buffer (ByValTStr), it copies the first SizeConst-1 characters and places a null character at the end. Because of this, you can’t pass a 128-character string as a field marked with SizeConst=128, but are limited to a 127-character string. If the memory layout of the structure is such that the byte following the string is unused (but still a part of the structure), you could extend the string by defining it with SizeConst=129. But if you’re constrained by the layout of the unmanaged structure whose definition you’re attempting to match, you must instead define the field as a character array marked with ByValArray and SizeConst=128. For character arrays, you can successfully fill every element with a valid character.
Listing 19.5 demonstrates both techniques of using a character buffer field with the GetVersionEx Windows API, with the following unmanaged definition:
BOOL GetVersionEx(LPOSVERSIONINFO lpVersionInfo);
LPOSVERSIONINFO is a pointer to an OSVERSIONINFO struct, which looks like the following:
typedef struct _OSVERSIONINFO{
DWORD dwOSVersionInfoSize;
DWORD dwMajorVersion;
DWORD dwMinorVersion;
DWORD dwBuildNumber;
DWORD dwPlatformId;
TCHAR szCSDVersion[128];
} OSVERSIONINFO;
Listing 19.5. Using Strings and Character Arrays to Represent Character Buffer Fields of Structures
Lines 4–14 define one version of the OSVERSIONINFO structure, which uses a string marked with UnmanagedType.ByValTStr to represent the buffer. To truly make it a TSTR buffer, the structure is marked with CharSet.Auto. (That’s the only reason that StructLayoutAttribute is used because LayoutKind.Sequential is the default for C# structs.) To differentiate between the two OSVERSIONINFO types we’re defining, we name this OSVERSIONINFO_1. The name has no effect on the proper operation with unmanaged code; all that matters is the memory layout. Both GetVersionEx methods are marked with CharSet.Auto to match the marking on the structures. A mismatch in the character set between function and structs passed to it can cause errors.
Lines 16–26 define an OSVERSIONINFO_2 structure that’s identical to OSVERSIONINFO_1 except that a character array marked with UnmanagedType.ByValArray is used to represent the character buffer. Lines 30–34 define GetVersionEx twice as overloaded methods, with each one accepting a different value type parameter. After the OSVERSIONINFO_1 instance is created in Line 38, Line 39 sets its dwOSVersionInfoSize field to the size of the structure using Marshal.SizeOf. The documentation for GetVersionEx states that this should be done before calling the method, and is typical for Win32 APIs.
Tip
Use Marshal.SizeOf in the System.Runtime.InteropServices namespace to determine the unmanaged size of a struct. This may be different than its managed size due to the marshaling of non-blittable fields, but it’s precisely the size required by the unmanaged function. A struct’s size often needs to be passed as a parameter to Win32 APIs.
Because value types can contain methods, the call to Marshal.SizeOf can be wrapped inside a simple method, as follows:
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Auto)] internal struct OSVERSIONINFO{ internal void Init() { dwOSVersionInfoSize = Marshal.SizeOf(this); } internal int dwOSVersionInfoSize; internal int dwMajorVersion; internal int dwMinorVersion; internal int dwBuildNumber; internal int dwPlatformId; [MarshalAs(UnmanagedType.ByValTStr, SizeConst=128)] internal string szCSDVersion;}
Because the OSVERSIONINFO_1.szCSDVersion string is being used as if it’s a character buffer, it must be initialized to the appropriate number of characters (128). This is done in Line 40 with one of System.String’s constructors. Remember that only 127 characters are usable due to the trailing null character used by the Interop Marshaler, but in this example we expect the string returned to be much shorter. Line 42 makes the call to GetVersionEx and Line 44 prints the string after the call.
The code in Lines 38–45 is duplicated in Lines 47–55 but uses OSVERSIONINFO_2 instead. Line 49 creates an array of 128 null characters, and Line 51 calls the GetVersionEx overload that works with the OSVERSIONINFO_2 structure. To extract the string after the call, Line 53 uses a System.String constructor that creates a string with the contents of a character array. String.Trim must be called on the new string to trim all the trailing null characters.
GetVersionEx places text in the szCSDVersion buffer only when a Windows service pack is installed. It returns a string such as “Service Pack 1.” If you run this listing on a computer without a service pack, you’ll see an empty string returned from both method calls.
Caution
Although defining a by-value System.String type to represent a character buffer can be done successfully for struct fields, it should never be done for a parameter (with the exception of signatures defined in VB .NET, due to the implicit transformation done with UnmanagedType.VBByRefStr). Defining such parameters can provide random results caused by string interning or strings being allocated in read-only memory. .NET strings are immutable and the CLR takes advantage of this fact, so you should never use the power of unmanaged code to overwrite a string’s memory.
Tip
Calling Marshal.SizeOf is a quick way to see if a value type you defined has an unmarshalable definition. For example, attempting to call it with a struct that has a StringBuilder field or a struct that uses ByValArray without setting the corresponding SizeConst value causes an ArgumentException with the message, “Type TypeName can not be marshaled as an unmanaged structure; no meaningful size or offset can be computed.”
Using Formatted Classes
StructLayoutAttribute can be marked on a class with LayoutKind.Sequential or LayoutKind.Explicit to enable marshaling to unmanaged code as a struct rather than a reference type. Using OSVERSIONINFO as an example, it could be defined as follows in C#:
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Auto)]
internal class OSVERSIONINFO
{
internal int dwOSVersionInfoSize;
internal int dwMajorVersion;
internal int dwMinorVersion;
internal int dwBuildNumber;
internal int dwPlatformId;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst=128)]
internal string szCSDVersion;
}
<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Auto)> _
Friend Class OSVERSIONINFO
Friend dwOSVersionInfoSize As Integer
Friend dwMajorVersion As Integer
Friend dwMinorVersion As Integer
Friend dwBuildNumber As Integer
Friend dwPlatformId As Integer
<MarshalAs(UnmanagedType.ByValTStr, SizeConst:=128)> _
Friend szCSDVersion As String
End Class
Tip
When defining a structure as a formatted class, you can define a default constructor to initialize its size field. For example, OSVERSIONINFO could be defined as follows:
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Auto)] internal struct OSVERSIONINFO{ internal OSVERSIONINFO() { dwOSVersionInfoSize = Marshal.SizeOf(this); } internal int dwOSVersionInfoSize; internal int dwMajorVersion; internal int dwMinorVersion; internal int dwBuildNumber; internal int dwPlatformId; [MarshalAs(UnmanagedType.ByValTStr, SizeConst=128)] internal string szCSDVersion;}
This is similar to the OSVERSIONINFO value type’s Init method from the previous section, but the client doesn’t need to remember to call it. Defining a separate method was done simply because value types cannot be defined with explicit default constructors.
The marshaling rules are the same as with a value type marshaled as a structure. Any methods, properties, or events are ignored by the Interop Marshaler; the instance fields (regardless of visibility) comprise the structure. The single difference between marshaling a value type as an unmanaged structure and marshaling a reference type as an unmanaged structure is that reference types have an implicit level of indirection that value types do not have. Table 19.1 summarizes how marshaling a type called MyStruct differs depending on whether it is defined as a value type or formatted reference type.
Table 19.1. Differing Levels of Indirection with Value Types Versus Formatted Classes
A formatted reference type passed by value should typically be marked with InAttribute and OutAttribute to ensure that data is copied back and forth.
Listing 19.6 is an update to Listing 19.5, using formatted classes instead of reference types. Notice that the PInvoke signatures pass the OSVERSIONINFO_1 and OSVERSIONINFO_2 instances by-value instead of by-reference. It’s crucial that these by-value parameters are marked as InAttribute and OutAttribute because reference types other than StringBuilder marshal as in-only parameters by default. This in-only behavior would not be noticed had the OSVERSIONINFO_1 and OSVERSIONINFO_2 types only had integer fields, but the string and character array fields make them non-blittable, forcing the copy to occur.
Listing 19.6. Using Formatted Classes To Represent Pointers to Structures
The main advantage of using formatted classes is that they give you the ability to pass null for what would otherwise be a by-reference value type parameter. You could even use a formatted class in place of a by-reference primitive type that could be null. For example, the GetWindowThreadProcessId function in USER32.DLL has the following unmanaged signature:
DWORD GetWindowThreadProcessId(HWND hWnd, LPDWORD lpdwProcessId);
This function returns the thread identifier for the thread that created the specified window, but also returns the window’s process identifier if a valid pointer is passed as the second parameter. A caller can pass null for the second parameter if it doesn’t care about the process identifier.
If we wanted to define a PInvoke signature such that a managed client could either pass a valid reference or null, we’d likely define the signature as either:
[DllImport("user32.dll")]
static extern uint GetWindowThreadProcessId(IntPtr hWnd, IntPtr lpdwProcessId);
or:
[DllImport("user32.dll")]
static extern uint GetWindowThreadProcessId(IntPtr hWnd, int [] lpdwProcessId);
This technique was discussed in Chapter 6, “Advanced Topics for Using COM Components.” The first technique is the most typical, and used throughout Appendix E, “PInvoke Definitions for Win32 Functions.” A third option would be to define a formatted class like the following:
[StructLayout(LayoutKind.Sequential)]
class RefInt32
{
public int Value;
}
Then you could define the signature for GetWindowThreadProcessId as follows:
[DllImport("user32.dll")]
static extern uint GetWindowThreadProcessId(
IntPtr hWnd, [In, Out] RefInt32 lpdwProcessId);
Using such a signature, you could pass null when you don’t want to pass a valid instance. The InAttribute and OutAttribute markings aren’t strictly necessary because the type is blittable and the Interop Marshaler doesn’t copy the instance during the call. It’s a good idea to include these attributes anyway, because the lack of copying for blittable types is really just an implementation detail of the Interop Marshaler.
Another option would be to define two overloads of the same PInvoke method—one that enables passing null as IntPtr.Zero and one that expects a valid instance:
// Call with IntPtr.Zero to pass null
[DllImport("user32.dll")]
static extern uint GetWindowThreadProcessId(IntPtr hWnd, IntPtr lpdwProcessId);
// Call with a valid integer value
[DllImport("user32.dll")]
static extern uint GetWindowThreadProcessId(
IntPtr hWnd, ref int lpdwProcessId);
It’s not always possible to use formatted classes with PInvoke, however. For example, consider the localtime API from MSVCRT.DLL:
tm* localtime(const time_t* timer);
The tm type is a struct that could be defined in C# as follows:
public struct tm
{
public int tm_sec; // seconds after the minute
public int tm_min; // minutes after the hour
public int tm_hour; // hours since midnight
public int tm_mday; // day of the month
public int tm_mon; // months since January
public int tm_year; // years since 1900
public int tm_wday; // days since Sunday
public int tm_yday; // days since January 1
public int tm_isdst; // daylight savings time flag
}
Because the localtime function returns a pointer to a struct, and because you can’t add a level of indirection to a managed return type, the PInvoke signature for localtime must be defined as follows:
[DllImport("msvcrt.dll")]
static extern IntPtr localtime(ref int timer);
To avoid using IntPtr, you might be tempted to define tm as a formatted class so you can simply return a tm instance from localtime. This does not work, however. The reason is that when a pointer to a struct is returned, the Interop Marshaler copies the data to the managed object then frees the unmanaged struct using the CoTaskMemFree API. But according to localtime’s documentation, it returns a pointer to a statically allocated tm structure that should not be freed by the user. Therefore, attempting to define and use tm as a formatted class causes heap corruption.
Tip
Whenever an unmanaged API allocates memory that it doesn’t want the user to free using CoTaskMemFree, either the IntPtr type or C# unsafe code must be used.
The Structure Inspector
When defining a value type or formatted class to be marshaled as an unmanaged structure, it’s often handy to view the type’s unmanaged memory representation. By doing so, you can ensure that all the fields reside where they should be and that they contain the data you expect.
Caution
Structures are not deep-marshaled by the Interop Marshaler, so fields of formatted classes, by-reference arrays, or pointers to structures are not supported. Instead, such fields must be defined as IntPtr for do-it-yourself marshaling. In addition, structure fields that are by-value arrays of other structures are not supported in version 1.0 of the .NET Framework.
Listing 19.7 defines a C# class with a single static method that can be used to print out the unmanaged memory representation of any value type or formatted class. By using methods of the System.Runtime.InteropServices.Marshal class (most notably, StructureToPtr), it prints out the structure’s contents in the following format:
Total Bytes = 16
01 00 00 00 _...
01 00 44 00 _.D.
01 00 64 00 _.d.
64 00 00 00 d...
After printing the total number of bytes in the marshaled unmanaged structure, it displays the hexadecimal values on the left and each byte’s character representation on the right. This class can be used as a simple debugging aid, without having to write unmanaged code in order to inspect the contents and layout of a struct after marshaling occurs.
Listing 19.7. MarshaledStructInspector.DisplayStruct Can Display the Unmanaged Memory Representation of Any Value Type or Formatted Class
Tip
A great way to debug any PInvoke failure is to define your own method in unmanaged C++ code with the same signature as the one you’re trying to call. Then temporarily switch your managed code to call your custom function instead, and inspect exactly how all data is presented to the unmanaged function.
Handling Variable-Length Structures and Signatures
Signatures used with PInvoke are static in nature. Any information in custom attributes, the parameters used in methods, and the fields used in structures, must be determined at compile time. Sometimes, however, unmanaged functions are dynamic in nature and require a variable number of parameters or variable-length structures. Fortunately, there are ways to achieve this behavior with PInvoke.
A Win32 security identifier (SID) is one example of a variable-length structure that must sometimes be passed to unmanaged functions. It is defined as follows in winnt.h:
typedef struct _SID {
BYTE Revision;
BYTE SubAuthorityCount;
SID_IDENTIFIER_AUTHORITY IdentifierAuthority;
[size_is(SubAuthorityCount)] DWORD SubAuthority[*];
} SID, *PISID;
The array field is problematic because there is no way to express such an array in a .NET structure. MarshalAsAttribute’s SizeParamIndex named parameter only works for parameters, not fields.
Listing 19.8 demonstrates one way to handle this variable-length structure when defining ConvertStringSidToSid, which is defined as follows in sddl.h:
BOOL ConvertStringSidToSid(LPCTSTR StringSid, PSID *Sid);
Listing 19.8. Handling Variable-Length Structures Using Multiple Definitions
In this listing, the SID structure is defined three times—as SID1, SID16, SID256, depending on the length chosen for the SubAuthority array. (The Value field is also defined as an array because SID_IDENTIFIER_AUTHORITY is simply a six-element byte array.) Because ConvertStringSidToSid defines its SID parameter as IntPtr, the caller of the method can choose which version of the SID structure to use to ensure that it has enough elements in the array. Before the call, Marshal.StructureToPtr can be used to convert either SID1, SID16, or SID256 to an IntPtr. After the call, Marshal.PtrToStructure can be used to convert it back. It turns out that defining the SID parameter as IntPtr is a requirement anyway (even if the structure were a fixed size) because it represents a pointer to a pointer to a SID structure that must be freed with the LocalFree API.
Tip
If you don’t want to define multiple copies of a structure to simulate a variable-sized array, you could define a pointer-to-structure parameter as a byte array instead. Then, the .NET caller could dynamically allocate an array of the exact size desired, fill its elements appropriately with the structure’s raw data, and pass it to unmanaged code. As long as the unmanaged function sees the expected memory contents, it doesn’t matter how you represent it on the .NET side.
That covers variable-sized structures, but what about methods that accept a variable number of arguments? You can define several overloaded methods that cover all possible ways in which you want to call the method. Using wsprintf from USER32.DLL as an example, you could define four overloads as follows:
With these four signatures, a user could call wsprintf with two, three, four, or five parameters. Make sure that the calling convention for any such methods is set to CallingConvention. Cdecl, because the caller must pop off the appropriate number of parameters in every case.
Using C# Unsafe Code
Using C# unsafe code, introduced in Chapter 6, in combination with PInvoke enables you to avoid complex Interop Marshaling while, at the same time, using data types more descriptive than IntPtr. Although avoiding complicated marshaling can improve performance, a big benefit of using C# unsafe code in PInvoke signatures is that you don’t have to figure out the rules of the Interop Marshaler when defining methods! Instead, the PInvoke signature can be a direct translation of the unmanaged signature (once you figure out what any C++ typedefs used as parameters really represent).
To demonstrate this technique, let’s look at the GetPrivateProfileSectionNames API used in the previous chapter. The unmanaged signature looks like the following:
DWORD GetPrivateProfileSectionNames(LPTSTR lpszReturnBuffer, DWORD nSize,
LPCTSTR lpFileName);
When we defined a PInvoke signature for this function in the previous chapter, we needed to understand how both string parameters are used to determine whether to define them as String, StringBuilder, or IntPtr types. If we use C# unsafe code instead, the unmanaged signature gives us all the information we need to define it in C#. We could define it as:
[DllImport("kernel32.dll")]
static unsafe extern int GetPrivateProfileSectionNamesA(
sbyte* lpszReturnBuffer, int nSize, sbyte* lpFileName);
or:
[DllImport("kernel32.dll")]
static unsafe extern int GetPrivateProfileSectionNamesW(
char* lpszReturnBuffer, int nSize, char* lpFileName);
With these signatures, the Interop Marshaler simply passes the pointer values for the first and third parameters, so any additional work is left for the users of these methods. No InAttribute, OutAttribute, or MarshalAsAttribute needs to be (or can be) applied to pointer parameters because the Interop Marshaler effectively treats them as by-value IntPtr parameters. Notice that we must choose either the ANSI or Unicode version of GetPrivateProfileSectionNames because the Interop Marshaler can only transform the character set of string parameters when they are defined as String or StringBuilder. In the ANSI version, sbyte is used to represent a one-byte character, and in the Unicode version, char is used to represent a two-byte Unicode character.
Caution
Although using C# unsafe code when defining a PInvoke method can look like a direct translation, don’t forget about differences between ANSI and Unicode. In C++, char* represents an ANSI string, but in C#, char* represents an unsafe Unicode string!
C# unsafe code makes it easy to define PInvoke methods without understanding their semantics, but calling them still requires such understanding, of course! In addition, callers of PInvoke signatures with unsafe data types have the burden of doing work that would ordinarily be done by the Interop Marshaler. However, this technique gives developers full control. It also opens the possibility of writing an application that can automatically produce correct PInvoke signatures by simply scanning unmanaged signatures (in a header file, for example). That’s because, unlike the case when you’re relying on Interop marshaling for non-blittable types, each unmanaged data type can be mechanically transformed into a compatible managed data type with no exceptions. Remember that C# unsafe code inherently is no more unsafe than using standard PInvoke; it’s just a convention enforced by the C# compiler because it’s easy for programmers to make mistakes when directly manipulating pointers.
Listing 19.9 updates Listing 18.6 from the previous chapter, using C# unsafe code instead.
Listing 19.9. Using C# Unsafe Code with PInvoke
Lines 6–8 define the unsafe Unicode version of GetPrivateProfileSectionNames. The ptr variable declared in Line 12 is treated like it was in Listing 18.6, except that it’s given the more specific char* type rather than IntPtr. The main difference between this listing and Listing 18.6 is in Line 21. Because the third parameter of GetPrivateProfileSectionNames is defined as char*, Line 21 pins a string and assigns it to a char* variable using C#’s fixed statement.
Because the first string parameter was allocated in unmanaged memory (Line 19), Line 26 creates a new .NET string with its contents (embedded nulls included) using an overloaded String constructor that accepts a char* parameter. Alternatively, Line 26 could have been the following to achieve the same result:
s = Marshal.PtrToStringUni((IntPtr)ptr, numChars-1);
Guarding Against Premature Garbage Collection
Great care is required when using PInvoke to ensure that garbage collection doesn’t interfere with the proper operation of your application. As described in the “Using Delegates as Function Pointers” section, unmanaged code is invisible to the .NET garbage collector, so it has no way to know when entities such as delegates might still be in use by unmanaged clients. Once a delegate is not being referenced by any managed code, it can be collected at any time.
Premature garbage collection mostly affects PInvoke rather than COM Interoperability because the CLR can take advantage of the standard reference counting used by COM components to manage object lifetime. When passing a .NET object to unmanaged code, the CLR increments its CCW reference count and ensures that it does not get collected until this reference count reaches zero (resulting from IUnknown.Release calls from unmanaged clients).
When passing a delegate to unmanaged code as a function pointer, however, the CLR does not keep it alive by default because there’s no standard way for unmanaged code to notify the CLR that it’s finished with the pointer. The CLR would have to keep delegates alive forever once they are passed to unmanaged code, and this is clearly not acceptable. Therefore, programmers must take explicit action to guard against premature collection when passing entities other than regular objects across the Interop boundary. (Using COM Interoperability often has the opposite problem of keeping objects alive too long!)
Premature garbage collection can affect more than just delegates, however. Using .NET classes that wrap Windows handles can suffer from the same problem, although in a more subtle way. Handles are another example of a communication mechanism that does not use reference counting on the objects they represent. Listing 19.10 demonstrates the problem by defining three classes:
• RegistryKey—A class that wraps an HKEY (a handle to a registry key)
• RegistryKeyEnumerator—A class that uses RegistryKey and enumerates subkeys for any registry key
• Client—A class that uses RegistryKeyEnumerator
See if you can identify the problem in this listing. The RegistryKey and related classes are just meant to demonstrate how premature garbage collection problems can manifest; if you were really interacting with the Windows Registry, you would use the Microsoft.Win32. RegistryKey class in the mscorlib assembly. This listing uses the ErrorHelper. GetErrorMessage function we defined in the previous chapter, so it should be compiled with ErrorHelper.cs from Listing 18.9.
Listing 19.10. A Demonstration of Incorrect Windows Handle Wrapping
The RegistryKey class, defined in Lines 6–39, contains a constructor that opens a registry key using the RegOpenKey API. It also contains a property that exposes the wrapped HKEY finalizer that closes the registry key using the RegCloseKey API.
The RegistryKeyEnumerator class, defined in Lines 41–84, is a simple enumerator that uses the RegEnumKey API to provide its data. Rather than defining and using RegOpenKey and RegCloseKey APIs to open and close a registry key directly, Line 56 creates a RegistryKey instance. For the HKEY that it needs to pass to RegEnumKey (represented as IntPtr), it extracts the value from the RegistryKey.Handle property in Line 57.
Finally, the Client class, in Lines 86–104, uses the RegistryKeyEnumerator class, and forces garbage collection and finalization of collected objects in Lines 97–98. This is done in order to make the program consistently fail. Without this determinism, failure could occur at seemingly random times depending on when and if garbage collection ever occurs before the program logic finishes.
So what’s the problem with this listing? Calling RegistryKeyEnumerator.MoveNext in Line 101 causes an ApplicationException with the message “The handle is invalid.” This exception is thrown in Line 77. That’s because the RegistryKey instance created in Line 56 is eligible for collection immediately after Line 57. At this point, there are no more references to the RegistryKey object, only to its handle. From the garbage collector’s perspective, there is no relationship between the value returned by the RegistryKey.Handle property. Therefore, when garbage collection occurs in Line 97, the RegistryKey instance is collected and finalized. The RegistryKey finalizer, defined in Lines 34–38, closes the registry key and therefore makes the handle stored in RegistryKeyEnumerator’s hKey field invalid before its enumeration functionality is used in Lines 100–102. The call to Reset in Line 100 isn’t affected because its implementation does a simple variable assignment, but the call to MoveNext causes the error as soon as it calls into unmanaged code. The RegEnumKey method detects that the handle no longer corresponds to an open registry key and returns an error code, which causes the exception to be thrown in Line 77.
To ensure that the RegistryKey instance is never collected until the program finishes using RegistryKeyEnumerator, a few simple modifications could be made. These are shown in Listing 19.11.
Listing 19.11. An Update to Listing 19.10 That Prevents Premature Garbage Collection
In this listing, the RegistryKey class has been changed to implement IDisposable. This is recommended to provide clients with a means of closing the HKEY without having to wait for garbage collection. Because RegistryKeyEnumerator now has a RegistryKey member (declared in Line 55), it also implements IDisposable and disposes the RegistryKey member when appropriate. Finally, the Client class now uses C#’s using construct so that RegistryKeyEnumerator.Dispose is implicitly called at the end of the listing. The listing now works as desired because the RegistryKey instance used by RegistryKeyEnumerator is a class member that won’t be eligible for garbage collection until RegistryKey itself is.
Although Listing 19.10 demonstrates premature garbage collection with the use of two classes, clients of a single handle-wrapping class can run into this problem, too. For example, imagine that we added a DeleteSubKey method to the RegistryKey class that calls the RegDeleteKey API:
public class RegistryKey : IDisposable
{
...
[DllImport("advapi32.dll", CharSet=CharSet.Auto)]
static extern int RegDeleteKey(IntPtr hKey, string lpSubKey);
public void Delete(string subKey)
{
// Delete the subkey
int result = RegDeleteKey(hKey, subKey);
if (result != 0)
throw new ApplicationException("Could not delete registry key: " +
ErrorHelper.GetErrorMessage(result));
}
}
A C# client might use this as follows:
public class Client
{
static readonly IntPtr HKEY_LOCAL_MACHINE =
new IntPtr(unchecked((int)0x80000002));
public static void Main()
{
RegistryKey key =
new RegistryKey(HKEY_LOCAL_MACHINE, "SOFTWARE\.NET and COM");
key.DeleteSubKey("TemporaryKey");
// The client forgets to call key.Dispose
}
}
The call to DeleteSubKey is the last time the RegistryKey instance is used, and the implementation of DeleteSubKey calls a PInvoke method with parameter types that don’t keep the object alive (a simple by-value IntPtr and string). Therefore, there’s a rare chance that the call to DeleteSubKey will cause an exception to be thrown. Right before the PInvoke call to RegDeleteKey, the garbage collector could free the RegistryKey instance because there’s no further use of it anywhere in the program. (Had this been passed to a PInvoke method, the instance would not be eligible for collection.) If this happens, and if the object’s finalizer runs before RegDeleteKey uses the HKEY passed to it, an ApplicationException would be thrown with the message, “The handle is invalid,” due to the finalizer’s call to RegCloseKey.
If the client had remembered to call Dispose after DeleteSubKey, this problem could not occur because the garbage collector would see that the instance is still being used after the call.
Tip
The pattern of implementing and using IDisposable for classes that wrap unmanaged resources (in other words, Windows handles) goes a long way in preventing premature garbage collection. The presence of a call to Dispose keeps an object alive at least until Dispose is called.
Although proper use of IDisposable can help prevent premature garbage collection, two simple mechanisms exist specifically to prevent premature garbage collection:
• The System.GC.KeepAlive method
• The System.Runtime.InteropServices.HandleRef value type
The System.GC.KeepAlive Method
The static KeepAlive method can be passed an instance that you don’t want to be collected prematurely. The passed-in instance is guaranteed to be ineligible for collection starting at the beginning of the method that contains the call to KeepAlive, and ending immediately after the call to KeepAlive. Therefore, this method is called after relevant PInvoke invocations, for example:
ObjectWrappingHandle obj = new ObjectWrappingHandle();
PInvokeMethod(obj.Handle);
GC.KeepAlive(obj); // Keep the object alive during the call to PInvokeMethod
Passing an object to any virtual method has the same characteristics as passing it to GC.KeepAlive, so using this method is simply a convention (and a convenience).
The System.Runtime.InteropServices.HandleRef Value Type
Remembering to use GC.KeepAlive in the appropriate places can be a difficult task, so the System.Runtime.InteropServices namespace defines a value type that can be used with PInvoke methods to prevent premature garbage collection in an easier fashion. This value type is called HandleRef, which has a constructor that takes two parameters—a System.Object representing the wrapper object that must be kept alive, and a System.IntPtr that represents the unmanaged handle.
When a HandleRef instance is passed to unmanaged code, the Interop Marshaler extracts the handle and passes only that to unmanaged code. At the same time, however, the marshaler guarantees that the object passed as the first parameter to HandleRef’s constructor is not collected for the duration of the call.
When defining a PInvoke signature that expects a handle, you should make the argument type a HandleRef rather than an IntPtr. This way, callers are forced to pass HandleRef and automatically avoid premature garbage collection without understanding this subtle issue. For the RegistryKey example, the PInvoke signatures could be defined as follows:
Whenever these are called inside the RegistryKey class, the caller can simply pass this plus the handle that should be passed to unmanaged code, for example:
Although the PInvoke signatures listed in Appendix E use IntPtr to represent handle parameters, you should seriously consider using HandleRef instead. Although using HandleRef is slightly slower, the gain in productivity by avoiding subtle bugs is well worth it.
Choosing the DLL Location or Name Dynamically
The PInvoke mechanism requires a static DLL name stored inside the DllImportAttribute pseudo-custom attribute. Because there’s no way to control the attribute’s contents at run time, this can be problematic for developers who wish to dynamically choose the location of a DLL.
Usually, the DLL name used with DllImportAttribute (or used with VB .NET’s Declare, which ends up emitting DllImportAttribute in metadata) does not include any path information. Such a DLL is located using the standard search strategy of LoadLibrary, such as in the current directory, the Windows system32 directory, or somewhere listed in the PATH environment variable. But what can you do if you want to specify the DLL at run time via a registry key, an environment variable other than PATH, or some other custom way? As long as you know the DLL name at compile time (in other words, only the location is dynamic), you can simply do the following:
1. Define your PInvoke signatures using the name of the desired DLL without a path. For example:
[DllImport("MyFunctions.dll")]
3. Before you call any of the PInvoke signatures for a given DLL, manually call the Win32 LoadLibrary API in KERNEL32.DLL using your dynamic path information:
// Get desired path from a custom location, such as the registry
string path = ...
// Call LoadLibrary with the desired path
LoadLibrary(path + "\MyFunctions.dll");
5. LoadLibrary can be called once you’ve defined a PInvoke signature for it, as done in Listing 19.3.
This works because once a DLL is loaded, the CLR uses it for all remaining PInvoke calls. Had LoadLibrary not been manually called in managed code, the CLR would call it with the exact string inside DllImportAttribute the first time a function in the DLL is called.
If you want everything about the loaded DLL to be dynamic, including its name, then your best choice (other than using C++) is to use Reflection Emit to dynamically define an entire PInvoke signature. This is similar to the use of Reflection Emit in Listing 19.3, and is demonstrated in Listing 19.12.
Listing 19.12. Dynamic PInvoke for When the DLL Name Is Not Known at Compile Time
Lines 10–11 show an example use of the DynamicPInvoke method defined in Lines 14–42—calling the MessageBeep API in USER32.DLL. The DynamicPInvoke method uses reflection emit much like Listing 19.3, creating a dynamic assembly with a dynamic module in Lines 18–24. Lines 28–32 call ModuleBuilder.DefinePInvokeMethod with all the information necessary to dynamically emit a PInvoke signature. Once CreateGlobalFunctions is called in Line 35, the global PInvoke signature is ready to be called. Line 38 obtains the MethodInfo instance for the method, and Line 41 invokes the method.
This DynamicPInvoke method is able to handle any number and types of parameters and any type of return value. It is a bit too simplistic to handle many PInvoke signatures, however, because it needs to provide a way of specifying by-reference parameters and attributes such as MarshalAsAttribute, InAttribute, and OutAttribute. All of this is possible with reflection emit; see the .NET Framework SDK documentation if you’re interested.
Example: Responding Immediately to Console Input
In managed code, a console program can call Console.Read or Console.ReadLine to respond to user input, but both calls wait for the user to hit the Enter key before giving control back to the user! An example of this behavior was seen in Listing 19.2. Developers often don’t want to wait for the Enter key but rather want to respond to key presses immediately. No methods in the .NET Framework provide this behavior, but Listing 19.13 accomplishes this task using PInvoke calls to the ReadConsoleInput API in KERNEL32.DLL.
Listing 19.13. Responding Immediately to Console Input Using ReadConsoleInput
Lines 4–52 define several structures that are used with the ReadConsoleInput method. For this example, it was only necessary to define INPUT_RECORD and KEY_EVENT_RECORD, but the remaining structs are defined as well so INPUT_RECORD can be defined completely.
Because ReadConsoleInput requires a handle to console input, Line 87 begins the Main method by making a PInvoke call to GetStdHandle. If that succeeds, Lines 96–109 continually call ReadConsoleInput and prints the character and corresponding virtual key code if it detects that a key was pressed.
Running this program produces output such as the following after pressing some keys:
Example: Clearing the Console Screen
Listing 19.14 demonstrates how to clear a console screen just like the CLS command in DOS. The System.Console class does not expose a way to do this.
Listing 19.14. Clearing the Console Screen Like the CLS DOS Command
Lines 4–25 define the necessary structures, and Lines 28–37 contain a Constants class with a few values used by the PInvoke methods. The five PInvoke methods used by this listing are defined in Lines 41–60.
Whereas the previous listing required an input handle, this listing requires an output handle, obtained in Line 64. Line 74 extracts the existing console settings, so Line 81 can fill the screen with spaces. Lines 85–86 ensure that the buffer’s attributes are the same as before, and finally Line 90 sets the cursor position to the (0,0) coordinate. This takes advantage of the fact that when the COORD instance was created in Line 71, its memory is automatically zeroed. Therefore, neither field needs to be explicitly set to zero.
Example: Using CoCreateInstanceEx to Activate Remote COM Objects
As promised in Chapter 6, the next example uses CoCreateInstanceEx to activate a COM object on a remote computer. CoCreateInstanceEx is defined as follows in objbase.h:
Listing 19.15 wraps a call to CoCreateInstanceEx inside a method called CreateRemoteInstance. This method exhibits similar behavior as calling Type.GetTypeFromProgID with the server name overload followed by Activator.CreateInstance, so this listing simply demonstrates how to accomplish the same task in PInvoke. The only compelling reason to use this method rather than Type.GetTypeFromProgID and Activator.CreateInstance is that the CLR calls CoGetClassObject instead of CoCreateInstanceEx, which does not behave appropriately on older versions of Windows, as described in Chapter 6. However, one could imagine defining a CreateRemoteInstance method that exposes more of CoCreateInstanceEx’s functionality in a user-friendly way, such as customizing authentication settings for the remote activation request.
Listing 19.15. Instantiating a Remote COM Object Using CoCreateInstanceEx
Lines 4–9 define the MULTI_QI structure. The pIID field must be defined as an IntPtr type because there’s no way to express a reference to a Guid value type as a structure field. Lines 11–18 define the COSERVERINFO structure, giving it a Unicode character set rather than marking the pwszName field with MarshalAs(UnmanagedType.LPWStr). The pAuthInfo field represents a pointer to a COAUTHINFO structure, which can be used to override the default activation security. Because we accept the default security in this listing, we don’t bother defining the COAUTHINFO structure. Even if we did define it, the pAuthInfo field would still require being defined as an IntPtr because it represents a pointer to a structure rather than an embedded structure.
In case you wanted to use COAUTHINFO in your application, here’s how it could be defined in C#:
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Unicode)]
internal struct COAUTHINFO
{
internal uint dwAuthnSvc;
internal uint dwAuthzSvc;
internal string pwszServerPrincName;
internal uint dwAuthnLevel;
internal uint dwImpersonationLevel;
internal IntPtr pAuthIdentityData;
internal uint dwCapabilities;
}
Line 22 defines the CLSCTX_REMOTE_SERVER used for the CoCreateInstanceEx call, and Lines 23–24 define a Guid instance representing the IID for IUnknown, which is needed to query the remote object for the IUnknown interface. Lines 26–36 define the two PInvoke signatures used for this listing. The definitions of both methods are straightforward except for the first parameter of CoCreateInstanceEx. The straightforward way of defining this parameter would be:
[In] ref Guid rclsid
However, UnmanagedType.LPStruct can be used in this specific case to convert a by-value Guid to [in] GUID* on the unmanaged side.
Tip
Other places in this book, including the back cover, state that UnmanagedType.LPStruct is not used in version 1.0 of the .NET Framework. As Listing 19.15 demonstrates, this is not quite true. In version 1.0, the one and only place it can be used is on a by-value Guid parameter in a PInvoke signature to add an implicit level of indirection. It can’t be used with other data types, and it can’t even be used on Guid fields; otherwise we could have avoided defining the pIID field of MULTI_QI as an IntPtr type.
Because UnmanagedType.LPStruct is often misused (due to its limited support and confusing name), it’s best to avoid using it altogether unless support is added in future versions of the CLR to make it useful on more than just Guid parameters.
The CreateRemoteInstance method is defined in Lines 38–76. The first thing it does is fill in a COSERVERINFO instance with the passed-in server name. The pAuthInfo field is set to null (IntPtr.Zero) in order to accept the default activation security. Line 50 calls CLSIDFromProgIDEx to extract the needed CLSID for the passed in ProgID from the local computer’s registry. Alternatively, the code could have called System.Type.GetTypeFromProgID and then extracted the CLSID from the returned type’s GUID property.
Line 53 creates a single-element MULTI_QI array that will hold the IID for IUnknown. MULTI_QI can be used to query for multiple interfaces in the same round-trip to the server, to optimize network performance. Because we need to pass a pointer to IID_IUnknown that is defined as an IntPtr, Line 60 pins the Guid instance using GCHandle and Line 63 calls AddrOfPinnedObject to set the pIID field to the appropriate address in memory. Lines 66–67 call CoCreateInstanceEx, and Line 75 extracts the returned interface pointer from the MULTI_QI structure. Because the pItf field was defined as a System.Object type, the Interop Marshaler automatically wraps the interface pointer in an RCW. Had pItf been defined as IntPtr instead, Line 75 would need to be changed to the following:
return Marshal.GetObjectForIUnknown(mqi[0].pItf);
Conclusion
PInvoke can be a tricky technology that takes lots of practice to get right. Hopefully this chapter, the previous one, and Appendix E give you all the information you need to dive into some “PInvoke challenges” with confidence.
Although certain types of failure during a PInvoke call do not occur in a predictable or easily-debuggable manner, there are several cases in which a descriptive exception is thrown. These cases mainly consist of limitations of the Interop Marshaler. For example, if you attempt to use UnmanagedType.AsAny to marshal a return type or by-reference parameter as void*, or if you attempt to return a System.Guid or System.Drawing.Color instance, you’ll get an exception (in version 1.0 of the CLR) that such a signature is not PInvoke compatible.
Tip
Don’t forget to consider security when writing .NET class libraries or other .NET applications that use PInvoke. Calling any function via PInvoke requires unmanaged code permission, so it’s safe by default. However, it’s often useful to carefully reduce security requirements to provide your functionality to clients that aren’t fully trusted. (In addition, reducing security with techniques such as using SuppressUnmanagedCodeSecurityAttribute can significantly boost performance.)
For example, the ColorfulConsole.WriteLine methods in Listing 19.4 could assert unmanaged code permission so partially trusted code (running from the Intranet zone, for instance) could still write to the console in multiple colors. Had the standard System.Console.WriteLine method required some sort of console-writing permission, we would want to demand this permission before asserting unmanaged code permission to be consistent with the .NET Framework and avoid exposing a potential security hole. See the discussion on security in Chapter 6 for more information.
One easy rule to follow is that you should never expose public PInvoke methods. Making them private to a class or internal to an assembly is a good defensive tactic to prevent the possibility of abuse by malicious code. The PInvoke signatures listed in Appendix E all omit the public keyword to make them private, which is the preferred visibility.