Category Archives: Debugging

Reply to HBGary — and personal notes.

One HBGary developper wrote a blogpost about windd entitled “Windd – Almost there, but not quite…“.
HBGary says *they* but I would like to say to readers that windd is a project that I developped and maintain alone, on my spare time.

More and more people are using windd so it looks I have to explain some things about the behavior of Windows Memory Manager and windd itself. Reading HBGary blogpost really made me feel enthusiastic because I recently complained about the lack of feedback on windd which is really frustrating when you lead a free project.

As you probably know early version of windd were open-source, but mainly because of the lack of feedback windd is no more open-source but still free. As far I have seen, open-source doesn’t mean people understand what you wrote or read the code, it is more like a philosophical aspect like “We have access to the source”. Anyway, people are still free to send me an e-mail, at matt/msuiche/net if they have questions about the internal behavior of Windd, to insult me or just to thank me.

Back to HBGary blogpost, we can read :

  • Is windd acquiring all of the available physical memory on the system?
  • Would a “raw format” image dump of a 64-bit vista machine load properly into HBGary’s Responder?
  • Should windd memory images that contain greater than 4GB of ram be considered admissible in court?
  • Was windd really the first tool to support physical memory acquisition on Windows 7? (as claimed by the author)

Author replied “No” to all questions above.

In this blogpost, I am going to provide details about the bug for both technical and no technical people by explain how windd works. The main problem with HBGary article is that they mix accessible physical memory and accessible physical memory address spaces. So I assume most people do not even know the difference between these two *things*.

In July 2008, Mark Russinovich wrote a very well explained article about Windows Physical Memory Manager entitled “Pushing the Limits of Windows: Physical Memory“. In this article Mark also used a tool written by Alex Ionescu (co-author of Windows Internals 5th) called “meminfo” to retrieve information related to Windows PFN Database.

In this article Mark explains what physical memory, physical address space and PFN Database are.
For lazy readers here is a short summary.


Figure above is the physical address space.
Red blocks are devices address space, blue blocks are physical memory and the “Inaccessible RAM” (only by Windows) is a reserved space in the physical memory for the Operating System to proceed to the translation from Virtual to Physical addresses.
Joanna Rutkowska paper entitled “Defeating Hardware Based RAM Acquisition” is a good reading if you want to know more about “Red blocks”.


Figure above comes from meminfo tool which display Memory Manager physical memory blocks. These entries describe how the physical memory is “splitted” in the physical address space which interpreted by the CPU. For the remind, DMA Access provides access to physical memory and not to the physical address space.

We firstly notice the highest physical page is 0x120000 (1179648) which correspond to size of the physical address space and NOT to the size of physical memory.
Secondly we notice physical addresses are above 4GB even if the machine has only 4GB installed. To retrieve the size of installed/detected physical memory we have to add the size of each block as follows:
(0x9F000 – 0x1000) + (0xDFE6D000 – 0x100000) + (0x120000000 – 0x100002000) = 0xFFE09000 (~4GB)

#1 Bug HBGary is talking about only concerns the RAW memory dump generation with windd (v1.3.0.x <= Version < v1.3.0.20091113 (fixed version)).
The bug was the following: Windd was reading blue blocks and wrote them directly in a raw dump file like the DMA-way, then it means red-blocks were missing. Impact was the PFN database was invalid which means Virtual to Physical address translation was impossible. BUT windd DOES acquiere all available physical memory. Present version of Windd produces a “CPU-like” memory dump (physical memory address space) and fills red blocks with null pages.

#2 Second question was about HBGary’s Responder. I don’t know this product and I never used it. But it would mean HBGary does not support DMA-style memory dump.

#3 For the third question, please refer to #1 and #2.

#4 Regarding the last question about the fact that I claimed that windd was the first tool support physical memory acquisition I do not remember saying that. I just remember I claimed several times windd was a great tool because it can produce Microsoft crash dumps which has great advantages mainly because of Windbg. Then, windd also aims at being used by troubleshooters and/or kernel developpers and not only by forensics investigators.

For instance in your blogpost I can read “NOTE: HBGary’s Responder does not yet fully support the automatic analysis of Windows 7 which is why HBGary had elected to not publicly advertise Windows 7 acquisition support” — The difference between windd and average memory acquisition tools is this point: This problem does not exist with WinDbg. Windbg supports Microsoft Crash Dump since Microsoft started to work on Windows.
Analysis is very important, if someone produces a dump and is unable to analyse it this is more or less like if someone would say “I have a new car but I do not know how to drive it”.

Thanks again to HBGary for helping me to improve windd utility and I hope they like the new version.

Demystifying new Windows 7 System Information Classes

This post is the first of a serie of articles/blogposts about new System Information Class under Windows 7 (32bits ATM) used by both NtQuerySystemInformation and extended version of this API called NtQuerySystemInformationEx introduced in Windows 7 and Windows 2008 R2.

First of all, here is the prototype of these functions.

NTSTATUS (WINAPI *NtQuerySystemInformationEx)(SYSTEM_INFORMATION_CLASS SystemInformationClass,
PULONG QueryType,
ULONG Alignment,
PVOID SystemInformation,
ULONG SystemInformationLength,
PULONG ReturnLength);

NTSTATUS (WINAPI *NtQuerySystemInformation)(SYSTEM_INFORMATION_CLASS SystemInformationClass,
PVOID SystemInformation,
ULONG SystemInformationLength,
PULONG ReturnLength);

As you can see there is two further arguments in NtQuerySystemInformationEx: QueryType and Alignment.

And here are new (and undocumented) system information classes added to Windows 7 that will be discussed in next blogposts.

typedef enum _SYSTEM_INFORMATION_CLASS
{
// NtQueryEx
SystemLogicalProcessorAndGroupInformation = 107,
SystemLogicalGroupInformation = 108,

SystemStoreInformation = 109,
SystemVhdBootInformation = 112,
SystemCpuQuotaInformation = 113,

// Removed in build 7100
SystemHardwareCountersInformation = 115, // uses KeQueryHardwareCounterConfiguration() instead

SystemLowPriorityInformation = 116,
SystemTpmBootEntropyInformation = 117,
SystemVerifierInformation = 118,

// NtQueryEx
SystemNumaNodesInformation = 121,
//
// Added in build 7100
//
SystemHalInformation = 122, // 8 bytes size
SystemCommittedMemoryInformation = 123,
MaxSystemInfoClass = 124
} SYSTEM_INFORMATION_CLASS, *PSYSTEM_INFORMATION_CLASS;

PS. For interested people the first issue of the Debugged! MZ/PE: MagaZine is available on Amazon.

Twitt This!

Edit: SYSTEM_INFORMATION_CLASS structure updated in build 7100.

Multi-Processors and KdVersionBlock

Tomorrow, I’ll publish a bugfix for win32dd about the following problem: on multi-processors computers a BSOD occurs when user try to generate a Microsoft Crash dump file through the -d option.

The problem is located inside KdGetDebuggerDataBlock function, when the function try to read KdVersionBlock field an invalid pointer is returned because this field is only valid in the 1st processor KPCR.

lkd> dt nt!_KPCR ffdff000
   +0x000 NtTib            : _NT_TIB
   +0x01c SelfPcr          : 0xffdff000 _KPCR
   +0x020 Prcb             : 0xffdff120 _KPRCB
   +0x024 Irql             : 0 ''
   +0x028 IRR              : 0
   +0x02c IrrActive        : 0
   +0x030 IDR              : 0xffffffff
   +0x034 KdVersionBlock   : 0x805562b8 
   +0x038 IDT              : 0x8003f400 _KIDTENTRY
   +0x03c GDT              : 0x8003f000 _KGDTENTRY
   +0x040 TSS              : 0x80042000 _KTSS
   +0x044 MajorVersion     : 1
   +0x046 MinorVersion     : 1
   +0x048 SetMember        : 1
   +0x04c StallScaleFactor : 0x6bb
   +0x050 DebugActive      : 0 ''
   +0x051 Number           : 0 ''
   +0x052 Spare0           : 0 ''
   +0x053 SecondLevelCacheAssociativity : 0x10 ''
   +0x054 VdmAlert         : 0
   +0x058 KernelReserved   : [14] 0
   +0x090 SecondLevelCacheSize : 0x80000
   +0x094 HalReserved      : [16] 0
   +0x0d4 InterruptMode    : 0
   +0x0d8 Spare1           : 0 ''
   +0x0dc KernelReserved2  : [17] 0
   +0x120 PrcbData         : _KPRCB
lkd> dt nt!_KPCR f9c2c000
   +0x000 NtTib            : _NT_TIB
   +0x01c SelfPcr          : 0xf9c2c000 _KPCR
   +0x020 Prcb             : 0xf9c2c120 _KPRCB
   +0x024 Irql             : 0 ''
   +0x028 IRR              : 0
   +0x02c IrrActive        : 0
   +0x030 IDR              : 0xffffffff
   +0x034 KdVersionBlock   : (null)
   +0x038 IDT              : 0xf9c30590 _KIDTENTRY
   +0x03c GDT              : 0xf9c30190 _KGDTENTRY
   +0x040 TSS              : 0xf9c2cd70 _KTSS
   +0x044 MajorVersion     : 1
   +0x046 MinorVersion     : 1
   +0x048 SetMember        : 2
   +0x04c StallScaleFactor : 0x650
   +0x050 DebugActive      : 0 ''
   +0x051 Number           : 0x1 ''
   +0x052 Spare0           : 0 ''
   +0x053 SecondLevelCacheAssociativity : 0x10 ''
   +0x054 VdmAlert         : 0
   +0x058 KernelReserved   : [14] 0
   +0x090 SecondLevelCacheSize : 0x80000
   +0x094 HalReserved      : [16] 1
   +0x0d4 InterruptMode    : 0
   +0x0d8 Spare1           : 0 ''
   +0x0dc KernelReserved2  : [17] 0
   +0x120 PrcbData         : _KPRCB

This piece of code is your friend if you are also experiencing some problem with it.

    //
    // Multi Processors (MP)
    // To ensure that it's running on a specific processor.
    //
    KeSetSystemAffinityThread(1);
    _asm {
       mov eax, fs:[0x1C]  // SelfPCR
       mov eax, [eax + 0x34] // KdVersionBlock
       mov KdVersionBlock, eax
   }
    //
    // Go back to default affinity.
    //
    KeRevertToUserAffinityThread();

MSDN documentation suggests to Windows Vista and later developpers to use KeSetSystemAffinityThreadEx instead of KeSetSystemAffinityThread and to use KeRevertToUserAffinityThreadEx instead of KeRevertToUserAffinityThread.

Even if there is no entry for KeRevertToUserAffinityThread inside the MSDN there is a blogpost from Windows Driver Kit (WDK) Documentation Blog about Windows Kernel Routine Name Conventions that says

Suffix – Ex – indicates that this is a new version of KeRevertToUserAffinityThread. “Ex” is an abbreviation for extension. It is a common Windows naming convention for new versions of a routine.

Thanks to Sebastien and Martim for reporting the bug.

Microsoft Crash Dump Analysis weaknesses.


Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /homepages/15/d187295720/htdocs/home/wp-content/plugins/deans_code_highlighter/geshi.php on line 2146

Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /homepages/15/d187295720/htdocs/home/wp-content/plugins/deans_code_highlighter/geshi.php on line 2146

Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /homepages/15/d187295720/htdocs/home/wp-content/plugins/deans_code_highlighter/geshi.php on line 2146

Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /homepages/15/d187295720/htdocs/home/wp-content/plugins/deans_code_highlighter/geshi.php on line 2146

Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /homepages/15/d187295720/htdocs/home/wp-content/plugins/deans_code_highlighter/geshi.php on line 2146

I’m going to discuss about Microsoft Crash Dump Analysis weaknesses, but in fact this blogpost is somehow an introduction to the next version of Win32DD 1.2. Indeed, the next version of win32dd will have crash dump generation implemented and some others things you’ll enjoy too.

Any reader who is interested in this topic is encouraged to refer to Andreas Schuster articles about the dmp file format, Mark Russinovich “Pushing the Limits of Windows: Physical Memory“, and Alex Ionescu’s excellent tool “meminfo“.

What’s the content of the full memory crash dump (*.dmp) files? In fact this is not a really a full memory snapshot. Only pages from MmPhysicalMemoryBlock ranges are copied.

For your information MmPhysicalMemoryBlock structure looks like that:

  1. span class=”co1″>// NumberOfPages * PAGE_SIZE is physical memory size.
  2.     PHYSICAL_MEMORY_RUN Run[1]; // NumberOfRuns is the total entries.

Here is how look the physical memory layout under Windows Vista 32bits with 1GB of RAM.
Only blue blocks are copied into the crashdump file while reading the MmPhysicalMemoryBlock, red blocks are reserved for devices, and others are not-reserved pages.


(Figure 1: Overview of Physical Memory Layout)

To identify these not-reserved pages I wrote a tool, available to download here (This version only works under Vista 32bits and above, with administrator privileges), to list drivers address ranges and system memory address ranges.


(Figure 2: Windows Vista 32-bits with 1GB of RAM)

In the sample above, there are 66 not-reserved pages (0,025%) including spare space at offset 0x00000000. (This page is well know from people who read Jonathan Brossard research about pre-boot password.). This page is not used during the O.S. is running, and the Run[0].BasePage is always equal to 1 (=0x00001000). It means there is at least one “not reserved” page with a fixed address which won’t be in the full crash dump memory during the BSOD process.

These pages could be used to hide code from crash dump without any hooks or structures modification. Full memory snapshot with classical tool can be a solution, but you no more benefit of the powerful crash dump analysis WinDbg’s feature. Win32dd 1.2 will take care of this problem to provide to forensics investigators a most powerful memory imager.

Few words about Microsoft interoperability initiative.

As you probably know, Microsoft released last month several thousands pages of documentation about office file format and Windows protocols.

It means numerous hundreds(thousands?) of functions/algorithms documentation and pseudo-code. But, are these pseudo-function right? It looks not.

While I was reading [MS-DRSR]: Directory Replication Service (DRS) Remote Protocol Specification, I was a bit curious to see the DecompressWin2k3() function (Thanks Aaron, Stefan , and Brendan).

This function is in fact the decompression algorithm called Xpress implemented for the first time in Windows XP and not Windows 2003 as say the name. Xpress algorithm works on 64kb chunks and is used in Windows hibernation file format, Windows Imaging Format (WIM) , Outlook, Exchange, and… LDAP replication service of Active Directory.

I’m quoting below, errors I found in the pseudo-code of DecompressWin2k3(). Here is the C implementation.

while (outputIndex < = outputSize) 
should be:
while (outputIndex < outputSize) 
    if (indicatorBit = 0) then 
        indicatorBit := copy inputBuffer[inputIndex] as 32-bit integer in 
        little-endian format 
        should be:
        indicator := copy inputBuffer[inputIndex] as 32-bit integer in 
        little-endian format 
        inputIndex := inputIndex + 4 
        indicatorBit := 32 
    endif 

    indicatorBit := indicatorBit - 1 

//* check whether the bit specified by indicatorBit is set or not 
//* set in indicator. For example, if indicatorBit has value 4 
//* check whether the 4th bit of the value in indicator is set  

    if indicatorBit bit in indicator is not set then 
        inputBuffer[inputIndex] := outputBuffer[outputIndex]
        should be:
        outputBuffer[outputIndex] := inputBuffer[inputIndex] 
        inputIndex := inputIndex + 1 
        outputIndex := outputIndex + 1 
    else 
        length := copy inputBuffer[inputIndex] as 16-bit integer in 
        little-endian format 
        inputIndex := inputIndex + 2 
        offset := length /8 
        length := length mod 8 

        if (length = 7) then 

            if (nibbleIndex = 0) then 
                nibbleIndex := inputIndex 
                length := inputBuffer[inputIndex] mod 16 
                inputIndex := inputIndex + 1 
            else 
                length := inputBuffer[inputIndex] / 16 
                should be:
                length := inputBuffer[nibbleIndex ] / 16 
                nibbleIndex := 0 
            endif 

            if (length = 15) then 

                length := inputBuffer[inputIndex] 
                inputIndex := inputIndex + 1 

                    if (length = 255) then 
                        length := copy inputBuffer[inputIndex] as 16-bit integer in little-endian format 
                        inputIndex := inputIndex + 2 
                        length := length - (15 + 7) 
                    endif 
                length := length + 15 
            endif 

            length := length + 7 

        endif 

    length := length + 3 

        while (not length = 0) 
        should be:
        while (length != 0) 
            outputBuffer[outputIndex] := outputBuffer[outputIndex - offset - 1] 
            outputIndex := outputIndex + 1 
            length := length - 1 
        endwhile 
    endif 

endwhile 

return