SmInfo: Inside Store Manager of Windows 7 and Windows 2008 R2 with Windd.
Store Manager (Sm*) is pretty new under Windows 7/Windows 2008 R2 kernel, this is a new management system to deal with both virtual and physical stores.
ReadyBoost (cache/files/logs, …) is one exemple.
Even through ReadyBoost had been firstly introduced into Windows Vista and Windows 2008 (Refer to Mark Russinovich writeup about Windows Vista Kernel for more information about ReadyBoost), Microsoft kernel developpers implemented ReadyBoost feature inside the Store Manager to make it more efficient.
In this blogpost, I am going to introduce a new WinDbg Extension called “sminfo” to retrieve information about the Store Manager from a Microsoft crash dump generated with Win[32/64]dd on a Windows 7 32-bits computer. (Thanks to Mark Wodrich for the help to use DML ;-))
Materials used are available here:
- Windd utility from this link. Windd works on x86 and x64 architecture from Windows XP to Windows 7 (for the crash dump feature).
- SmInfo Debugger Extension from this link. The following extension works only on x86 of Windows 7 and Windows 2008 R2 with Microsoft crash dumps and (local or remote) Kernel debugging.
Step 1# Plug a USB Key and tell to Windows that you want to use this USB drive as a ReadyBoost extension.
Step 2# Dump the physical memory into a Microsoft crashdump using the following command: “win32dd.exe /d /f readyboost.dmp”.
Figure 1 – Windd [Click to enlarge]
Step 3# Open WinDbg, load crashdump, configure symbols, load sminfo extension.
Figure 2 – Windbg + SmInfo [Click to enlarge]
0: kd> !sminfo
Store Manager (ReadyBoost) Debugger Extension – v0.1
Copyright (c) 2009, Matthieu Suiche http://www.msuiche.net
Windows 7 x86 and Windows 2008 R2 x86 ONLY!sminfo CACHE – Enumerate caches and display corresponding information.
!sminfo LOG [cacheindex] – Display entries from cache.
Step 4# Have fun with !sminfo commands!
Figure 3 – !sminfo CACHE [Click to enlarge]
Figure 4 – !sminfo LOG [Click to enlarge]
Figure 5 – !dml_proc [Click to enlarge]
As you can see, we can retrieve various information such as the file name, its size, cached pages and application list. So basically with this debugger extension you can have access to ReadyBoost log which can be helpful to troubleshoot your system.
Additional ressources
Developping Debugger Extensions – PDC Conference
The debugger extension – Several parts
MemInfo: Peer Inside Memory Manager Behavior on Windows Vista and Server 2008
PS. In February 2010, I will be in Washington D.C. to talk about Mac OS X Kernel internals and physical memory analysis at BlackHat. Description is available on BH website.
Reply to HBGary — and personal notes.
One HBGary developper wrote a blogpost about windd entitled “Windd – Almost there, but not quite…“.
HBGary says *they* but I would like to say to readers that windd is a project that I developped and maintain alone, on my spare time.
More and more people are using windd so it looks I have to explain some things about the behavior of Windows Memory Manager and windd itself. Reading HBGary blogpost really made me feel enthusiastic because I recently complained about the lack of feedback on windd which is really frustrating when you lead a free project.
As you probably know early version of windd were open-source, but mainly because of the lack of feedback windd is no more open-source but still free. As far I have seen, open-source doesn’t mean people understand what you wrote or read the code, it is more like a philosophical aspect like “We have access to the source”. Anyway, people are still free to send me an e-mail, at matt/msuiche/net if they have questions about the internal behavior of Windd, to insult me or just to thank me.
Back to HBGary blogpost, we can read :
- Is windd acquiring all of the available physical memory on the system?
- Would a “raw format” image dump of a 64-bit vista machine load properly into HBGary’s Responder?
- Should windd memory images that contain greater than 4GB of ram be considered admissible in court?
- Was windd really the first tool to support physical memory acquisition on Windows 7? (as claimed by the author)
Author replied “No” to all questions above.
In this blogpost, I am going to provide details about the bug for both technical and no technical people by explain how windd works. The main problem with HBGary article is that they mix accessible physical memory and accessible physical memory address spaces. So I assume most people do not even know the difference between these two *things*.
In July 2008, Mark Russinovich wrote a very well explained article about Windows Physical Memory Manager entitled “Pushing the Limits of Windows: Physical Memory“. In this article Mark also used a tool written by Alex Ionescu (co-author of Windows Internals 5th) called “meminfo” to retrieve information related to Windows PFN Database.
In this article Mark explains what physical memory, physical address space and PFN Database are.
For lazy readers here is a short summary.
Figure above is the physical address space.
Red blocks are devices address space, blue blocks are physical memory and the “Inaccessible RAM” (only by Windows) is a reserved space in the physical memory for the Operating System to proceed to the translation from Virtual to Physical addresses.
Joanna Rutkowska paper entitled “Defeating Hardware Based RAM Acquisition” is a good reading if you want to know more about “Red blocks”.
Figure above comes from meminfo tool which display Memory Manager physical memory blocks. These entries describe how the physical memory is “splitted” in the physical address space which interpreted by the CPU. For the remind, DMA Access provides access to physical memory and not to the physical address space.
We firstly notice the highest physical page is 0×120000 (1179648) which correspond to size of the physical address space and NOT to the size of physical memory.
Secondly we notice physical addresses are above 4GB even if the machine has only 4GB installed. To retrieve the size of installed/detected physical memory we have to add the size of each block as follows:
(0×9F000 – 0×1000) + (0xDFE6D000 – 0×100000) + (0×120000000 – 0×100002000) = 0xFFE09000 (~4GB)
#1 Bug HBGary is talking about only concerns the RAW memory dump generation with windd (v1.3.0.x <= Version < v1.3.0.20091113 (fixed version)).
The bug was the following: Windd was reading blue blocks and wrote them directly in a raw dump file like the DMA-way, then it means red-blocks were missing. Impact was the PFN database was invalid which means Virtual to Physical address translation was impossible. BUT windd DOES acquiere all available physical memory. Present version of Windd produces a “CPU-like” memory dump (physical memory address space) and fills red blocks with null pages.
#2 Second question was about HBGary’s Responder. I don’t know this product and I never used it. But it would mean HBGary does not support DMA-style memory dump.
#3 For the third question, please refer to #1 and #2.
#4 Regarding the last question about the fact that I claimed that windd was the first tool support physical memory acquisition I do not remember saying that. I just remember I claimed several times windd was a great tool because it can produce Microsoft crash dumps which has great advantages mainly because of Windbg. Then, windd also aims at being used by troubleshooters and/or kernel developpers and not only by forensics investigators.
For instance in your blogpost I can read “NOTE: HBGary’s Responder does not yet fully support the automatic analysis of Windows 7 which is why HBGary had elected to not publicly advertise Windows 7 acquisition support” — The difference between windd and average memory acquisition tools is this point: This problem does not exist with WinDbg. Windbg supports Microsoft Crash Dump since Microsoft started to work on Windows.
Analysis is very important, if someone produces a dump and is unable to analyse it this is more or less like if someone would say “I have a new car but I do not know how to drive it”.
Thanks again to HBGary for helping me to improve windd utility and I hope they like the new version.
BlackHat Webcast – New Frontiers In Forensics [Slides]
First, I would like to thanks people who attended to this Webcast and to BlackHat folks for inviting me and making this webcast great!
If you missed it, slides are now available at the following link: New Frontiers In Forensics [PDF]
People can access to Win[32|64]DD page here: http://windd.msuiche.net.
And media materials should be available on BH Website.
Windd 1.3 Final! (x86 and x64)
EDIT: 1.3.20091113 version contains a fix for incorrect size bug and raw memory dump.
EDIT: 1.3.20091024 version contains a fix for networking feature under Vista and Later.
Download windd 1.3
Win32dd and Win64dd are finally mature enough to be released which is a very good news.
First, I would like to thanks Nicolas Ruff, Andreas Schuster, Scott Noone from OSR Online, Rob T. Lee, Laurent Gaffie, Jimmy Marchetto and Sol_Ksacap for providing either assistance, feedbacks and/or beta-testing for this version.
Compability List:
Raw memory dump:
- Windows 2000 (32-Bits)
- Windows XP (32-Bits and 64-Bits)
- Windows 2003 (32-Bits and 64-Bits)
- Windows Vista (32-Bits and 64-Bits)
- Windows 2008 (32-Bits and 64-Bits)
- Windows 7 (32-Bits and 64-Bits)
- Windows 2008 R2 (32-Bits and 64-Bits)
Microsoft crash dump:
- Windows XP (32-Bits and 64-Bits)
- Windows 2003 (32-Bits and 64-Bits)
- Windows Vista (32-Bits and 64-Bits)
- Windows 2008 (32-Bits and 64-Bits)
- Windows 7 (32-Bits and 64-Bits)
- Windows 2008 R2 (32-Bits and 64-Bits)
Features:
- Raw dump generation
- Standalone Microsoft crash dump generation
- Network support (client + server)
- SMB path support
- MD5, SHA-1 and SHA-256 hash support
- Support 3 mapping methods for both full crash dump and raw memory dump generation
- Support 3 content rules
- Fast
- 32-bits and 64-bits support
- Can hibernate the system.
- Can generate a Blue Screen of the Death
- Support of machine with more than 4GB of RAM.
Microsoft Windows has an internal limitation which does not allow to generate a Microsoft Full Crash dump if the local machine has more than 2GB of physical memory. Of course, this limitation does not affect windd but it was funny and a good surprise to see Windbg correctly works with 8GB Microsoft crash dump (successfuly tested by Jimmy).
Links:
windd main page
Download windd 1.3
How to rule Windbg?
Debug Tutorial Part 4: Writing WINDBG Extensions
R.I.P. Xpress – Welcome TLZ
I was reading an article about Windows 8 and 9 (which should support IA-128 architecture) when I highlighted:
Researched new algorithms and programming methods to build Hibernate/Resume Integration API that can integrate and utilize the new TLZ file compression engine for the Hibernate/Resume component of new Windows 8 Operating System.
Using C and C++ programming languages in SourceInsight, developed a 100% functional C wrapper for C++ functions and the Hibernate/Resume Integration API, which will be used in Windows 8 replacing Windows Vista’s Xpress compression engine.
Apparently and according to his resume the author, Bo Qin, is a student at University of Washington. That is cool to see that some academics are working on cool projects (while some people are wasting time to find a way to write an exploit which will be used by script-kiddies or stupid consultants and while media are claiming white-hats are challenging Microsoft).
Anyway, Xpress compression algorithm, introduced in Windows XP and still used in Windows 7 and actually used for Windows Hibernation, Hyper-V, Windows Mobile, SMB protocol etc., should be replaced by TLZ algorithm that should be introduced in Windows 8.
