PART 4: Fun with Debugging, Crash Dumps, and Traces
Debugging Slang
Apoology
A spelling mistake with hidden memory dump and log analysis meaning.
MedioCriticalSection
Related to mediocrity at work.
SPASM
Stored Procedure in ASM.
NoOO
A new old programming language.
AI
Analysis Impossible.
To Come Out of the Shell
To leave CUI and start wandering in GUI space.
3D Weekend
A weekend not spent before a 2D screen.
To Crawl into (One's Shell)
Submitted by Malcolm McCaffery
To retreat into one's CLI or otherwise isolate oneself so as to avoid undesirable bugs, situations, or interactions with programs.
Example: After getting repeated Outlook hangs every day, I crawled into my shell and just used telnet to access my email for nearly a week."
Bad Feeling
The feeling you have when you see "bad" in a memory address.
Examples: I have a bad feeling that ALPC wait chain I'm looking at now will point to something ominous. The ServerThread address in the output of !alpc /m command contains "bad". I have a bad feeling about this (pointer).
The Valley of Crash Dumps
Example: My evening excursion to the Valley of Crash Dumps:
Early Debugging
It's not "Debug early in software lifecycle". It's debugging early in the morning.
Example: Debugging while on a 5am train.
CHARLATAN
From: "Crashes, Hangs As Ram LATency ANswer". A person who explains software problems as effects of faulty RAM or viruses.
Diagnostics and Debugging in Science Fiction
Here’s an incomplete list (continued from Volume 9a) of SF short stories, novellas, and novels I have read by the time of this writing with my summaries and thoughts.
"I think I accessed it in a memory core somewhere." (The Reality Dysfunction: The Night's Dawn by Peter F. Hamilton)
James Bond’s Bugcheck and Error
This is Bug Check 0x007: INVALID_SOFTWARE_INTERRUPT.
0:000> !error 7
Error code: (Win32) 0x7 (7) - The storage control blocks were destroyed.
Two-field System Agriculture
I can't recall any single day when Windows 10 wasn't updating itself.
Proposal: a system and method of daily updates based on two-field agriculture. Two VMs, one is being updated, on the other isolated you work. Then you switch between them.
Bugs and InfoSec
A connection of bugs with InfoSec in Russian: инфосекомое (инфосек + [насек]омое).
Program Evolution
Program evolution after numerous updates: stable -> crashes -> leaks + crashes -> hangs.
Program evolution explained: e vol(ume) (poll)ution.
Roman + Hex
If Roman + Hex numeric system were used there could have been BugCheck LIVE in addition to BugCheck DEAD.
Debugging Curiosities
Trace Messages
Examples of trace messages encountered in real software logs:
unrecognized error code
Some aspiring analysists reported errors that were parts of files names. So we devised a rule: When "error" is in the file name found in the log it may not be an error. Example:
...
Excluded from processing: "data.txt"
Excluded from processing: "error.log"
...
Some apps cannot handle success. From a log file:
App returned unhandled STATUS_SUCCESS
Moscow Scare
We thought moshost was related to Moscow but it is just "Downloaded Maps Manager".
0:010> kc
# Call Site
00 ntdll!NtWaitForMultipleObjects
01 KERNELBASE!WaitForMultipleObjectsEx
02 KERNELBASE!WaitForMultipleObjects
03 kernel32!WerpReportFaultInternal
04 kernel32!WerpReportFault
05 KERNELBASE!UnhandledExceptionFilter
06 ntdll!TppExceptionFilter
07 ntdll!TppWorkerpInnerExceptionFilter
08 ntdll!TppWorkerThread$filt$3
09 ntdll!_C_specific_handler
0a ntdll!_GSHandlerCheck_SEH
0b ntdll!RtlpExecuteHandlerForException
0c ntdll!RtlDispatchException
0d ntdll!KiUserExceptionDispatch
0e moshost!ScopedWatchdogTimer::WaitCallback
0f ntdll!RtlpTpWaitCallback
10 ntdll!TppExecuteWaitCallback
11 ntdll!TppWorkerThread
12 kernel32!BaseThreadInitThunk
13 ntdll!RtlUserThreadStart
Vacuum Needs PDB
Even vacuum needs a PDB file:
************************************************************************
*** ***
*** ***
*** Either you specified an unqualified symbol, or your debugger ***
*** doesn't have full symbol information. Unqualified symbol ***
*** resolution is turned off by default. Please either specify a ***
*** fully qualified symbol module!symbolname, or enable resolution ***
*** of unqualified symbols by typing ".symopt- 100". Note that ***
*** enabling unqualified symbol resolution with network symbol ***
*** server shares in the symbol path may cause the debugger to ***
*** appear to hang for long periods of time when an incorrect ***
*** symbol name is typed or the network symbol server is down. ***
*** ***
*** For some commands to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: PVOID ***
*** ***
************************************************************************
My Surname Decomposed
VO ST OK OV. Push V. Push O. ST: Software Trace. OK. Pop O. Pop V.
Slavery
Slavery implicated in the latest software incident:
0:000> !lmi Module
[…]
Pdb: c:jenkins-slave...
[…]
PORCA
PORCA (Pattern-Oriented Root Cause Analysis) sounds like ПОРКА in Russian (means "flogging", a medieval RCA)
Double Fee Request
0:000> .cxr 0055f244
eax=00000001 ebx=fffffffe ecx=0025b4c0 edx=00feefee esi=00000024 edi=00000002
eip=00feefee esp=0055f6a8 ebp=00000000 iopl=0 nv up ei pl nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202
feefee ?? ???
Word Symmetry and Soviet History
"VDNK"h and "NKVD" double words have word reversal symmetry.
Everything You Need for Debugging
Extra brain, massage device, bug samples for comparison.
Bugs in the System
When I saw the spine of this book "Bugs in the System" on the bookshop shelf, I rushed to dig it out, but it turned out it is about entomology, not software systems. Still may be useful.
Direct HR Reporting
Interesting function in a crash dump: FailFast::ForHR - software faults are reported directly to HR.
The Devil at My Heels
0:666> k
[...]