System Rescue with Knoppix

Kyle Rankin

Quinstreet Inc.

Author of Knoppix Hacks and Knoppix Pocket Reference

http://greenfly.org/talks/knoppix/rescue_hp.html

What is Knoppix?

A complete Linux distribution that runs directly from a bootable CD-ROM, Knoppix has many uses:

What is Knoppix?

A complete Linux distribution that runs directly from a bootable CD-ROM, Knoppix has many uses:

Demonstrate Linux
Portable Linux Distribution
Install Linux
System administration
Rescue disc
Live CD creation
Many other uses

What is Knoppix?

A complete Linux distribution that runs directly from a bootable CD-ROM, Knoppix has many uses:

Demonstrate Linux
Portable Linux Distribution
Install Linux
System administration
Rescue disc
Live CD creation
Many other uses

System Rescue with Knoppix

Linux Rescue
Reinstall boot loaders Image damaged drives Restore partition tables Recover deleted files Compromised system forensics

System Rescue with Knoppix

Linux Rescue	Windows Rescue
Reinstall boot loaders Image damaged drives Restore partition tables Recover deleted files Compromised system forensics	Reset the Windows boot loader Restore System Files Reset Windows Passwords Edit the Windows Registry

Why Knoppix?

Excellent hardware detection
Safe
Useful rescue tools: lilo, grub, dd, gpart, dpkg, rpm, etc.
Full Internet access for troubleshooting
~~Games!~~Productivity Applications

Reinstall Boot Loaders

Reinstall Boot Loaders

Chroot makes it easy

Knoppix defaults to nodev. Mount explicitly:
knoppix@tty1[knoppix]$ sudo mount -o dev /mnt/hda1
Make changes in /etc/lilo.conf or /boot/grub/menu.lst
Restore lilo:
knoppix@tty1[knoppix]$ sudo chroot /mnt/hda1 lilo
Restore grub:
knoppix@tty1[knoppix]$ sudo chroot /mnt/hda1 grub-install /dev/hda

Image Damaged Drives

Don't use dd.
Use dd_rescue and dd_rhelp

Image Damaged Drives

Why use dd_rescue and dd_rhelp?

dd doesn't handle bad blocks well (even with noerror)
dd_rescue does, but...
dd_rescue is slow
dd_rhelp uses dd_rescue, but skips bad block clusters until later so...
dd_rhelp can be resumed

Image Damaged Drives

Get dd_rescue and dd_rhelp

Image Damaged Drives

Get dd_rescue and dd_rhelp

dd_rescue binaries at http://www.garloff.de/kurt/linux/ddrescue
extract dd_rescue binary from tarball
dd_rhelp source at http://www.kalysto.org/utilities/dd_rhelp/index.en.html
untar, configure, make dd_rhelp source
copy dd_rescue and dd_rhelp binaries to ~/.dist/bin

Image Damaged Drives

Use dd_rhelp

mount the destination drive
knoppix@tty1[knoppix]$ sudo mount -o rw /dev/hdb1 /mnt/hdb1
run dd_rhelp
knoppix@tty1[knoppix]$ sudo dd_rhelp /dev/hda1 /mnt/hdb1/hda1_rescue.img
fsck the image
knoppix@tty1[knoppix]$ sudo fsck -y /mnt/hdb1/hda1_rescue.img
mount the image loopback, or reimage to another drive

Restore Partition Tables

Restore Partition Tables

Gpart

short for "Guess Partition"
works best with primary partitions
not perfect

Restore Partition Tables

Run Gpart

First run gpart without the -W option to test
root@ttyp0[/]# gpart /dev/hda
Then use -W to actually write the changes
root@ttyp0[/]# gpart -W /dev/hda /dev/hda
Tweak the results

Recover Deleted Files

Recover Deleted Files

Let's clear out this directory...

Recover Deleted Files

Let's clear out this directory...

root@tty1[/]# rm -rf ./*

Recover Deleted Files

Let's clear out this directory...

root@tty1[/]# rm -rf ./*

. . .

Recover Deleted Files

Let's clear out this directory...

root@tty1[ / ]# rm -rf ./*

. . . / ?!

Recover Deleted Files

Let's clear out this directory...

root@tty1[/]# rm -rf ./*

CTRL-C!

CTRL-C!

Recover Deleted Files

Use unrm and lazarus from The Coroner's Toolkit to recover deleted files.

Recover Deleted Files

Use unrm and lazarus from The Coroner's Toolkit to recover deleted files.

Prepare the partition to recover (/dev/hda1 here)
Prepare a data partition (/dev/hdb1 here)
Run unrm
Run lazarus on unrm output
Review lazarus output

Recover Deleted Files

Prepare the partition to recover

Turn off the machine!
Boot from a rescue disk (like Knoppix) that won't mount the drive by default
Mount the partition read-only
root@tty1[]# mount -o ro /dev/hda1 /mnt/hda1

Recover Deleted Files

Prepare a data partition

At least 220% of recovery partition free space
100% for unrm, 100% for lazarus, and ~20% for overhead
Must be a different partition than the one to recover
Mount read-write
root@tty1[]# mount -o rw /dev/hdb1 /mnt/hdb1

Recover Deleted Files

Run unrm

Make a directory to store the unrm output
root@tty1[]# mkdir /mnt/hdb1/unrm
Run unrm
root@tty1[]# unrm /dev/hda1 > /mnt/hdb1/unrm/output

Recover Deleted Files

Run lazarus on unrm output

Make a directory to store recovered blocks in unrm dir
root@tty1[]# cd /mnt/hdb1/unrm; mkdir blocks
Run lazarus on the unrm output file and output
root@tty1[unrm]# lazarus -h -w . -D blocks output
(-h: HTML output, -w: where to store HTML frames, -D: where to store files it recovers)

Recover Deleted Files

Review lazarus output

Grep within blocks directory for keywords
View the HTML output
root@tty1# mozilla file:///mnt/hda2/unrm/output.frame.html

Compromised System Forensics

Compromised System Forensics

When collecting forensics data:

Compromised System Forensics

When collecting forensics data:

Respect Order of Volatility
Get as much data as possible
Automate data collection
Analysis is up to you
Use The Coroner's Toolkit (specifically grave-robber)

Compromised System Forensics

Two Step Process:

Compromised System Forensics

Two Step Process:

Collect forensics data (30 mins - 1hr)

Live machine vs. "Corpse"

Analyze forensics data (days, weeks, months)

Compromised System Forensics

Collect forensics data

Mount the corpse (hda1 here) read-only and destination (hdb1 here) read-write
Create a destination directory for grave-robber (/mnt/hdb1/data here)
Start script to save all grave-robber output
$ sudo script /mnt/hdb1/grave-robber-output
Run grave-robber within the script session on the corpse
root@0[knoppix]# grave-robber -c /mnt/hda1 -o LINUX2 -d /mnt/hdb1/data -v
Type exit to exit script

Compromised System Forensics

Analyze Forensics Data

body: Database of information for all files and directories on the system
body.S: Like body, but only shows SUID files
command_out: The output of various commands like df, dpkg/rpm, and lsof run by grave-robber
conf_vault: A full copy of "interesting" files like config files
pcat/icat/proc: Running process information. Useless with a corpse
trust: .forward, .rhosts, and crontab/at output
user_vault: User shell histories, ssh keys, etc.

The Windows "Recovery" Disk

What it can do
View text files Check the disk for errors Fix the MBR and boot.ini with defaults Partition drives and format them Restore system files from CD

The Windows "Recovery" Disk

What it can do	What it can't do
View text files Check the disk for errors Fix the MBR and boot.ini with defaults Partition drives and format them Restore system files from CD	Edit text files Browse outside of %SYSTEMROOT% and the root folder Copy to floppies or CD Edit registry Much, much more

Knoppix the Windows Recovery Disk

Knoppix the Windows Recovery Disk

What Knoppix can do

Write to the full disk (even NTFS)
Fix boot.ini
Scan for viruses
Install files from .cabs
Blank passwords
Edit the registry

Reset the Windows Boot Loader

Reset the Windows Boot Loader

The Knoppix equivalent of fdisk /mbr:
knoppix@tty1[knoppix]$ sudo install-mbr /dev/hda

Restore Corrupted System Files

Mount the Windows filesystem read-write
Find a .cab file on the system that contains the file you need
Then use cabextract to extract the file:
knoppix@tty1[knoppix]$ cd /mnt/hda1/winnt/system32/
knoppix@tty1[system32]$ sudo cabextract -F ntdll.dll "/mnt/hda1/winnt/ServicePackFiles/i386/sp4.cab"

Reset Windows Passwords

Get the latest chntpw deb from http://packages.debian.org/unstable/admin/chntpw
Extract the binary from the deb:
$ alien --to-tgz chntpw_0.99.2-1_i386.deb
$ tar xvzf chntpw_0.99.2.tgz ./usr/sbin/chntpw
$ mv ./usr/sbin/chntpw ~/.dist/bin/
Mount the Windows partition read-write
Change to the winnt(or windows)/system32/config directory
Blank the Administrator password:
$ chntpw SAM
Blank other user passwords:
$ chntpw -u username SAM

Edit the Windows Registry

Be Careful
Use chntpw with the -e option on the registry hive
$ chntpw -e SOFTWARE
Browse the registry with cd, ls, cat
Add, edit, and delete keys
q to quit, ? for help
Unmount the filesystem to ensure changes

Any Questions?

Other Resources

Official Knoppix Website:

http://www.knoppix.org

Unofficial Knoppix Forum (English):

http://www.knoppix.net

Knoppix Hacks:

http://www.oreilly.com/catalog/knoppixhks

Knoppix Pocket Reference:

http://www.oreilly.com/catalog/knoppixpr