I had noticed my system getting flaky recently. As soon as I could, I started backing up all my data files. When my hard drive failed, I lost a very small amount of data. It took me about 8 hours to rebuild the system on a blank hard drive. 

First, I had to install Windows XP from scratch. This is a necessary step, since my main system is a laptop. I had to be able to connect to another computer on the network, which is where I had backed up all my files. I thought about trying to create a "network aware" DOS boot disk, but I decided it would be faster (about 45 minutes) to simply reinstall Windows XP.

One of the main gotchas about Windows XP is that it is very picky about the hard drive partition layout. I had previously used Powerquest's Drive Image to backup entire Windows partitions. But if I restore them to a different location on the hard drive, they often would fail to boot. Or worse, I would discover that the drive letter references stored in the registry were pointing to the wrong place. 

What I had to do was to re-create the partition layout that existed on the failed drive. Based on past experience, I had made screen captures of the partition layout when I had changed it. I typically used Powerquest's Partition Magic to make these changes, such as resizing partitions, and re-allocating free space. Here is what the drive looks like today, as displayed from Partition Magic:

Here are the steps that I followed to do this:

1. Format the blank drive
2. Partition the formatted drive
3. Install DOS 6.22
4. Install Windows XP
5. Connect to the network and copy files to various partitions
6. Restore the Windows registry files
7. Boot into the restored Windows XP
8. Assess the damage - (Any software that won't run)

Steps 1 and 2 - Format and partition the blank drive
I used Partition Magic to create the empty partitions, and to format them. I wanted to duplicate the exact layout I had on the failed drive. In my first attempt at recovery, I did not do this. Eventually, I realized my mistake, and decided to start over again.

 The C: boot drive and the H: Windows system drives were formatted as FAT32 to aid in any future recovery efforts. When you boot from a DOS diskette, you generally can't access any NTFS partitions. There are some useful programs from Powerquest that allow you to manipulate the Windows registry from DOS - but the file has to exist on a FAT32 partition to use the program.

The first primary partition originally held the Dell Diagnostics files. But I had never backed up that partition, since I never actually used it. Those files were lost as part of the drive failure.

Steps 3 and 4 - Install DOS and Windows XP
I installed DOS 6.22 from diskette. I like the ability to boot into DOS without having to use a boot diskette.

Then I ran setup from the Windows XP CD. As shown above, I installed into a primary partition immediately following the DOS partition. I end up with a BOOT.INI file in the DOS partition. This allows me to edit the file without loading Windows XP first. 

I also turned off System Restore on all the empty drives. This "clean install" of Windows XP does not need to even know about the extra logical drives.

Step 5 - Connect to the network and copy files to various partitions
I copied the files one partition at a time. For example, I had a prior backup of the C: drive which included the DOS versions of both Partition Magic and Drive Image. As shown above, I have one logical partition for most programs (D:) one partition for most data files (E:), one partition for a swap file (F:), and so on. It took several hours to copy the 20 GB of data files across the network.

Step 6 -  Restore the Windows registry files
Jcopy can't copy the Windows XP / 2000 / NT registry files (these files are locked by those Windows versios). But I have found a program that can both copy and restore the registry. It is named ERUNT, and it is available here:
http://www.larshederer.homepage.t-online.de/erunt/

This program uses some serious magic to copy the registry files while Windows is running. It will copy the files by default to C:\ERDNT. It also creates ERDNT.INF and ERDNT.EXE in that same directory. The ERDNT program will restore the registry files to the "correct" location on your hard drive. 

One key is that the drive letter in the ERDNT.INF file has to be the same as the drive letter where Windows is installed. But more importantly, all the other drive letters have to be correct, based on what is stored inside the Windows registry files. If those drive letters don't match, then Windows will be very unhappy!

You can change the drive letter assignments in Windows XP using Administrative tools / Computer Management / Disk Management. Since I had the correct drive letter assignments, I simply ran the ERDNT program, which restored the registry files that I had saved from the day before.

Step 7 - Boot into the restored Windows XP
I re-checked the items in C:\BOOT.INI. I had the proper identification of my "Production Windows XP", so I restarted the computer, and booted into my production environment. I got a message about a new hard drive, and had to reboot again.

Step 8 - Assess the damage (Any software that won't run)
I knew that things were going TOO smoothly for me to be done. One of the problems with the Windows registry is that it stores path names using short file names. This is necessary because certain device drivers are loaded before Windows, so the long file names can't be used.

One perennial problem is Norton SystemWorks. The directory structure looks like this:
H:\Program Files\Norton SystemWorks\Norton Utilities
H:\Program Files\Norton SystemWorks\Norton Cleansweep
H:\Program Files\Norton SystemWorks\Norton Antivirus

Using short file names, you have H:\PROGRA~1\NORTON~1\NORTON~1, H:\PROGRA~1\NORTON~1\NORTON~2, and H:\PROGRA~1\NORTON~1\NORTON~3 stored in the Windows registry. Depending on the EXACT order of copying the files from backup, the short file name for any of the three sub-directories may not match what is stored in the registry. 

In my case, I got an error message from Norton Antivirus telling me to uninstall and re-install the program. I actually had to uninstall and re-install the entire Norton SystemWorks set of programs. I also had a Visual Basic Add-In that did not work. But that was the extent of software programs that I found - my system had been fully restored.

Symptoms of impending hard drive failure

1. Very slow hard drive - due to numerous retries, operations like copying files, or deleting files, had gotten noticeably slower. This was before I heard any weird click/clack noises.

2. My machine rebooted about once daily, while it was sitting idle. This has NEVER happened before with Windows XP. Then the machine would say there was no hard drive, and give me the options to retry (via F1), or go into the BIOS (via F2). Usually I had to turn the PC off for an hour before it would boot successfully.

3. At first I heard a few click/clack noises - only seven on Friday night. But as each day went by, they happened more frequently. On the day the drive failed, I would experience 1-2 minutes of "freeze-up" and hear the click/clack noises while the drive head would retry over and over and over again.

4. Right before the drive failed, I got a hard data error. I had run Drive Image to back up my Windows partition. While copying the 1.5 GB file, I got a CRC error. I could hear the hard drive making numerous retries, but there was no longer any click/clack noises. Instead, it would repeat the same drive activity over and over and over again. At this point, I gave up trying to get data from the drive.