Hard Drive and File Wiping
Update:
As there seems to be a lot of people still do not understand what a full disk wipe with one pass is, I’ve written a newer article on this. It demonstrates with screenshots, what happens to data after a single pass wipe.
Read that article: anti-forensics
Many people are under the impression that hard drives need to be wiped with multiple passes to prevent recovery of data. This is simply untrue with modern hard drives. According to the National Institute for Standards and Technology, “Studies have shown that most of today’s media can be effectively cleared by one overwrite.”
You may be confused between disk wiping and file wiping and deletion. Wiping a hard drive involves using software or a hardware device to completely write over every bit of a hard drive. This will prevent the recovery of nearly all data on that hard drive. There are methods to “recover” some things which I will explain in a bit.
File wiping involves using software to completely write over the contents of a file. The entry for that file in areas such as the file allocation table is usually removed as well. Wiping files is better than pressing the delete key on your keyboard but remnants of these now wiped files can still be found in other places on the hard drive. This is especially true if the file is copied back and forth between volumes, has been cached to the disk from RAM and numerous other operations done by the operating system.
Regular file deletion does not really delete the contents of a file. On a Windows XP system this includes choosing a file and pressing the delete key to move it to the recycle bin. As well as the files emptied from the recycle bin and files deleted by holding down shift while pressing delete to bypass the recycle bin. Think of your hard drive as a book. The book has a table of contents with chapters which represent files. The only way to find chapters (data) is through the table of contents. When a file is deleted its entry in the table of contents is removed, leaving the data in that chapter in the book but no actual reference to where it is in the table of contents. In reality, something similar happens on the hard drive. Those leftover contents will eventually be written over as the space they take up is needed.
So why are there so many recommendations for multiple passes during disk wiping?
Some recommend physically destroying a hard drive or writing to it 3, 7 and even 25 times as the only reliable methods of getting rid of data. This really is not the case. Data is stored magnetically and are represented by 1′s and 0′s. In older hard drives it is possible to view previous states that these magnetic areas existed in. Such as a 1 used to be a 0. This is done with an electron microscope in the examples that I’m aware of. Even though this is possible, it would still be nearly impossible to get enough correct readings to put together a document, picture or anything else. There is currently no public example of this method actually returning any useful results.
Modern hard drives are even more efficient, making it harder to read what state bits were previously in.
Data Destruction Methods
The simplest form of data destruction is simple overwriting of the entire hard drive. As mentioned above, wiping a modern hard disk once is enough to prevent recovery of data.
Another simple method of data “destruction” is encryption. Encrypting a hard disk with full disk encryption will effectively render that data unreadable as if it had been overwritten with random characters.
Degaussing is one of the best but most expensive methods. It involves using hardware which renders previous data on a hard disk unreadable by changing the magnetic alignment of areas of the hard disk.
Another sure-fire method is physical destruction to the platters inside of the hard drive. This can be done by smashing, grinding and shredding them. You can burn them and dip them in corrosive acid as well. Essentially, anything that can cause total destruction to the platters will destroy the data on them.
If you’re in the habit of hording copyrighted material that does not belong to you on opitical storage media such as CDs and DVDs then the quickest way to destroy this data is in the microwave. In today’s world, there is really no need to horde pirated data on optical media.
It is much easier and safer to store it on encrypted hard disks. However, if you ever find yourself in a situation that involves federal agents beating down your door, you may want to throw your stash in the microwave. However, there’s always that awkward situation where you have to explain why you have fifty microwaved DVDs and CDs in your microwave.
Disk Wiping Software
I’m sure you’ve heard of DBAN or Darik’s Boot and Nuke. Most people who work in IT have. This is because it works and it is very effective. You can pop the CD in, go through a few menu’s and then leave the machine running while DBAN does all the work. It can wipe every hard disk connected to the system in succession. There are options to do more than one pass, which you should avoid unless you don’t mind waisting your time.
Another method I use quite a bit is to just hook a drive up to a Linux system or pop a bootable Live CD in the machine and boot into a Linux environment to use the “DD” command. It can be as simple as this: dd if=/dev/zero of=/dev/[DISK HERE]
Remember to read the man page on DD if you plan on using it. There is also DCFLDD which can perform the same actions and more. DCFLDD has been geared towards computer forensics and security.
For file level wiping I’m a fan a Jetico’s BCWipe. The software is highly customizable and different wiping options can be setup to run at different times. It can wipe free space or unallocated space on a hard disk which is where older “deleted” files reside. This will prevent recovery of data using forensics and data recovery software from unallocated space. It will also wipe file slack. Data is split between clusters on the hard disk. Files are rarely the perfect size to always fill every cluster up, so what is leftover after the end of that file in a cluster is file slack. It can contain remnents of previous files. It can also wipe and clean old file entries, the swap file, recently used file lists and many other things including custom locations. Lets just say that if BCWipe is used correctly, it can really make a computer forensics examination a pain in the ass and probably render any examination of the drive irrelevant depending on the type of evidence that needs to be collected.
Don’t limit yourself to just this software. There is a lot of file level wiping software out there. Some free and some not so much. The reason I have listed BCWipe is that I personally use it and find it very reliable and effective. Another bit of free software that I find useful is CCleaner, which is very similar to BCWipe. You must turn on actual overwriting of files manually within the settings of the program. I use it alongside BCWipe to cover a larger area of temp files, recent file lists and other areas history and artifacts may be lurking.
A great method of confirming that your hard drive has been fully wiped is to open the physical disk with a hex editor like WinHex and confirm that the wiping pattern matches what you’ve chosen. I personally just use zero’s.
In conclusion…
If you have any comments, suggestions or ideas please comment. Share the names of any software you use as well. If you have developed your own software or want to and would like to promote it then think about creating an account on the anti-forensics forum to post a topic about it.
Well almost….If you read NIST Special report 800-88 single pass processes such as is initiated by Secure Erase technology is an acceptable form of purge level sanitization… However, software based overwrite will NOT effectively CLEAR all recoverable data from the media surface. Referencing the NIST 800-88, you will note that software and externally initiated overwrite processes are classified as CLEAR level processes. Rather, they are susceptible to data recovery using keyboard level recovery efforts. Processes such as Degaussing and Secure Erase are classified as Purge level processes where they are not susceptible to laboratory level recovery efforts.
So, what is this single pass process called Secure Erase that can eliminate data beyond other overwrite processes… It is a technology initially developed by IBM as a feature to their TravelStar line of drives. The protocol was elaborated by the UCSD’s Center for Magnetic Recording Research at the request of the NSA. Today, Secure Erase is integrated in to all standards compliant ATA storage devices manufactured since 2001.
Secure Erase is initiated by a command sequence, and once initiated, can not be stopped until the process is finished.
So, if SE is so effective, why isn’t everyone using it? Well… this is due to the fact that although it is highly effective, it is seen by ost PC manufacturers as a potential security vulnerability. This is due to the fact that if a virus, or malware were to initiate the SE process, the data contained on the drive would be eliminated with no hope for recovery. Accordingly, the BIOS, hardware and OS manufacturers have implemented features to inhibit the passing of the SE Init command to devices connected to the host controller… Ultimately, anyone producing software to launch SE would find themselves in a position where their product would not work on the majority of the equipment out there… and if it did work, there is a good chance that the host controller hardware should not permit access to the Host Protected Area and other Protected Service Areas on the drive,. Whereby rendering the processed device with recoverable data.
Why Multipass? well the concept goes a lot further than attempted recovery by Magnetic Force Microscopy (MFM). The concept comes down to, what is a reliable process to assure effective coercion of every pole, whereby eliminating any trace of latent data? Many processes proselytize 3 passes including a pass of all ’1′s, random data, and all ’0′s. As you say, others go as far as many more passes. Regardless, no matter how many passes are done, regions such as the G-List sectors, HPA, and the regions beyond the Device Control Overlay still may contain recoverable information (less probable in the DCO regions).
Degaussing is a cat and mouse game. As drives increase in media density, the need for more powerful degaussers is required. Currently, a common 500 Gig device will require a degausser capable of 11,000 oerstead to effectively achieve coercion of the entire media surface on a disk stack. The issue with Degaussing is not only the concern about assuring the power of the degausser being suitable for the task, but that the process disables the electro-mechanical components of the drive before effective coercion may be achieved. This means that, assessment of the effectiveness of the process will be complicated. Also as a connection free process, any logging is done manually and may be susceptible to human error.
Encryption is an effective solution for the protection of live data. It is NOT considered best practice for the protection of End Of Life Data. Referencing government protocol established for the protection of higher level classified data the concept is that data must be eliminated beyond any recovery effort using current or future technologies. Accordingly, encryption is the process of using a key to obfuscate the original data. Regardless of the key strength, a key can exist, or be recreated using technology available in the future to create and process such complex keys.
Physical destruction is a good alternative to permanent data destruction. However, again, when handling high levels of classified data, the particles from the process must be smaller than the smallest recoverable element, a single data sector, or a particle smaller than 1/250th of an inch (formerly 1/125th of an inch up to March ’08). However, for most of us, shredding, smelting and other physical destruction processes will do the job. The one issue with physical destruction is that it is often conducted at an offsite facility, which means that unprotected devices are shipped off to this site.. The handing off of unprotected (or poorly protected) storage hardware exposes the organization to the potential for loss of assets in transit. Regardless of who is in possession of your drive, the data is still owned by you, which means, it will be your org filing the mandatory disclosure notice, and that the carriers name will only be a footnote on the filing. This is another good reason to favor in-house processing, or pre0-processing should your policy dictate physical destruction for higher levels of data classification.
Microwaving optical media is dangerous, and produces toxic fumes. If you feel compelled to practice this method of media destruction, think twice, it may be amusing to watch, but the risk may not be worth the perceived gain.
Software based technology may be suitable for the clearance of non-confidential information, and suitable for the home user. In the enterprise, where multiple levels of data classification exists in a work environment, short of inventorying your data assets by class, then it is difficult to assess handling procedures for by device. Where confidential or personally identifiable information exists, clear based processes will not be an adequate process.
Secure Erase can be used effectively and can be affordable when approached through the use of purpose built Secure Erase appliances. These devices such as those manufactured by Ensconce Data Technology (www.deadondemand.com) enable users to effectively purge all data from the drive, beyond forensic effort in a process that requires between 17-35 minutes per 100 Gig of volume space. Secure ErRase is also a green solution, where the device is rendered re-usable at the end of the process (unlike degaussing).
Just my 5 cents…
Thank you for the time to write a very well thought out and informative reply!
I’m guessing the Federal Government has put in standards for multiple passes and complete incineration or destruction of every particle of drives because they want to stop future recovery attempts if technology were ever to ever get to that point.
I recommend encryption as a technique because you can fully encrypt a hard disk beforehand but you can’t fully wipe a disk beforehand and still use it. All of these suggestions are from the perspective that your activities violate some sort of law in your country.
Great comment on the HPA (host protected area) as well. I should note that DBAN does not clear this area of the drive either.
I was going to add a link to The Great Zero Challenge (http://16systems.com/zero.php) because it is still a widespread myth that a single pass is not effective. It is very effective and all that is needed.Essentially, it is a challenge to recover data from a hard drive that has been fully wiped with a single pass.
As for magnetic force microscopy and using electron microscopes, from what I’ve read the process is very cumbersome and long and the chances of correctly identifying the previous state of a bit are very low. So take a simple word document or picture for example…
For a simple forensics examination or even data recovery your forensics software will first need to identify a header or footer for the file (or a complete file table which is highly unlikely for the same reason as below). Once it finds this, it will need to rely on the data not being fragmented across the hard drive.
Even a simple text document with one sentence will be unrecoverable after a single pass. You will need 8 consecutive successful bit recoveries to recover a byte. In a text document that byte represents a single character. If even one of those bits is recovered incorrectly through magnetic force microscopy then that byte is completely different.
Thank you again though for a very informative reply. This is good stuff. I figured microwaving your optical media would produce some sort of toxic fume but that would be the last thing on my mind if I were to be destroying potential evidence.