How to Stop File Corruption when Migrating Data

By: Nate Latessa, VP of Corporate Services

Data corruption can severely disrupt business processes. In the worst case, it isn’t noticed for a long time, and a broken file is copied to multiple locations. By the time anyone realizes there is a problem, recovering the lost data may be difficult or impossible. Adhering to data management best practices should minimize the chances of undetected data corruption and provide a way to restore data integrity.

Causes of file corruption

Several kinds of events can result in a damaged file.

  • Bit rot in storage. While it’s rare, disk sectors can go bad, damaging parts of files or entire directories.
  • Unreliable data channels. File downloads without data integrity validation sometimes result in corrupted files. This has become less common with the widespread adoption of HTTPS.
  • Improper removal of storage. Computers cache data rather than writing everything immediately to a file. Unplugging a storage device instead of using the system dismount command is likely to leave parts of a file in a bad state.
  • Power failures. Losing electric power has the same effect as unplugging a device, and it always happens when it’s most inconvenient.
  • Software errors and incompatibility. This problem is most common with software that has a narrow user base and a proprietary format. Creating a file with one version of the software and opening it with a different one might damage it.
  • Malware. Ransomware and other types of malware can make files unreadable, and meeting the extortionist’s demands doesn’t always help.

Non-standard file formats

File types used by only one application, such as proprietary software, are the hardest to recover from corruption. The software sometimes isn’t robust, writing bad files if they get too big or aren’t edited in just the right way. Compatibility issues can also occur from one version of the application to the next and many times, third-party tools are not able to recover these uncommon file formats.

Sometimes this results in the appearance of a broken file when it’s actually intact. If the application that opens a file is an older version of the one that created it, it may display alarming error messages that don’t explain the real problem. In these cases, opening the file with the latest version should fix the problem. Make sure not to save the file with the old software, or the damage could become permanent.

Preventing file corruption

With the right procedures, there is little risk in transferring files from one location to another. The following steps will avoid the large majority of data corruption or make it easy to recover.

  • Use the same software at all locations. Transferring files between different applications or different versions of an application can damage files. Use the same software and the same version of it for all processing of a given file format. Keeping an application up to the latest version will allow you handle all versions of its files.
  • Back up all files. If you discover that a file is damaged, you should be able to restore it from backup files with little or no loss of information.
  • Use secure transfer protocols. Secure transfer doesn’t just prevent interception. It checks that the received data hasn’t changed from what was sent, whether because of malice or channel noise.
  • Put equipment on an uninterruptible power supply (UPS). A UPS prevents unexpected power loss and its attendant file damage.
  • Use application file recovery features. Many applications can repair damage to their own files. Microsoft Word, for example, offers a text recovery feature when it detects file damage. However, compare the file against the most recent, validated version to ensure nothing was lost.
Previous PostNext Post