Synergy Broadcast Systems

Are External Hard Drives a Good Archive for Video?

by cjones on September 22, 2010

in Broadcast & Cable,Education,Healthcare,Technical

Post image for Are External Hard Drives a Good Archive for Video?

This is the second article in our series on “What’s the Best Choice for Archiving Video?” in the cable, education and healthcare space.  We reported on July 28th that Sony discontinued it’s line of AIT Storage products which served as a great archive for users with large video libraries. On September 8, 2010 we looked at whether you should consider using DVD as a possible archive source. Today we will look at using external hard drives.

What is an External Hard Drive?
An external hard drive is basically a standard hard drive that is connected to a server or PC with a USB or other type of cable. Today you can easily purchase hard drives in SATA and IDE in portable enclosures or cases with USBeSATApeSATA, IEEE 1394 Firewire interfaces to connect to a computer. USB may be the most familiar and easiest to use. You can also make your own portable drive by purchasing a standard hard drive and a buying a separate enclosure. External hard drives come in all sizes so does it make sense to trust these devices as a back-up?

My personal experience with external hard drives is limited to the USB variety and as a way to move files from one location to another when the files are too big for a flash or thumb drive so I decided to research the topic to see what other folks have experienced and if there are any specific “use cases” with large video libraries.

I found quite a few sources that had good things to say about external hard drives. The majority of the comments were associated with the USB interface and the general consensus is they have three strong benefits.

  1. They are cheap.
  2. They are easy to use.
  3. They are small thus portable.

All of these are great reasons to be attracted to external hard drives because they meet the primary requirements of most users.  However, what’s the other side of the coin? Are there reasons not to use external hard drives? Do they have any negatives? Here’s what I found:

Higher than average failure rate. This was a bit surprising. I found multiple articles and forum posts about very high failure rates with external drives. I was not able to find any industry sources that tracked or documented the problems but did find a lot of user experiences that were negative. Most of them seemed to focus on single PC usage where the USB drive was used to back-up a personal computer or photo archive. Many users commented on problems with the controller interface as the problem versus the drive itself but if you can’t get the drive to communicate with your system it’s going to be hard to get to your data. The closest thing to an industry study was done by Carnegie Mellon University that focused on hard drives in general and large storage systems rather than smaller system use. What they found was that failure rates were significantly higher on user reported failures versus manufacturer’s lab test results with mean time between failure. This is not a surprise as “testing” environments are historically atypical of user environments so the results will vary widely.

In checking with people I know to take an informal poll I did not find anyone who had experienced a high failure rate with these drives so in lieu of real experience or data it makes sense to do your own research before you buy. It also makes sense to perhaps purchase back-up drives for your back-up drives just in case you do experience a failure.

Manual space management. This refers to determining how much you can actually put on a drive. Unlike a simple PC back-up, video files are big and as you fill up the drive you may run into situations where a file is spread over more than one drive. This is not a good situation when doing a manual back-up. To avoid this situation you have to pay attention to the size of files being backed-up and the space remaining on the drive. This results in less efficiency because you may not be able to fill up all drives to capacity resulting in wasted space.

Manual back-ups are costly. The cost I’m referring to is your time and effort to manage multiple drives and files. As your library grows it will become more difficult to keep up with the inventory on each drive and when to rotate the drives. There is software available to do some of this work for you but the consensus seems to be that the utilities that come with the drives is limited at best. There are third party products available as well but no usage data was found to be helpful.

On the surface, external hard drives are cheap, easy to use and portable, which makes them perfect for personal computers, but they also come with problems if you want to use them to back-up a large video library. So, it may make some sense to ask a few questions to determine the viability of this approach:

  1. How much video content do you back-up now? If you just need to maintain a few hundred hours then external hard drives might be the best choice. But, if you have thousands of hours in your library a manual solution may not be in your best interest.
  2. How long do you have to keep it? There may be laws associated with archiving public meetings or other content that require you to maintain copies for a specific period of time.
  3. Can you store content on read/write devices? In some circles an archive is only considered an archive if the data cannot be changed which dictates the type of format you can use.
  4. Can you store content in different formats? As an example, if you broadcast your content as MPEG2 files normally can you archive them as Windows Media files? This could work when there are no plans to re-air a program so you transcode it to a smaller format to take up less space.

Do you have any experience with USB or external drives in general? Have you experienced or know others who have experienced a high failure rate with these drives? What do you use now for an archive? Please leave a comment and share any knowledge you have about using external hard drives to backup your video library.

video storage epaper

For more information on storage and how to protect your video assets please download our free e-paper: A Guide to Video Storage.

{ 2 trackbacks }

Is World BackUp Day Your Annual Reminder to Backup Your Data?
May 19, 2013 at 8:40 AM
Is Network Attached Storage a Good Video Archive?
March 3, 2014 at 11:23 AM

{ 6 comments… read them below or add one }

cjones October 28, 2010 at 11:27 AM

Greg:

Thanks I will definitely take a look. Thanks for sharing.

Listened to the interview and it is filled with great information and advice regarding storage and data management practices.

Thanks again.

Chuck

cjones October 28, 2010 at 11:25 AM

Jerry:

We will address NAS in a future article. I decided to write about individual technologies rather than jumble them all together and NAS is next up for a post.

Thanks for commenting.

Chuck

Jerry Robinson October 26, 2010 at 12:24 PM

While all of the experiences mentioned with ‘portable’ USB or similar single drives have happened to us here, no one mentioned the solution that we have been using for a couple of years with no problems. NAS or Network Attached Storage has served both as a project and as an archive backup in large reliable RAID drives (2+ TB) for less than a thousand dollars each. We don’t tote them around, we don’t drop them and we don’t let the environment get to them. They reside somewhere safe and are attached to the network, readily accessible to all who have a need for them. Check them out!

Greg October 26, 2010 at 10:44 AM

Great article, Chuck. Please check out my blog post on this subject. It includes a link to an interview with Mike Mihalik, Product Manager at LaCie who raises some issues that many folks may not be aware of.

http://makebettermedia.blogspot.com/2009/06/hard-drive-perils.html

Chuck Jones September 30, 2010 at 9:17 AM

Dale:

Your experience is very similar to what I found searching forums about this topic. Archiving and backup in general is an area that has tons of solutions but seems to fall short in areas of reliability and cost unless you’re an enterprise user.

I hope to shed some light on other options in the next few articles.

Thanks for sharing your thoughts on this.

Dale Berkebile September 29, 2010 at 7:58 AM

Chuck this is an interesting article. I have been using external hard drives for years to try and back up all our working files for client projects. Over the years I have used USB, Firewire and Ethernet based drives. Honestly I have been unhappy with all of them. In the end they seem to always crap out on me. I have worked with Lacie, Maxtor, Seagate and none have really met with my standard for a business solution.

On a personal level they may be great, but for my business I have wasted countless hours trying to recover data once the drive has been used for a while (months or years sometimes) and seems to be going on the fritz. Beyond the hundreds of hours of my life wasted over the years, I think of what my billable time is worth and I am saddened at the cost of this probably several tens of thousands of dollars. That being said, maybe a proper sever set up or a disc raid would have made more sense, but as a small business I thought going with the cheap external hard drives would be the best solution. Looking back I think I was incorrect as with anything you get what you pay for. I am thinking the next time we invest in a backup system it will either be external backup and/or quality sever and raid system.

Leave a Comment

Previous post:

Next post: