Aug 25, 2011

Business Expansion and Server Solution // Dilemmas & Suggestions

If you're not a business don't tune out just yet.  There will be some valuable info for you too.  Whether you're a blogging mom, music/movie junky, a small retail business or a larger production studio you've all probably felt the growing pains of digital storage.  You're left with a few option, buy another external drive, try yourself or pay someone to install a larger internal drive, buy a new computer or maybe abandon technology and go back to sticks and stones.

Let's lay out a few numbers.  Some basic math (if you can call it that) 1024 MB = 1 GB.  And in kind, 1024 GB = 1 TB.  A typical HD movie from iTunes will set you back 2-4 GB each.  A song from iTunes about 9 MB. If you have a 250 GB hard drive once you subtract room taken up by your OS, applications and such, you'll probably have 180-200 GB.  SO, you could put 67 HD movies, 22,200 songs or 9,090 RAW photos.  Sounds like a lot, until you run out of space.  You're not just going to have all of one and none of the others.

So at Sansom Media Inc. our dilemma is having footage spread over too many hard drives.  Here's our workflow.  Every project gets four hard drives, yup 4.  Two Lacie Rugged (500 GB or 1 TB each, depending on the project), and two 3.5 inch bare drives, usually 1 TB each.  I'll use our last commercial shoot for Rogue Fitness as an example.  While in the field, we use the two Lacie Rugged drives along with ShotPut Pro.  We'll either offload cards throughout the day or do them all at once, again depending on the project.  ShotPut Pro transfers the data to each Rugged and verifies everything.  This is a lot better than just copying them using Finder and dragging and dropping.  Mac's will write invisible files when you do this and you don't want extra stuff like that with your footage.

Once we finish the shoot we put them in our Pelican 1050 case (it fits two perfectly) and we carry them with us in a silver briefcase handcuffed to our wrist (slight exaggeration).  When we arrive to the studio we use Final Cut 7's Log and Transfer tool to transcode the footage to Apple ProRes 422.  These are now the files we'll work with.  And make sure you separate your raw footage drives to two different locations.  The two bare drives are now clones of each other, one being a back up while editing.

We have so many drives now and we're constantly pulling clips from each to create content for clients.  We need a solution that will store and distribute all the transcoded footage to multiple edit bays, be RAID 6 and not cost heaps of cash.  Here are the options we've found:

DROBO S/FS/Pro/800i/1200i $400 for 4 TB or $4,000 for 16 TB (drives not included)

  • from USB 2.0 to iSCSI gigabit ethernet protocol but has a max of 125 Mb/s throughput, you can only 3-5 HD streams while editing.
  • so easy a caveman's dog can use it.
  • dual disk, RAID 6 style, back up
  • great for business that have a lot going on but files are not HD video. (ie. documents, point of sale, brochures, images, small videos for social media. general data storage and retrieval.
  • range of costs to buy and lower cost to expand

Promise Technology - Pegasus - $2,000 for 12 TB

  • uses Thunderbolt and has a max of 10 Gb/s throughput or 1,125Mb/s  That's 1.25 GB/second. FAST!
  • Drives aren't able to provide this speed yet (still trying to figure this one out).
  • brand new and not industry tested
  • no network RAID capabilities
  • superb for high end editing and animation.
  • average cost to buy and expand

G|SPEED FC XL - $11,000 for 16 TB

  • 4Gb/s fiber channel
  • all sorts of RAID configurations
  • network friendly editing
  • fast performance
  • high cost to buy and expand

It's important to remember that RAID 5 or 6 save the space of two drives for back up so 16 TB is more like 11 TB in a RAID 6.

Our final opinion at this point.  If you're a music junkie or just have large amounts of file storage needs look at a Drobo or Drobo S.  You can have all your music, videos, family photos, Office files and more and not worry about storage again for a long while.  You can also use  file sharing on the computers at home to access files on that drive.  Wifi works, hard wired Gigabit ethernet is much better.

If you're a small retail business give the Drobo FS or 800i a try.  For small production companies, do individual workstations dedicated to specific clients and maybe one bay for the smaller, less frequent group and when you need to access media from the other dives use file sharing on the computers.  You can even write a simple apple script that will tell the computers to automatically connect to each other when  turned on.

For the larger production studios that want to have multiple bays edit off of one server the G|SPEED FC XL (fiber channel, extra large) will probably be your best bet.

Hopefully this has been helpful and not confusing.  If anything it's a good place to start as you search for increased storage solutions that suit your needs.  We'd love to hear what you've found.  Comment and share it with everyone.

Also, check out the workflow video Chase Jarvis Inc. did.  It's another great resource.


bellor decisis said...

Todd, if you work with ProRes422 and are OK with a limit of about 75MB/s to your edit stations then a RAID array shared over gigabit Ethernet is another good solution. To set up a system for three editors you’re probably looking at $10K-$12K for a 16-bay array, a gigabit switch and a multi-port Ethernet adapter for your server. If you’re thinking about going this route you ABSOLUTELY want to look for a system with a proven track record for video editing and a company set up to give you support and assistance. Small-Tree Communications [ ] and Maxx Digital [ ] are two good places to start.

Please note also that a fibre channel system will require a fibre switch that will set you back a couple of grand, fibre channel adapters for each edit station and your server at about a grand apiece, software such as Apple’s Xsan or Tiger Technology’s MetaSan for each edit station and the server, a dedicated server and someone knowledgeable and experienced to set up and manage the network. Fibre channel is NOT for the newbie. An editing system for three editors will probably run you $20K-$25K or more.

For the money and the performance the new Pegasus Thunderbolt arrays look like the way to go for the small shop that doesn’t need access to a large archive and can manage without shared storage…assuming you can edit your projects on a MacBook Pro, an iMac or a Mac mini. Get a Pegasus for each machine, and an extra one to use to quickly copy files from one Mac to another. Think of it as sneaker net without the wait of FireWire 400!

The trip to the Grand Tetons looks fabulous. Happy Birthday!


Unknown said...

Todd, that last comment was from me. Sorry for the confusion: I logged in with the wrong account.

I hope you're doing well.


Douglas Learner said...

Me as in Douglas Learner...