Business Expansion and Server Solution // Dilemmas & Suggestions

If you're not a business don't tune out just yet.  There will be some valuable info for you too.  Whether you're a blogging mom, music/movie junky, a small retail business or a larger production studio you've all probably felt the growing pains of digital storage.  You're left with a few option, buy another external drive, try yourself or pay someone to install a larger internal drive, buy a new computer or maybe abandon technology and go back to sticks and stones.

Let's lay out a few numbers.  Some basic math (if you can call it that) 1024 MB = 1 GB.  And in kind, 1024 GB = 1 TB.  A typical HD movie from iTunes will set you back 2-4 GB each.  A song from iTunes about 9 MB. If you have a 250 GB hard drive once you subtract room taken up by your OS, applications and such, you'll probably have 180-200 GB.  SO, you could put 67 HD movies, 22,200 songs or 9,090 RAW photos.  Sounds like a lot, until you run out of space.  You're not just going to have all of one and none of the others.

So at Sansom Media Inc. our dilemma is having footage spread over too many hard drives.  Here's our workflow.  Every project gets four hard drives, yup 4.  Two Lacie Rugged (500 GB or 1 TB each, depending on the project), and two 3.5 inch bare drives, usually 1 TB each.  I'll use our last commercial shoot for Rogue Fitness as an example.  While in the field, we use the two Lacie Rugged drives along with ShotPut Pro.  We'll either offload cards throughout the day or do them all at once, again depending on the project.  ShotPut Pro transfers the data to each Rugged and verifies everything.  This is a lot better than just copying them using Finder and dragging and dropping.  Mac's will write invisible files when you do this and you don't want extra stuff like that with your footage.

Once we finish the shoot we put them in our Pelican 1050 case (it fits two perfectly) and we carry them with us in a silver briefcase handcuffed to our wrist (slight exaggeration).  When we arrive to the studio we use Final Cut 7's Log and Transfer tool to transcode the footage to Apple ProRes 422.  These are now the files we'll work with.  And make sure you separate your raw footage drives to two different locations.  The two bare drives are now clones of each other, one being a back up while editing.

We have so many drives now and we're constantly pulling clips from each to create content for clients.  We need a solution that will store and distribute all the transcoded footage to multiple edit bays, be RAID 6 and not cost heaps of cash.  Here are the options we've found:

DROBO S/FS/Pro/800i/1200i $400 for 4 TB or $4,000 for 16 TB (drives not included)

• from USB 2.0 to iSCSI gigabit ethernet protocol but has a max of 125 Mb/s throughput, you can only 3-5 HD streams while editing.
• so easy a caveman's dog can use it.
• dual disk, RAID 6 style, back up
• great for business that have a lot going on but files are not HD video. (ie. documents, point of sale, brochures, images, small videos for social media. general data storage and retrieval.
• range of costs to buy and lower cost to expand

Promise Technology - Pegasus - $2,000 for 12 TB

• uses Thunderbolt and has a max of 10 Gb/s throughput or 1,125Mb/s  That's 1.25 GB/second. FAST!
• Drives aren't able to provide this speed yet (still trying to figure this one out).
• brand new and not industry tested
• no network RAID capabilities
• superb for high end editing and animation.
• average cost to buy and expand

G|SPEED FC XL - $11,000 for 16 TB

• 4Gb/s fiber channel
• all sorts of RAID configurations
• network friendly editing
• fast performance
• high cost to buy and expand

It's important to remember that RAID 5 or 6 save the space of two drives for back up so 16 TB is more like 11 TB in a RAID 6.

Our final opinion at this point.  If you're a music junkie or just have large amounts of file storage needs look at a Drobo or Drobo S.  You can have all your music, videos, family photos, Office files and more and not worry about storage again for a long while.  You can also use  file sharing on the computers at home to access files on that drive.  Wifi works, hard wired Gigabit ethernet is much better.

If you're a small retail business give the Drobo FS or 800i a try.  For small production companies, do individual workstations dedicated to specific clients and maybe one bay for the smaller, less frequent group and when you need to access media from the other dives use file sharing on the computers.  You can even write a simple apple script that will tell the computers to automatically connect to each other when  turned on.

For the larger production studios that want to have multiple bays edit off of one server the G|SPEED FC XL (fiber channel, extra large) will probably be your best bet.

Hopefully this has been helpful and not confusing.  If anything it's a good place to start as you search for increased storage solutions that suit your needs.  We'd love to hear what you've found.  Comment and share it with everyone.

Also, check out the workflow video Chase Jarvis Inc. did.  It's another great resource.

UPDATED: Multi-Machine - Infield Log, Transfer and Edit Network with One Mirrored RAID

Today we're talking local server networking with computers using a Mirrored RAID set, Mac computers and Final Cut Pro.  This will work with Final Cut Pro X when it comes out.  In fact when you no longer have to transcode things your life will be much nicer.  The big thing here is hardware and how its set up.  Below I've included the copy for how to set up your server and best practices when using it.  Can anyone draw?  A diagram would be nice but not from my hand.  I have no creative drawing abilities in my bones.


Let me know how this works for you and if you have any suggestions to improve it.  I've used it and love it.  It can be adapted and modified to meet your specific needs and workflow.  I prefer to offload raw footage to the drives using ShotPut Pro and then work off a copy of the footage.  I'll have myself or lead editor working while I have the assistant pulling broll and whatnots.  When that isn't possible this workflow is exactly what you need.


There are images at the bottom of the post that you can download and use as a visual reference.


**** A somewhat important point not to forget with the Mirrored RAID work flow: when ingesting footage from the secondary (laptop), the editor on the primary (MacProo) needs to manually drag (import) the media that has been logged and transferred to the RAID into their FCP project.  Even though they are working on a FCP files that is the exact same name, the files are stored on two different machines (each machine creates it's own autosave of it's project) meaning the footage doesn't automatically poputlate on the FCP project NOT ingesting the footage.  Ideally with bins matching the file structure on the RAID. ie. Day 01 // 09_23 card 01_a // (media clips).****


-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_


NETWORKING: EDIT WITH TWO COMPUTERS FROM ONE DRIVE

For best results use a MacPro with separate FW800 Busses, front and back

Set up two hard drives — one on the front FW800 buss and the other on the rear FW800 buss. Open disk utilities and select each new volume and partition to one volume. Do each one twice, for good measure calling one shoot_region_a and shoot_region_b (short names, lower case and no spaces is ideal).

NOW select shoot_region_a and select RAID (seen in blue above). From the left column select the two hard drives shoot_region_a & shoot_region_b and drag them into the white box to the right. Youʼll want to tell it to automatically rebuild -- this is found in options... Once it has completed this, there will be only one drive on the desktop of the MacPro.

NEXT plug an ethernet cable between the two machines -- the MacProʼs have two so you can still connect hardline to the internet to upload content to an FTP or whateve -- and open System Preferences on the MacPro and select sharing under the Internet & Wireless section. Tick the File Sharing box. When you have a green light youʼre done. You can, if youʼd like, to name the computer as per your needs. ie. MacPro or Main.

Repeat this File Sharing process on the MacBook Pro. Maybe label the computer name laptop or something. Again, when you have the green light youʼre good to go.

Okay, now in the Finder bar on the laptop (or secondary machine) select Go and then Network.

Youʼll select the MacPro or primary machine and Connect As a register user. Youʼll need to know the admin login name and password for your machines. When you are connected youʼll see a list of the volumes available. Select the Mirrored RAID you created and it will appear as that (unless youʼve change your icons.

You should now be networked to your Mirrored RAID, having access to both machines.

WORKFLOW

This is a setup for direct Log & Transfer and not to be used to copy raw digital files to drives. See Infield Workflow for example.

I suggest using your secondary machine to Log and Transfer your footage and pull broll selects for your editor.

Create a FCP project file and name it exactly what youʼve name it on the principle machine. This makes FCP not confused when it is looking for files as the secondary and primary will be saving and naming everything the exact same. When you ingest footage youʼll want to create a new folder in your scratch disk for each day, ie. Day 01. Then a new folder for each card you offload and I like to label it with the time (military), card number and camera. 08_21 card 01_b. This helps you know when the footage was shot and who shot it.

You can use this same principle in FCP as you ingest and create a bin for each card ingested. Thatʼs youʼre call.

So create new sequences and label them as per your needs and begin cutting selects to those sequences. Once youʼre done pulling the specific work. Create a new FCP Project and label it accordingly. ie. Awesometown Selects. Save it to the Mirrored RAID, close it on your machine and then the principle machine can open it, copy the sequences or whatever makes the editor happy.

• It is wise to change the FCP Autosave to 10 mins from 45.

** This will work with any two Macs -- laptop or desktop or iMac as long as ʻhostʼ is a MacPro with two separate FW800 ports on the front and rear AND you have two separate licenses of Final Cut Pro.

*** Any other questions please ask me.

UPDATE: AF100 Offload Workflow

A little update regarding the AF100 Workflow post from last week.  Tim and I both received our Cube yesterday and great news, it works perfectly!  Oh, so awesome.  It holds four SD/SDHC/SDXC, and the micro equivalents.  Coupled with ShotPut Pro, winner winner chicken dinner.  Sweet as. Magical.  Stellar.  Unreal.  ...you get the idea.

No more late nights offloading footage in field.  Set it and forget it.  Are you shooting a ridiculous amount of footage everyday with three or four cameras using SD media and need eight slots?  Well when Tim gets back from Australia we'll put both Cubes on one laptop and see if the bus power and computer can do it.

$99 for ShotPut Pro and $68 for a rushed Cube could be some of the best money I've spent.

Panasonic AG AF100 Workflow

Let me preface this post with a short scenario.  You're filming with your Panasonic AG AF100 -- or maybe even two -- for the last 12 hours.  It's now the end of the day and the last thing you want to do is go to your hotel room and offload a mano's (spanish) worth of cards.  Normally you would sit down with your laptop, hook up your two bus powered drives, launch ShotPut Pro and feed in one card at a time.  Fastforward several hours and come 2am you're done.

Well my hero, Tim Irwin, asked me the other day if I knew of any multi-slot card reader for SDHC/SCXC cards and I said, I just so happen to.  But I've been so busy I haven't done much research on them.  They're made by a Chinese comapny called Elecom.  And they make various version of them.  Some with two SDHC slots and a few USB slots and the one we like -- what I call the SDCube (Elecom MR-C27 SD Card Reader) -- with four slots.  It comes in black and white and has six sides, hence the cube.

I sent Tim the link I had and he has since found a website, Geek Stuff 4 U, that distributes them AND the site is in english.  The SDCube runs for 3,980 yen, about $50.

We've both got one on order and will let you know how it works.  Tim is taking one to Australia this week and will put it though the paces.  Ideally, you hook up your SDCube, load your 4 shot SDHC cards, and tell ShotPut Pro to copy the four cards to the two drives and then you fall asleep while it does all the work.  We'll see if it really works that good.

Elecom makes a bunch of other cool little card readers that may better suit your needs.  Check 'em out.

Information Sheet

• MAKER/MODEL -- Elecom/MR-C27 SD Card Reader
• INTERFACE -- USB2.0/1.1
• POWER SUPPLY -- USB Powered
• TRANSFER RATE -- 480Mbps
• DIMENSIONS -- 36.0 x 36.0 x 38.5mm