Storage solutions

[NickT]

On another thread,

So anyone got any tips on how to deal with all those megabytes?

I currently favour RAID5 boxes from a number of different manufacturers. RAID 5 needs four drives one of which is used for parity. If you put 4X4TB drives in a RAID 5 box you'll get 12TB of storage. With RAID5 you can lose one disk and not lose any data. The box will alarm and prompt you to replace the failing drive then rebuild itself. For that reason I always keep a spare drive of the same capacity at hand.

I currently run 4 of these RAIDS. Shoot to one, backup nightly to two more then a further box which is rotated off site every couple of weeks or so.

If the studio burns down then I would lose 2 weeks with of work (or however long it was since I rotated the offsite). It is unlikely that a client would lose data however because I will do a manual offsite backup of any job that hasn't been delivered.

I also run clones of the boot drives on my admin machine and workstations so if an internal (boot) drive fails I can just restart off the clone, i.e. not have to wait hours for software to re-install etc.

Be interested to hear what others are doing.

[Chris Gahran]

If you need to sit on line I guess Raids are a great solution.

For me, jobs under 40GB get burned to DVD and put in the job envelope. Slim jewel cases taped together in pairs stand upright in the job envelope and five pairs of slim case DVDs is not very thick.

Jobs over 40 GB get burned to bare hard drives plugged into a Hard Drive Dock. Currently copying to hard drives using FW800 but will move up to eSATA by putting PCIe eSATA cards in my MacPros.

I rarely have clients request legacy images. I charge for retrieval and delivery so most clients only ask once. I also charge for archiving images so all my clients usually say "no thanks we have a copy." (I have a client that has about ten LaCie portable drives [the orange edged drives] foam cut in a large Halliburton silver case for traveling to China when it's printing time. Impressive.)

Note that if you charge your client for archiving, that is a legal agreement and you are responsible for those images.
 
David Grover years ago showed you can Zip fff Files.

Save TIFFs using Image Compression LZW to reduce the size of a TIFF by two-thirds. (That is 60 -> 20.)

See if you can work with 8-bit files instead of 16-bit files.

At the end of retouching a PSD file throw away any unused layers to reduce the size of a final layered PSD. (I keep a layered working file for myself.)

Keep good notes as to where images/jobs are stored. It will save your sanity next time you're looking for a file delivered fourteen months ago.

Chris

[NickT]

Keep good notes as to where images/jobs are stored. It will save your sanity next time you're looking for a file delivered fourteen months ago.

Chris

I use disktracker for this, it's old and clunky but it does help me find stuff.

[Hassilistic]

For me, jobs under 40GB get burned to DVD and put in the job envelope. Slim jewel cases taped together in pairs stand upright in the job envelope and five pairs of slim case DVDs is not very thick.

Jobs over 40 GB get burned to bare hard drives plugged into a Hard Drive Dock. Currently copying to hard drives using FW800 but will move up to eSATA by putting PCIe eSATA cards in my MacPros.

I have been where you are Chris and would highly recommend the following:
- Use GOLD DVD's http://www.mam-a-store.com/golddvd.html always, the readability and shelf life is much better.
- Use Enterprise class HDD such as Hitachi HGST.  I never had one crash for last 6 yrs You can pick a 1 or 2TB for really low prices in amazon.

I love my Hitachi's so much 2 years ago I followed them into their new guise G-Technology and got myself an 8TB G-RAID drive http://www.g-technology.com/products/g-raid-thunderbolt which I daisy chain up to six in Thunderbolt configuration.  A second failsafe backup goes on my office network Synology RAID server.  The latter is moderately expensive, but on a whole different plain altogether and worth every penny for what it can do all by itself.

[PNWMF]

So much to say, would like to listen more first.  A storage solutions is about workflow, copies made, archive and backup.  Disks and boxes are in there, but part of a larger solution.  Current size and annual growth also plays into it.

RAID5 is eh, especially given the large data sets in use.  Branded boxes are eh, with Synology being pretty good, others being eh to really really bad. Filesystems currently in use really aren't ideal, given their ability to silently corrupt.

Enterprise disks v consumer disks is a proven fallacy:
https://www.backblaze.com/blog/enterprise-drive-reliability/
https://www.backblaze.com/blog/hard-drive-reliability-stats-q1-2016/

[Chris Gahran]

I have been where you are Chris and would highly recommend the following:
- Use GOLD DVD's http://www.mam-a-store.com/golddvd.html always, the readability and shelf life is much better.

Please see : http://www.digitalfaq.com/reviews/dvd-media.htm

I don't know their level of expertise but they do seem very knowledgeable. Their preference is Verbatim DVD-R and only one type of Verbatim manufactured.

At one time Apple sold Apple branded DVDs but stopped selling blank discs. I called Apple to confirm they ceased selling blank DVDs and when they confirmed their cessation I asked what should I now buy? Without hesitation I was told to start using Verbatim.

Burn at 4X, allow Verify to run and you'll have "archival" copies of your files. I have fifteen year old Verbatim DVDs that read without problems. I have ten year old hard drives that suffer from "data rot" which makes an image file not readable by Photoshop. (If I must have the file I dig out the DVD.)

Chris

[NickT]

I don't think anything is foolproof, Ive had old cds and DVDs, (so called archival discs) fail... The stuff that I need is currently on 4 different RAID5 boxes plus some older HDS. As I need more capacity I just add another box and copy the archive onto the newer bigger box. Granted I'm not a fashion photographer so the volumes are manageable.

[Chris Gahran]

Nick,

I agree nothing is foolproof. (BTW, "fool proof"!?) Did you burn your discs at 8X or 16X? Write errors increase exponentially as burn speed increases. Burning discs faster than 4X while satisfying is unwise. Just my advice based on my readings and experience.

Chris

[NickT]

Hey Chris
Yes I remember the advice around slower write speeds.
What are other people doing?

[Chris Gahran]

The website I referred to earlier (http://www.digitalfaq.com/reviews/dvd-media.htm) has a wealth of information. Most info is about six years old but there are recent updates which implies to me that DVD recording is a mature process.

I was seeking recommendations for burn speeds and learned that 6X is the fastest continuous burn speed. Faster burn speeds are pulsed bursts of the laser to gain speed. They recommend writing at half the rated speed of the disc but for "archival discs" they recommend using 4X.

They rate gold discs worse than best dye based DVDs.

Chris

[Hassilistic]

Thanks Chris that is an interesting link with a lot of useful information.  For my part, I haven't had any trouble at all, perhaps its because I have only used the best though a fraction more expensive, but thats what you get.  I am talking about Mitsui also marketed in USA as MAM, they are even the Medical Industry standard for Archival data.  Can find more info here:

http://www.conservationresources.com/Main/section_6/section6_12.htm
http://www.mediasupply.com/mama.html
https://www.mam-a.com/24kt_gold

NickT> to answer your question.. I have used only Plextor recorders/players since 2001 and I swear by'em, CD's / DVD/ and now Blurays, simply no other comes close, when I lost the readability on some of my older CD's captured between 1997-2001, only Plextor could extract the data, not even my pioneer recorder.  Since then I have learned to Burn on Mitsui discs or the German brand Intenso, using only Plextor recorders with its own propriety software (very simply interface) and more importantly in real time speed, meaning 1X, I kid you not.  All the music industry early CD records that came out in the 80's work till this day, and out preform the ones that later followed in the 90's and years that followed, as higher recording speeds became possible and deteriorated the quality all together.  Those early recordings are highly sought after.

* On another note, new media such as SHM-CD is the bomb once it becomes available for personal recordings will make normal CD's obsolete.  I own a few recordings and its unbelievably good.
http://www.positive-feedback.com/Issue39/shmcd.htm

Cheers,