Longterm Storage - Servers or Multiple External HDs - What do you use?

[DenisM]

Since moving to MFDB a year ago, storage is becoming more and more of a issue for me.

Yesterday, for the first time ever, I got a frantic call from a client asking if I still had images shot mid-2007 (on a DSLR) as they had lost the DVDs and had never copied the images to their server at the time. Fortunately, I did. Twenty minutes later, two copies were picked up by a bike and were on their way. Very good for cementing customer relations! 

I have three questions..........

1. Do you tend to store the raw file (with Flex/Phocus edits embedded), the fully edited TIFF or both?

2. Do you use a server arrangement or, given their incredible low cost now. duplicate/triplicate external HDs?

3. What make of storage (server or external HD) do you use? I'd like to have expert opinion on reliability etc.

For what its worth, I used to store TIFF and edited RAW. However, since much of my work involves minimal tweaks in Photoshop, I have started only storing the edited RAW files reasoning that I can always save out another set later if required - as happened yesterday.

On the storage issue, rightly or wrongly, I've gone the external HD route. Two x 500Gb external HDs are dirt cheap now and take up very little space.

What are the rest of you using?

Thanks.

Denis

[DenisM]

Derek,

Thanks for the tip on Western Digital. Will only use WD external HDs from now on.

I am almost invariably shooting single and multi-shot to a laptop, very rarely to card. At the end of a shoot, I copy the lot onto a WD My Passport USB drive - fantastic pieces of kit!

This has the benefit of:

a. Not having to power up the laptop when I get back to base.

b. Giving me a backup copy of the day's shoot.

This is all done in the time it takes to put away the gear.

On my return to base, I copy the RAW files from the WD Passport to my main editing machine. After editing, I copy the edited RAWs to two external HDs.

I catalogue by client. i.e. On, for example, an external HD of RAWs there will be a folder for each client and, within each folder, dated sub-folder containing the RAWs from that date. It has worked so far.

Denis

[mauro risch]

I've been storing all my digital and analogical life in different medias since the early 90's.
First there were the Syquest discs. Horrible and slow. The Iomega zip disks replaced the
Syquests for a cheapest and more reliable media.
In 96 I've started to backup everything on CD-Rs. The cost went really down on the ratio per MegaByte
saved. I was told that the CDs would only be guaranteed for 5 years. So, after a long time using CDs,
with not one problem at all, I've moved to DVDs. From 6.000 CDs I was able to reduce my library
to something under 1200 DVDs. I'm already thinking on the transition from DVDs to Blue-Ray discs.
I don't trust in a Hard Drive as my only backup for my library. I also have lost a few Seagate Barracudas.
In the 90's I would pay $1 per megabyte recovered. A bit too expensive.
I catalog every disc using ExTensis Portfolio. The counter total now is 285.462 images. Including all the
scanned chromes from my previous life with film (35mm to 8x10').
I do keep all my RAW files as well as my PSDs, with all the layers unchecked(hidden) to minimize file size.
I hope this will give you an idea of a different method of archiving your library.
Cheers

Mauro

[alexkent]

Mauro

I do keep all my RAW files as well as my PSDs, with all the layers unchecked(hidden) to minimize file size.

??

how does making layers invisible reduce the file size?
saving without 'Maximising Compatibility' (including a composite version in a layered file) will reduce the filesize but i don't see how making layers invisible will.

alex.

[Monty Rakusen]

I'm just investing in my first drive archive system.

I'm getting the following

Wiebetech RTX200 enclosure from the Digital Plumber.co.uk

and 4X

http://www.microdirect.co.uk/(36576)Samsung-1000GB-Spinpoint-F-SATA-II300-7200rpm.aspx

I have been archiving FFF to DVD for several years and its horribly slow with a lot of failure rate. Also I get about 250 (i've not counted them) DVDs to a plastic crate and my study in the house is beginning to fill up! I dread the phone calls asking to find a picture I took 2 years ago! Tiffs are written to DVD and filed in a fire proof room which used to be my dark room and I will continue with this...its only fff that I need to store on drives.
The system will be fully manual. when I finish for the day I will back up onto the two drives in the enclosure and then a drive will come across from the house and receive a copy. This will happen untill they are full then I will put in the house copy and a spare drive and make a duplicate copy, one for the house, one for the studio. Then I buy two more drives for about £120 and start again. These are each 1000gb roughly equivalent to 250 DVDs each. The cost is quite a lot but just think of the time and space I will save!

If this is a success I may do the same with the tiff.

Oh and I need a esata card.

The sun is shining in Yorkshire but I'm off to Ireland all week!

Kind regards

Monty Rakusen

[mauro risch]

Hi Alex,

You should try it first. One PSD file with 10 layers(for instance), saved with all of them checked(the little eye on the left, active) will be a much bigger file than the same PSD with all the layers unchecked( the eye on the left, inactive).
You might be surprised with the difference in the file size. I normally get something around 30% more disk space, this way.
Sorry for going a little bit out of the subject.
Cheers.

Mauro

[alexkent]

You should try it first. ...
...I normally get something around 30% more disk space

5 min on a sunday afternoon test;

Original PSD, with layers visible and 'Maximise Compatibility' turned on: 476.1MB
Layers Visible but Maximise Compatibility turned off: 328.4MB
Layers Invisible, Maximise Compatibility turned off: 328.4MB
Layers Invisible, Maximise Compatibility turned On: 476.1MB

?

i can only suspect that your space saving is coming from turning off maximise compatibility at the same time as hiding all the layers.
i'm only really questioning this because i often have layers in PSD files which are not used in the final composite (but were used to get to the final, so if i need to make changes these layers become useful).
when archiving files i mark these unused layers in some obvious way, amend the layer titles with NOT IN FINAL for example, but having a correct set of layer visible/invisible switches is important to me!

[alexkent]

oh sorry, i didn't explain.

Saving a Photoshop document with 'Maximise Compatibility' enabled includes a flattened version of the image alongside all the layers.
The reason to use this is that many applications which claim to be able to open Photoshop documents, can't actually understand layers, so they just read the composite image.

turned on:
- bigger file
- longer save

turned off:
- smaller, faster files!
- opening in applications other than Photoshop version 4 upwards, may not work.

whether its a worthwhile trade off depends entirely on your workflow.

[pbeaurline]

After years of trying every sort of backup device and routine, and experiencing (almost, knock on wood) every sort of data loss, I have finally settled on a system that lets me sleep at night, doesn't require much intervention, and keeps our files easily searched and retrieved.

It basically consists of an onsite fileserver with multiple drives for live access to our entire image collection, and an offsite server with removable drives for permanent backup. There are 2 active drives on both servers for overnight backup of files currently in use - active jobs and the most recent file archive drive that is still being added to. The offsite server is in an unattached garage, for fire protection. Active jobs, when complete, are moved to the archive and cataloged, and a dvd is made. The dvd, while seemingly overly redundant, has rescued us a couple of times from simple human errors.

All of this can be put together surprisingly cheaply. Gigabit ethernet is cheap now, and large ATA drives can be put in almost anything. For several years, I gaffer taped or otherwise rigged drives into hand-me-down pc boxes, until they got too hot. I recommend now the stand alone boxes that will hold 6 or more drives.

It works for me...

Philip