Monday, 8 November 2010

Aperture Workflow with ReadyNAS

Over the past few months I've had a number of requests from the ReadyNAS forum & elsewhere for info on how I implement my Aperture Workflow with a ReadyNAS device. This is a short overview of how I work.

I use Aperture primarily on my iMac that sits in the same room as my Ultra-4 and NV+ NAS, wired to the same GbE Switch, but also on my MacBook when travelling. I have adopted a Referenced Library approach that maximizes my Aperture performance while enabling me to store images on the NAS that can then be accessed by other applications, as follows:

Edit: I have completed some new performance testing of hosting Aperture Libraries directly on  a ReadyNAS via AFP. See end of the post for an update





iMac Workflow


The Aperture Library itself is hosted on the local HDD of the iMac - this ensures the fastest performance when importing and editing photos.

In general, the workflow I use is as follows:

Import Images » Coarse Image Management » Relocate Masters to NAS » Backup Library » Photo Editing

Import Images:


When importing images, I import them directly into my library - ie at this point I am working with a Managed Library for those new images. I do this as it is the simplest, fastest option giving me direct access to all photos on my local disk. I don't wish to worry about where the photos are or referencing files at this stage - I'm more interested in the images themselves and getting them from my camera(s).


In Aperture 3 there is a new "Backup" option in the Import panel - this option allows you to make a copy of all the imported images to a secondary location immediately upon import. I have that permanently set to point at a backup folder on my NAS with a preset to create a simple folder structure based on photo date. (See opposite) This way, even though I'm working with a "managed" library at this point for the newly imported images, I maintain a second copy of those imported images on my ReadyNAS should I screw up the import or subsequent steps and can clear my memory card(s).

Coarse Library Management:


At this point I have a new project within Aperture with a dump of newly imported photos - some good, some not so good - but all already backed up to my NAS. So it's time to do some basic photo management: Reject the clearly bad images I really don't want; organize the remaining images into separate projects and/or albums; perhaps even a little processing on some key images that catch my eye (eg exposure & color correction), geotagging, adding a few comments and tags  that immediately come to mind.

The net result of this step is that I have all the photos organized, and ready for more detailed processing / rating / tagging etc. But first…

Relocate Masters:


This is the point where I move these newly imported images to be part of my Referenced setup by storing them on my ReadyNAS rather than within the Aperture Library. Simply by selecting the projects, I can go to the menu File » Relocate Masters… and this enables me to move the master images directly to my NAS.

In the dialog that appears (See opposite) you can specify a custom subfolder format for storing these images - by making a custom preset and using that, I can select a master folder (that Aperture always defaults to) and organize my images in appropriate subfolders. I don't have to play about specifying anything else.

In my case subfolders are always based on the Aperture Project name & date the photos were taken.

So relocating the image masters becomes:
  • Select Projects
  • Go to: File » Relocate Masters... (See Dialog Opposite)
    • Aperture dialog always opens to the same master folder
    • I double check that the correct preset is selected
  • Press the "Relocate Masters" Button

Backup Library


At this point I now have all my photos on the NAS, in a simple folder structure where I can access them for backup or from other applications and I have a secondary copy elsewhere on the NAS made at import. All that remains is to backup the library from the local HDD, which is a one click process to update my Vault(s) setup in Aperture and also stored on the NAS, while overnight both the Vaults and  master images are automatically backed up to my backup NAS (NV+) as well as to my offsite Crashplan account.

Summary


With backups complete, and photos on the NAS I can now continue with the more valuable process of actually working with the photos.

Comparing the above  to a fully managed library flow - it really just consists of one extra step (Relocate Masters) that, once setup, requires only couple of mouse clicks and a brief wait while photos are archived to the NAS. But it adds significant benefit in being able to leverage my NAS for both backing up and sharing those photos with others and other applications

MacBook Flow


The iMac flow works fine when I'm at home, but when traveling I have Aperture installed on my MacBook so that I can load images from my camera to the MacBook for backup. In this instance, as I have no NAS available, I find it easiest to work with a completely Managed Library flow.

In the first instance I perform the same 2 initial steps as per my iMac flow - Import  & Coarse Library Management. I still create a backup during import, but to a USB drive simply as a precaution. I also make a vault backup to the same USB drive, again as a precaution. (I have a pocket sized HFS+ formatted 500GB USB drive that serves well for this)

When I get home, I use the Drag & Drop capability of Aperture to export the new projects from my Macbook to the iMac as follows:
  1. Drag the new Projects from Aperture, to the Desktop of my Macbook (or to a ReadyNAS Share). This creates a new library containing only the new photos, including the master images and any edits/changes I may have made

  2. From my iMac, I connect to the MacBook/ReadyNAS and drag the newly created library into my iMac library. This gives the option to merge that library into the existing one.

  3. At this point I have done the equivalent of the first 2 steps of the iMac flow (Import & Coarse Management) and so can relocate the master images as before and continue with my iMac flow

Conversely, if there are images in my iMac library that I wish to take with me on the MacBook, I can reverse the above steps. When dragging a project from a Referenced library, Aperture will ask if you want to consolidate the images also; I use that so that the macbook gets a complete copy of library and masters - it's far easier than trying to deal with a referenced library on the MacBook which offers no real advantages

Summary


Overall I find Aperture's ability to work with Referenced Images to be very powerful and very easy to use, but it is not always the best option as described for my MacBook based library. Also, the ability to drag & drop projects into and out of Aperture may often be overlooked but makes a multi-host work flow like mine extremely simple in practice, without the need for complex 'synchronisation' of libraries.



13 Nov: Update on Aperture Performance with Network Hosted (AFP) Library


I am involved in some new performance testing related to using Apple applications with data and libraries hosted via AFP on a ReadyNAS device. This testing has shown a significant improvement when hosting Aperture Libraries on the ReadyNAS.

With Aperture 3.1, OSX 10.6.5 and RAIDiator Firmware 4.2.15, accessing Aperture Libraries from an AFP Volume is significantly faster than I have previously experienced  - with smooth scrolling and a very responsive interface. There is some lag when rendering images, but the responsiveness of the interface makes this very tolerable. Exporting of projects to the NAS and the creation & use of Vaults performs acceptably also. (Note: my previous testing was only with Aperture 2 so the new library format used in Aperture 3 may also played a part in the improvement)

Overall, while hosting a full Library on the NAS is certainly slower that hosting as I describe in this post, it is very much a usable setup and makes sharing Libraries between clients via the ReadyNAS a valid option.

15 comments:

  1. Thanks for sharing your workflow. I've been using a ReadyNAS NV for a few years and skipping the backup/local import. I just import and point the files directly to the NAS (referenced). This seems to work fine, however, on a fairly frequent basis when I open Aperture, it gets confused and doesn't see the ReadyNAS as online (even though it's mounted on the desktop). I then have to go back through the large portions of the library and reconnect the broken referenced files.

    Any idea what might be causing this? The ReadyNAS is plugged into a Gigabit hub then hardwired throughout the house. It's using RAIDiator 4.1.6

    The reason I avoided using any Managed photos is that the size of the library would be crazy large (there are over 30,000 referenced photos in the library)!

    I regularly rebuild and repair the Aperture Library, so I think it's clean.

    Thanks for any insight.

    Angelo

    ReplyDelete
  2. Hello Angelo

    Just for clarification in case it was not apparent - all my photos are referenced on my NAS; I only use a managed library as a temporary mesaure to speed up the import process as described in the post.

    I've seen something similar but not quite the same. With Aperture 3, when I open the application but do not have my NAS share mounted then Aperture will (correctly) show those images as offline. If I then mount the share Aperture *still* shows them as offline and I must restart Aperture to correct this. I did not have this issue with Aperture 2 which would detect the share being mounted and so this is a regression that has yet to be fixed. Maybe a restart would help you rather than fixing the links?

    The one other suggestion I would make is that you consider upgrading your NAS firmware - v4.1.7 has been out for many months and, while you don't say specifically the protocol you are using, both CIFS and AFP services were updated in 4.1.7. Also be careful if you have both AFP and CIFS enabled on your NAS - Aperture will distinguish between the 2 even if you mount your shares to the same mount points. So if you access your share via CIFS in one session, and then try to access via AFP in the next - Aperture will consider the referenced location as different (or did - haven't actually checked this in a long time as I don't use CIFS)

    ReplyDelete
  3. SPhardy,

    Thanks for the reply. I think it could be related to the way Mac OS is connecting. The problem is I need to keep CIFS, NFS, and AFP all running. For general windows and streaming access the regular stuff, but when I want to back up or copy a mac bundle file, there are funky naming issues that will error out if I don't have AFP running.

    Sometimes I can "trick" Aperture into realizing the volume is mounted by removing it and re-mounting it while Aperture is open, other times it just ignores this. Frustrating, as there dosen't seem to be any rhyme or reason to when it bugs out and when it jumps back to all the referenced images "magically" reconnecting on it's own!

    If I force connect via SMB (command K), then drag this alias into a finder window toolbar, will that force Mac OS to connect this way every time? Maybe reconnect all broken images after this and see if they stick?

    I hope Lion cleans up networking a lot - it's been a mess for a long time.

    Angelo

    ReplyDelete
  4. I have strong suspicions that Aperture is getting confused with the various mounts you are doing. When presented with a dialog box for opening or saving files etc. the way OSX presents mount volumes makes it very easy to mix which protocol you are using - I've experienced the same as I use NFS a lot also.

    I would suggest that before you start Aperture, you carefully make sure that the share containing your images is mounted via the appropriate protocol and mount point. When I've done that I have never had issues with Aperture not being able to find my referenced images

    ReplyDelete
  5. Looks like Lion just makes it worse! Thought I was going a little crazy there for a bit, but then found this forum thread:

    https://discussions.apple.com/thread/3192306?start=0&tstart=180

    Doesn't matter what brand of NAS you are using, ANY networked volume seems to break connections to referenced masters. Ugh.

    ReplyDelete
  6. Yes - I've hit this issue also. Not nice, but I do expect Apple to fix it as the issue affects their own products too

    ReplyDelete
  7. Apple has released OSX 10.7.2 and Aperture 3.2 which, together, appears to address the issue of breaking connections to referenced masters stored on network volumes.

    ReplyDelete
  8. Can I get a little more help on this setup... I just got the readyNAS Ultra4 and set up a 1TB iSCSI volume. But I do not see how to mount the volume on my macs... I see all of the other things like AFP and such. But I want an iSCSI so I can format and use as a Mac volume for Aperture files...

    Please help!

    ReplyDelete
  9. OSX does not include native iSCSI support. You need to install an iSCSI Initiator.

    These are available for purchase from Atto (http://attotech.com/) or Studio Networks (http://www.studionetworksolutions.com/)

    ReplyDelete
  10. Once I'm in iSCSI mode I can do everything you advise above? and I will be able to see and use Aperture from all of my macs?

    ReplyDelete
  11. Yes - though nothing I describe requires iSCSI. For example, referenced images can be stored on any network device and accessed by standard protocols like AFP.

    ReplyDelete
  12. Is AFP the better way to go since I'm all macs?

    ReplyDelete
  13. It depends on your overall requirements. iSCSI and AFP both allow you to access network storage, but in very different ways and each has benefits and drawbacks.

    I'd suggest researching both thoroughly before settling on which to use

    ReplyDelete
  14. Have you tried using Dropbox to hold the library? That way all macs can point to the same file...

    ReplyDelete
  15. I have considered using Dropbox but decided against it - the large data changes when importing to Aperture and relatively slow internet uplink data rates would (IMO) make this method far too slow to be practical.

    As Aperture supports simple merging of libraries, I find it easier/faster to do that over a LAN connection as described, even if it requires more manual effort. Also it avoids any possibility of data corruption due to accidental simultaneous access of the library from multiple macs

    ReplyDelete