In an all electronic workflow, producers and directors cannot rely on video tapes or film as an emergency backup. Previously, a still photography or filmaker would keep their negative or positive, and strike new prints as needed. With the advent of digital photography, and soon in affordable digital filmmaking and video production, there is no physical backup, but only the bit buckets, which when arranged properly would describe a vast landscape, a human face, or an airplane landing.

In the still photography realm, image makers have embraced the new workflow, because they realize the potential for new creativity:

Every copy of the image is an original
Since the image is already digitized, it is easily shared
Digital photos are more easily manipulated and cropped
The arduous process of scanning film is eliminated

However, since the bits become supreme, over the film strip or video tape, special storage considerations must be analyzed and scrutinized. Also, still photography describes a single moment in time, while moving images records that same scene but at 24 to 60 frames per second, so a 2 MB still frame file, can become a 48 MB file per second for video, and with Panasonic DVCPRO HD codec 60 GB per hour. How does one store that much data for archiva purposesl and retrieval in the future?

Today, we have current technology, and very soon the future will be now (Blu-Ray Disc, holographic storage, VXA 3, etc). It used to take a simple trip to the Comdex trade show in Las Vegas, to get a bead on storage, but for all practical purposes Comdex has ended, and may never return. Instead, we need to look to manufactucturer events and niche events like Macworld Expo, Storage Networking World, or AIIM Expo.

To manage and harness the growing terabyte requirements in entertainment consider these storage solutions:

* prices from summer 2005 survey

During the Panasonic CES 2004 keynote address, Andrew Nelkin, Vice President of Panasonic Consumer Electronics Company first hinted that by the first quarter of 2006, Panasonic would produce an affordable high definition camcorder using solid state flash memory devices (secure digital memory card or SD cards). The magic date became Torino 2006, the home of the next Winter Olympic Games, where Panasonic traditionally has a commanding sponsorship.

Fortunately, Panasonic did not wait until CES 2006 to announce the new products, but instead had a very large presence at NAB 2005 for the professional DVCPRO HD AG-HVX200 rollout, and IFA 2005 for the consumer MPEG-2 based camcorder rollout. Both products are expected in the fall of 2005.

Unlike traditional portable video capture devices, these new camcorders have no need for magnetic tape. Video tape has it litany of problems including jams, dropouts, snags, tape breaks, curl, and video tracking along with environmental issues like susceptibility to humidity, magnetic field erasure, and simple aging. The era of the cassette is over, and the opportunity to be creative faster and more robustly has arrived.

The first generation product, with a 2 GB to 16 GB starting storage capacity, will need a new workflow. The recording medium, a reusable SD memory device, once filled, needs to be put somewhere, and a storage area network (SAN) or RAID device would be ideal to protect the invaluable electronic image capture. As the era of solid state recording dawns, notably other backup devices are either already mature or nearly ready to archive the DV, DVCPRO HD or MPEG-2 recording.

Today, DVD+R makes an entry level archive medium, with a 4.38 GB (17.5 minutes in DV format, 11 minutes DVCPPRO HD 720p/24) storage platform. DVD+R is cheap, reliable, and long lived. On the horizon, look to Blu-Ray Disc with a capacity of about 25 GB, five times the capacity of DVD+R. Also on the cusp is holographic storage at some 300 GB a cartridge.

panasonic AG-HVX200 camcorder P2 memory slots

panasonic AG-HVX200 camcorder side view

There’s been a dream of mine to have multiple cheap machines networked in a way to form a hive mind, so commands on one distribute to all and work together. This is loosely called grid computing. Imagine 3 iMac G5′s, Power Mac G5′s, or PowerBook G4′s networked together to work as one. The future is now!

At NAB 2005, Apple announced Final Cut Studio, which allows distributed Compressor rendering/encoding, so through a dedicated gigabit ethernet or firewire infrastructure, you combine a few machines into a working super computer. Mac OS X Tiger has as it’s base a technology called Xgrid, which is a protocol for clustering machines together out of the box.

The secret to success is the gigabit ethernet connection running at 1 Gbps or 128 MB/s. so now in the new computing model, the CPU comes second to the network connection. In a few years 10 gigabit will be standard, passing 2 Gbps or 4 Gbps fibre channel.

So now that you can grid the machine together, how about storage? Apple has an Xsan solution. Xsan costs just below $1500 a workstation to get the computers to share storage ($999 Xsan seat license, $499 fibre channel PCI-X card). A much more elegant and economical solution, that works on panther 10.3.5 and above is iSCSI.

you remember SCSI? all macs had it before ATA. Storage engineers never gave up on SCSI, and are reintroducing the SCSI protocols, but over gigabit ethernet! On a more modern mac, you already have the port, or can easily get a $20 network PCI card (recommended approach to separate the storage network from the IP network). The beauty of iSCSI it can be entirely software based, so yes you get a performance hit, but imagine sharing all your hard drive on a small gigabit network using software SAN. This is revolutionary. Current vendors that support the Mac are ATTO technology and Studio Network Solutions. These companies are just the beginning to a wonderful marriage of storage, grid computing, and the Mac.

apple Xserve G5 cluster nodes

Apple is moving the Mac to an Intel chipset. Until that day in 2007, where all new Mac machines will be Intel inside, expect many surprises. The glue to any good chipset deployment is not only the underlying semiconductor design, but the application environment to write to the hardware, or application program interface (API). Apple Xcode is the technology that drives the software engine. When the multicore comes out with hyperthreading, then we’re talking a good move toward Intel. The chip will have at least two onboard processors, and two virtual processors. All eyes are watching the Intel Fall Developer Forum, to see the processor roadmap.

Along with advanced processors, comes 64-bit computing, which allows access to more than 4 GB of RAM (the 32-bit limit). Today’s Power Macs can hold 8 GB of RAM. I was told by Adobe, that Creative Suite 2 needs around 2 GB of RAM for all it’s applications, and Apple Motion definitely needs all it can get (4 GB at least). the future is always in flux.

At the end of spring and now into the summer, the GPU market has reached an inflection point. Unlike the 90 nm and 65 nm semiconductor barrier, both NVIDIA and ATI chug along with product announcements and advancements. ATI announced at Computex CrossFire, a method to bond multiple PCI/AGP video cards together for near double the performance. NVIDIA had already announced a similar solution they called SLI. NVIDIA stirred the pot again with the GeForce 7 series. if the GeForce 6 series did not scream enough, we have the latest and greatest with more transistors and higher performance.

NVIDIA spurs on interest in its product and introductions with unique characters, usually buxom women scantily clad to attract the gamers and young folk of the planet. if you recall NVIDIA creates these female persona to demonstrate the real time rendering of characters in cinematic motion. all in all we have luna, nalu, dusk, and dawn (summary below). Will these chipsets make it the Mac?

GPU NVIDIA history

G70
GeForce 7800
luna demo woman

NV40
GeForce 6800
nalu demo woman

NV35
GeForce FX 5900 series
dusk demo woman

NV30
GeForce FX 5800 series
dawn demo woman

NVIDIA digital production pipeline

Almost since the beginning of desktop computing, the ability to share has been essential. Flash forward 30 years, and not only are computers networked, behold the pervasiveness of the internet to every workstation and laptop, but soon every computer will have shared storage. The simplest form of sharing would be networked attached storage (NAS), where you use built in networking protocols (TCP/IP, SAMBA, NFS, AppleTalk, etc.) to mount a remote volume over the internet or over a local area network (LAN) to a client system. You can now parcel files, using the file sharing aspects of the operating system.

A more robust system in the NAS space, uses an intelligent controller, or server (Mac OS X Server, NAS head, etc.) to deliver storage connectivity.

Companies like NetApp and BlueArc have mastered the front end of the storage array, and allow on the fly configuration, modeling, and allocation of storage resources as the needs of the enterprise change. Certain applications, like databases, require block level access to storage devices, as if they were directly attached to an individual computer. Application performance may suffer if the program is looking for block level access, but only sees NAS, however, in general, NAS is fast and is a completely viable storage platform.

Be it a NAS environment or a storage area network (SAN), most likely SAN is the storage subsystem architecture anyway. Currently there are three ways to attach a SAN to a workstation or workgroup:

Fibre Channel
Gigabit Ethernet, GbE (iSCSI)
IEEE 1394a or 1394b (FireWire)

Each methodology has is benefits and pitfalls, but all offer rugged access to shared storage resources. With a shared storage solution, the workstation can boot from the SAN, share files and applications (if the software licensing agreement allows it), and make the backup and restore aspect of data management easier.

Xserve RAID SAN