Holographic Data Storage (1963-2012)

Short answer: holographic data storage stores a page of bits at once as an interference pattern throughout the thickness of a light-sensitive material. It was analysed in theory by P. J. van Heerden of Polaroid in 1963, was researched for decades (the IBM Research review of May 2000 describes it as having been of interest 'for three decades'), reached a standard in May 2007 (ECMA-377, a 200 GB nominal write-once cartridge) and a vendor-announced 300 GB drive (InPhase Tapestry 300r), but the sources read describe no lasting commercial product: InPhase's drive was repeatedly delayed and the company was reported out of business in February 2010, and the standard was not followed by drives in any source I opened.

Roots in holography, 1948-1965. The Nobel Prize essay by Klaus Biedermann says Dennis Gabor conceived wavefront reconstruction in 1948 to correct aberrations in electron microscopes, that the mercury arc lamp limited his experiment to an object of a few millimetres, and that a breakthrough came in 1963 when Leith and Upatnieks at the University of Michigan made three-dimensional images with laser light (their theory paper appeared in the Journal of the Optical Society of America in 1962, per a search excerpt of its abstract). The same essay says Yuri Denisyuk realised in 1962 how to combine Gabor's method with Lippmann's colour photography. Sean Johnston's account of Denisyuk says he recorded standing-wave interference through the depth of a thick emulsion, published first in 1962, and that his work gained little attention until three American groups rediscovered the geometry in autumn 1965. Thick, volume recording is the property that later storage proposals relied on; neither source says Denisyuk or Leith proposed data storage, so this card does not claim that.

1963: the capacity theory. P. J. van Heerden of the Polaroid Corporation published 'Theory of Optical Information Storage in Solids' in Applied Optics volume 2 number 4 (pages 393 to 400, 1963). The abstract says that photography stores information in an essentially two-dimensional medium, that three-dimensional optical storage is possible in semitransparent coloured materials such as alkali halides with colour centres, that information is stored by interference between pairs of plane parallel waves, and that capacity behaves 'as if every little cube with sides equal to the wavelength of light acts as an independent information storage cell', giving 10 to the 12 to 10 to the 13 bits per cm3. It also describes a 'ghost image' property that makes the scheme suited to associative memories. As a cross-check, one bit per cube of side 632.8 nm (helium-neon red) is about 3.9 x 10 to the 9 bits per mm3, about 0.49 GB per mm3 (calculation), which matches the 500 MB per cubic millimetre that Wikipedia gives for that wavelength and is consistent with van Heerden's range. The Applied Optics issue listing (search excerpt) shows a companion van Heerden paper, 'A New Optical Method of Storing and Retrieving Information', on pages 387 to 392; the abstract of that paper was not read.

1990s: research platforms. The IBM review (Ashley and ten co-authors, May 2000) says IBM Research took part in the DARPA-initiated consortia on Holographic Data Storage Systems (HDSS) and PhotoRefractive Information Storage Materials (PRISM), and that interest revived because liquid-crystal panels (as spatial light modulators) and camcorder camera chips (as detectors) had become available. Its reference list includes a 1994 Science paper by Heanue, Bashaw and Hesselink and a 1995 Scientific American article by Psaltis and Mok, which were not opened. IBM built three testers: the PRISM tester, DEMON I and DEMON II. On DEMON I the authors report 1200 holograms superimposed in an 8 mm thick lithium niobate crystal doped with iron and read back with a raw bit error rate below 2 x 10 to the minus 8 using an 8:12 modulation code. DEMON II used a 1024 x 1024 pixel reflective modulator made at IBM Yorktown (12.8 micrometre pixels), a camera with 12 micrometre pixels at 41 frames per second, a 532 nm diode-pumped laser and a 30 mm effective focal length, to show high areal density with short-focal-length optics; this card does not quote a density figure for it because the figure was seen only in a search excerpt.

What the 2000 review said about materials and prospects. Two material families are weighed: photorefractive crystals such as lithium niobate, which are rewritable but lose data when read unless a two-colour 'photon-gated' scheme is used, and organic photopolymers, which are sensitive and have large dynamic range but shrink as they polymerise (the review reports shrinkage reduced to less than 0.1 percent in one material) and are hard to make millimetres thick. The review's outlook weighs four product scenarios: a stationary media block of about 25 GB with access of the order of 10 microseconds as a cache between memory and disk, a rotating disk with terabytes per platter, a CD-ROM-like read-only disk (the review says single-exposure full-disk replication had been demonstrated), and a rack-scale library. It also states that holographic storage 'currently suffers from the relatively high component and integration costs faced by any emerging technology' while magnetic disks had an established infrastructure; no cost figure is given there.

InPhase Technologies, 2000-2010. Computerworld (February 2007) says InPhase was spun off from Bell Laboratories in 2000; The Register (June 2008) calls it a spin-off venture of Lucent, and Wikipedia says it came out of Bell Labs after roughly a decade of research. The Tapestry HDS-300R was described by InPhase in February 2007 as a write-once format for archiving, with 300 GB on a single disc, 20 MB/s, a 100,000 hour mean time between failures and a 50 year expected lifetime (all vendor claims), in a 5.25-inch cartridge 3 mm thick holding a platter 1.5 mm thick. InPhase said each disc could hold 600 GB with the other 300 GB taken by error correction, and that photopolymer media from Bayer MaterialScience would be made by Hitachi Maxell (vendor statements). The chief executive said the company would not go into the backup market. The company's roadmap in that article was an 800 GB second generation with about 80 MB/s by the end of 2008 and 1.6 TB by 2010; The Register and Wikipedia give the long-term transfer rate as 120 MB/s, so the sources disagree on the roadmap rate, and none documents a second-generation product.

Delays and the end of InPhase. Computerworld (February 2007) said beta units had gone to users in December 2006 and bulk shipments would start in July 2007. The Register (June 2008) says the 300 GB drives had been promised for three years, with dates of late 2006, February 2007 and May 2008, and that staff were cut: a tipster said roughly half of 130, while the chief executive told a local paper it was fewer than 40. A SPIE Newsroom item seen only as a search excerpt quotes InPhase's chief technology officer saying that 150 Gb evaluation drives were planned for May 2008 with the 300 GB product in the fourth quarter, that a delay was needed for more work, and that issues included susceptibility to shock and vibration and the reliability of the field-replaceable 405 nm laser. By September 2009 The Register described InPhase as in hibernation with a late-2009 ship date still pending. TV Technology reported on 8 February 2010 that InPhase had gone out of business and had assets seized by the Colorado Department of Revenue for back taxes, that its 60 employees had received final pay, and that it 'was unable to bring a product to market'. The sources conflict on what shipped: Computerworld reports beta shipments in 2006, TV Technology says no product reached the market, and Wikipedia's InPhase page says the first reader was offered in May 2008 while also saying no release date was visible. This card does not settle whether any drive was delivered to customers.

After 2010 (Wikipedia, secondary). Wikipedia's InPhase article says Signal Lake Venture Capital took a majority stake in March 2010, that InPhase filed for Chapter 11 protection on 17 October 2011, that its assets were sold at auction in March 2012 to Akonia Holographics (launched 10 August 2012), and that Apple announced it was acquiring Akonia on 30 August 2018. Wikipedia's article on holographic data storage instead says InPhase's assets and know-how were acquired by Apple, so the two pages differ on the chain. The Register's September 2009 article describes GE's separate 'micro-hologram' work: reflective holograms small enough to be read by Blu-ray-type optics, with GE claiming 500 GB on a CD-sized disc and a roadmap past 1 TB for 2011 to 2012 (vendor claims); no source I read says what became of it.

Optware and the Holographic Versatile Disc, 2004-2007. Computerworld (August 2004) quotes Optware of Yokohama as saying it had achieved reliable recording and playback of digital movies on a transparent holographic disc, using a 532 nm green laser on a 12 cm disc with a separate red laser reading a servo layer, and a mirror layer between them that reflects green and passes red and stops scattering that would add noise. Optware planned to commercialise 200 GB discs for enterprise users in the first quarter of 2006 and said capacity could reach 1 TB (vendor claims), and said Sony had ordered collinear equipment to research holographic storage (company statement). The Horimai and Tan paper in Applied Optics (2006) says collinear technology bundles reference and signal beams on the same axis through a single objective lens, so that a servo can focus, track and address and the system can have CD and DVD upper compatibility. Press releases seen as search excerpts say Ecma's General Assembly created Technical Committee 44 on 9 December 2004 at Optware's request (one release misprints the year as 2005), that six companies formed the HVD Alliance in February 2005, and that Ecma planned standards for a 200 GB cartridge, a 100 GB read-only disc and a 30 GB card, with submission to ISO in December 2006.

ECMA-377, May 2007. The standard's introduction says the HVD Alliance proposed it to Ecma in October 2004 as the first of a family of holographic media and that the General Assembly adopted it in May 2007. It specifies a 120 mm disk, 2.3 to 2.6 mm thick, in a cartridge 153.0 mm x 135.0 mm x 11.0 mm; a 532 nm pulsed recording and reading laser (50 ns pulses, numerical aperture 0.50) and a separate 655 nm tracking beam; a spatial light modulator of 358 x 358 elements carrying a data page of 192 x 192 pixels; Reed-Solomon error correction in two stages; a rotation speed of 300 rpm; and pre-embossed pit tracks at a pitch of 1.6 micrometres. The disk is of the 'Phi' type, meaning the recording layer can be fixed (made insensitive to light), and finalising ends further recording. The user data zone has 18,880 pit tracks and 4,720 hologram tracks and 'may contain 191 520 000 holograms recording 173 104 615 user Data Pages'; the medium must show no uncorrectable errors after 1 million readouts. The nominal capacity is 200 GB. Using the standard's own figure of 188 user packets of 1,024 bytes per 208 pages, 173,104,615 pages carry about 160 GB of user data (calculation, my reading of the format), which is below the nominal 200 GB; the difference is not explained in the standard text I read and may be a counting convention. Wikipedia says the companion ECMA-378 (100 GB read-only) was published with it on 11 June 2007; that was not checked against Ecma's own page.

Why it stalled: what the sources say. No source gives a single cause. Technical: the IBM review's list of materials problems (shrinkage, thickness, destructive readout, near-perfect optics, alignment) and the InPhase shock, vibration and laser-reliability statement. Market positioning: InPhase aimed only at archiving and said it would stay out of backup (Computerworld, 2007), and The Register's 2009 comment says tape was more reliable than it used to be and had a clear capacity roadmap, which is an editorial opinion. Corporate: TV Technology says a lead investor was seeking more money for nearly a year before the shutdown. Wikipedia's HVD article says delays were 'likely due to' drive and disc economics, lack of compatibility with other standards and competition from Blu-ray and streaming, and that development was 'abandoned due to funding issues'; both statements are Wikipedia's own inference and the cited ZDNet article concerns InPhase, so they are not independently confirmed for Optware. The sources I read do not say what happened to Optware itself after 2007.

What the sources do not settle. Whether any InPhase 300 GB drive was delivered to a customer; the second-generation transfer rate (80 or 120 MB/s); the chain of ownership of InPhase's assets after 2010 (two Wikipedia pages differ); the exact date of the first ECMA-378 publication; the difference between the nominal 200 GB of ECMA-377 and the figure derived from its page counts; what became of Optware and GE's micro-holographic work; any Bell Labs research results of the 1990s (only the spin-off statement was read); and the date and density of IBM's DEMON II result.