HDF5 file format

HDF stands for Hierarchical Data Format. It is a versatile file format originally developed at the NCSA, and currently supported by the non-profit HDF Group. HDF5 is its latest variant, and it is described to be, among other things:

• A versatile data model that can represent very complex data objects and a wide variety of metadata.
• A completely portable file format with no limit on the number or size of data objects in the collection.

Since version 3.4, the Huygens Software can read microscopic images stored in this HDF5 format. In version 3.5, it became the primary Huygens format also for writing. Since the 3.5 release the H5 format is solely used for the measured PSFs as it saves, contrary to the ICS format, as well the

• Beamfilling factor
• Imaging direction
• Coverslip position

Specifications

Our intention in creating the Huygens microscopic HDF format is that it should be possible to open and view the images and metadata with standard tools like 'hdfview '.

To that end we use the HDF5 format as straightforward as possible:

1. We use HDFs native capability to store multi dimensional data of various data types: unsigned byte, signed 16 bit integer, and 32 bit float. Complex numbers are stored as a compound type of two 32 bit float numbers with field names "r" and "i". Unsigned bytes with a precision of 1 bit are used to store binary data. The dimensions are XYZT-Channel, the sampling densities are stored in HDF dimension scales.
2. We use HDF attributes for all microscopic data. Except geometrical data, we store all microscopic parameters as channel arrays. All the attribute names should be descriptive enough.

The metadata in Huygens, contrarily to what is common in microscope file formats, does not describe the microscope configuration but instead describes the physical parameters which govern the image formation.

Next to the parameter values, we also store parameter states as enumerated attributes, describing to what degree the parameter can be trusted:

• Default A reasonable fall back value.
• Estimated Value was inferred from information found in the original file. A fair estimate, but not more.
• Reported Reported by the file, and judged as pretty reliable.
• Validated The user has confirmed the value.

Sample images

The following zip archive contains five sample images, in 8 bit (unsigned byte, synthetic raw data), 16 bit (signed integer, its deconvolved version), 32 bit (floating-point, an experimental PSF), 2x32 bit (complex pairs, the FFT of the PSF), and binary (a mask for the deconvolved version):

SVI_HDF5_samples.zip (10231 kB 2009-11-23 15:41)
md5 checksum: 6ae77836b6f7a85be65ee34903f9b1d5

Data and metadata viewer

You can use the cross-platform hdfview Java application to open these files, browse the parameters, and view the image.

Reading HDF5 files in Mathematica

This notebook contains some simple code to work with Huygens HDF5 files in Mathematica:

<kbd><a href="../wikifiles/HuygensHDF5.nb">HuygensHDF5.nb</a>

<small>(24 kB 2009-11-18 16:29)</small></kbd>

Use these commands to read Huygens HDF5 files:

Open the Huygens HDF5 file to view its contents

names = Import["a.h5"]

The variable names now contains a list with all data names in the file.

View some metadata

All microscopic parameters are stored in the group PhysicalData. This shows, for example, the emission wavelength:

Import["a.h5", {"Datasets", "/a/PhysicalData/EmissionWavelength"}]

Note that the main group name /a depends on the file.

View the image

The image, which is called Image, is stored in the dataset ImageData:

im = Import["a.h5", {"Datasets", "/a/ImageData/Image"}]

This shows the 64^th slice of the 1^st time frame of the 2^nd channel:

Image[im[[2, 1, 64]]] // ImageAdjust

Reading HDF5 files in Matlab

Use these commands to read Huygens HDF5 files in Matlab:

Open the Huygens HDF5 file to view its contents

The function hdf5info return a structure whose fields contain information about the contents of the HDF5 file:

>> fileinfo = hdf5info('a.h5','ReadAttributes',false);
toplevel = fileinfo.GroupHierarchy;

The main group (-+/a+- in this example) contains three subgroups:

>> toplevel.Groups(1)

ans =

      Filename: 'a.h5'
          Name: '/a'
        Groups: [1x3 struct]
      Datasets: []
     Datatypes: [1x1 struct]
         Links: []
    Attributes: []

View meta data

The function hdf5read accepts full pathnames. All microscopic parameters are stored in the group PhysicalData. This shows, for example, the emission wavelength:

>> hdf5read('a.h5','/a/PhysicalData/EmissionWavelength')

ans =

  1.0e-006 *

    0.5200
    0.5200

View the image

This shows the 64^th slice of the 1^st time frame of the 2^nd channel:

>> im = hdf5read('a.h5','/a/ImageData/Image');
>> imshow(im(:,:,64,1,2));

Note that the imshow command requires the Image Processing Toolbox.