An HDF5 to Video Exercise

Icon


Contents:

1   Overview
2   HDF5 video grayscale notes
3   Sample grayscale output
4   HDF5 video color notes
5   Sample color output



1. Overview.

This page discusses an exercise related to HDF5 to video conversion. There's also a first take on a new utility program (h5dumptopng). The page and the program may be of interest to people working with HDF5 or video codecs in general.

If you have questions or comments about related issues, write to me at the address below. I'll expand the article as necessary. BTW I'm seeking employment. For a resume and background information, click here.

oldcoder@yahoo.com




2. HDF5 video grayscale notes.

HDF5 is a data-file format that's used for a wide range of purposes. A company working on surveillance software processed a video stream and saved the processed stream as an HDF5 data file, said file containing one dataset per frame. They wanted to be able to transform the datasets back to a standard video format (preferably H.264 MP4).

The processed stream was unusual in some respects and there were questions about the extent to which conversion back to video would degrade salient features of the data. So I tried some related experments.

2.1 The first step was to convert the HDF5 data file to text format. To do so, I used:

h5dump  brgraydata.h5  >  brgraydata.h5dump

2.2 The output looked similar to this:

HDF5 "brgraydata.h5" {
GROUP "/" {
  GROUP "Output_Group" {
    DATASET "0001" {
    DATATYPE  H5T_IEEE_F32LE
      DATASPACE  SIMPLE { ( 480, 704 ) / ( 480, 704 ) }
        DATA {
        (0,0):   -1, -1, -1, -1, -1, -1, -1, -1, -1, ...
        (0,15):  -1, -1, -1, -1, -1, -1, -1, -1, -1, ...
        (0,30):  -1, -1, -1, -1, -1, -1, -1, -1, -1, ...
        (0,45):  -1, -1, -1, -1, 54, 88, -1, -1, -1, ...
        (0,60):  -1, -1, -1, -1, 62, 75, -1, -1, -1, ...
        (0,75):  -1, -1, -1, -1, 90, 42, -1, -1, -1, ...
        (0,90):  -1, -1, -1, -1, -1, -1, -1, 90, -1, ...
        (0,105): -1, -1, -1, -1, -1, -1, -1, 81, -1, ...

In this case, 480 was the height of a 2D matrix and 704 was the width (both numbers in bytes).

For this input file, each data point mapped directly to a pixel. The data consisted of grayscale values from 0 to 255, with -1 used to represent pixels that weren't of interest. Note: A decision was made to treat the -1s as equivalent to 0s.

2.3 I wrote a Perl program named h5dumptopng to convert data of this type to PNG format. If you'd like to see the source code, click here.

Technical notes on the program:

  • h5dumptopng produces PNG files with color palettes although grayscale PNG files could have been used for this data. The reason is that ffmpeg, which plays a role in this experiment, doesn't work well with grayscale PNG files.

  • h5dumptopng uses ppmtoppm and pnmtopng from the netpbm project.

2.4 I used h5dumptopng to transform the text file produced in step 2.1 into a series of 365 PNG files.

2.5 As it turned out, the data had small gaps in it. Specifically, reference frames were missing (one out of every ten frames). This left gaps in the PNG filename sequence. So I resequenced the PNG filenames. This left me with 365 PNG files named brgraydata-0001.png, brgraydata-0002.png, etc. without gaps.

2.6 Then I created an H.264 MP4 video file using the PNG files from step 2.5. To do so, I used ffmpeg as follows:

ffmpeg  -y  -r  2  -i  brgraydata-%04d.png \
-vcodec  libx264  -vpre  lossless_max  brgraydata.mp4

The settings shown here were an attempt to produce output that was close to lossless.

2.7 The original HDF5 data file was over 470MB in size. The PNG files were much smaller, about 1.6MB total, and the MP4 file was about five times smaller than that (350KB). But the MP4 file, though not completely lossless, did a fairly good job of preserving salient features.

As a test, I converted the PNG files into one series of human readable PGM files and the MP4 file into a second series of similar files. Then I compared the two sets of PGM files. Even focusing on pixels in active areas, many of the corresponding pixels in the two sets were identical. Additionally, in one representative frame, all of the pixels that differed did so by a maximum of one grayscale step.

So H.264 MP4 appeared to be a viable format for the data in this case.

2.8 Subsequently, I performed similar tests using three additional formats: Dirac DRC, Theora OGV, and HuffYUV AVI.

H.264 MP4 (with high-quality settings) seemed better than Dirac DRC and Theora OGV from both fidelity and size perspectives. However, the differences weren't dramatic.

HuffYUV AVI produced much larger output than H.264 MP4 (hundreds of times larger), but it didn't seem to be more accurate. The two formats both exhibited "off by one" errors. This was interesting, because HuffYUV AVI is theoretically 100% lossless. So it's possible that the "off by one" errors weren't significant; they may have been inevitable artifacts.




3. Sample grayscale output.

Here's part of a sample input frame from the grayscale-data portion of the experiment (on the left, converted from HDF5 format) and the same piece from the corresponding output frame (on the right, extracted from the final MP4 file):


Image


In this example, the corresponding pixels on the two sides differ by at most one grayscale step.




4. HDF5 video color notes.

After the grayscale results were in, I received some similar data in what I'll refer to as HDF5 BGR format. The structure of a dataset was essentially the same as before, but matrix rows contained three bytes per pixel; the first byte provided a blue component, the second one a green component, and the third a red component.

I added support for BGR and RGB formats to h5dumptopng, then converted the HDF5 input file to a set of PNG files using the same procedure as before.

The HDF5 input was 875MB in size. The PNG conversion step produced 226 PNG output files totalling 14MB in size.

I used ffmpeg as before to convert the PNG files into an H.264 MP4 video clip (5MB in size). Note: There were no missing frames this time, so no resequencing of filenames was required.

The HDF5 input included periodic reference frames in this case (one out of every ten frames). The reference frames were simply normal scenes. The other frames were mostly black. The H.264 MP4 output displayed the reference frames quite well. However, the black frames weren't preserved; large sections of these frames were mapped to gray.

The problem turned out to be that ffmpeg doesn't work reliably with paletted PNG files. So I modified h5dumptopng to make it produce non-paletted output, regenerated the PNG files, and tried H.264 MP4-encoding those.

H.264 MP4-encoding worked fine after the change. The differences between input and output pixels were greater for color data than they'd been for grayscale data, but for the frames that I checked, they were still below levels that could be detected easily by the human eye.




5. Sample color output.

Here's part of a sample output frame from the color-data portion of the experiment. A normal video stream was processed. The resulting data was stored in an HDF5 file. h5dump was used to convert the HDF5 file to text. h5dumptopng was used to convert the text to a set of PNG files. ffmpeg was used to convert the PNG files to an H.264 MP4 file. mplayer was used to extract some frames. The GIMP was used to edit one frame and extract the portion shown here.


Image



Hosting provided by Zymic.

For acknowledgments related to CSS and other code used, click here.

Linked or embedded works (software, books, articles, etc.) are presented under their own licenses.

Other unique content on this page, excluding screenshots, is distributed under the following license: C.C. Attribution NonCommercial ShareAlike 3.0. In some cases, screenshots are distributed under the same license. In other cases, the associated program's license applies.

image Valid XHTML 1.0