Share via

PSH5X–A Windows PowerShell module for HDF5

  • Article

Just saw the announcement from the HDF5 group that they’ve released PSH5X-a Windows PowerShell module for HDF5. They also have some good resources up to help get started – including the pseudo-mindmap of the provider….

For folks not familiar with HDF5 -

The HDF5 technology suite is designed to organize, store, discover, access, analyze, share, and preserve diverse, complex data in continuously evolving heterogeneous computing and storage environments.

HDF5 supports all types of data stored digitally, regardless of origin or size. Petabytes of remote sensing data collected by satellites, terabytes of computational results from nuclear testing models, and megabytes of high-resolution MRI brain scans are stored in HDF5 files, together with metadata necessary for efficient data sharing, processing, visualization, and archiving.

PSH5X logoPSH5X is a Windows PowerShell module for HDF5. It leverages PowerShell's provider model to produce a file system-like experience for HDF5 (an often cited metaphor). PSH5X helps you perform simple housekeeping tasks such as renaming HDF5 links or copying HDF5 objects, but it can also create new HDF5 items (HDF5 objects, links, attributes) and read or write HDF5 dataset and attribute values. Did you ever ask questions similar to the following?

  • How many groups and datasets are there in an HDF5 file?
  • What fraction of the total file size can be accounted for by HDF5 datasets?
  • Which path names containing the string 'H2O' lead to HDF5 datasets?

You'll find that these are examples of the proverbial 'one-liners' in PSH5X.

After years of uncontrolled growth of a bewildering jungle of scripting technologies on the Windows platform, there's, finally, a one-stop automation hub, Windows PowerShell. You may not be aware of it, but it comes with every modern Windows desktop or server installation. You can view PSH5X as a ramp leading straight into the fast lane on the PowerShell highway. There you will have access to a myriad of helpful cmdlets to get almost every HDF5 job done. For example, have a look at FAQ 2.01 if you ever wondered how to get data from HDF5 into Excel.

There are already several excellent scripting interfaces available for HDF5 including Andrew Collette's h5py Python module. Most people would probably agree that for something as wonderful and multi-faceted as HDF5 there can hardly be too many good choices. With PSH5X we're adding another powerful tool to the arsenal and hope that, with your help, it will find its "niche" in the ecosystem.

Questions? Check out a few remarks on terminology, a list of PSH5X cmdlets, the FAQ, the tutorial, advanced features, and several limitations and known issues.