Class CompressedTableHDU

java.lang.Object
nom.tam.fits.BasicHDU<BinaryTable>
nom.tam.fits.TableHDU<BinaryTable>
nom.tam.fits.BinaryTableHDU
nom.tam.image.compression.hdu.CompressedTableHDU
All Implemented Interfaces:
FitsElement
public class CompressedTableHDU extends BinaryTableHDU
A header-data unit (HDU) containing a compressed binary table. A compressed table is still a binary table but with some additional constraints. The original table is divided into groups of rows (tiles) and each tile is compressed on its own. The compressed data is then stored in the 3 data columns of this binary table (compressed, gzipped and uncompressed) depending on the compression type used in the tile. Additional data columns may contain specific compression options for each tile (i.e. compressed table row) individually. Table keywords, which conflict with those in the original table are 'saved' under standard alternative names, so they may be restored with the original table as appropriate.

Compressing a table HDU is typically a two-step process:

  1. Create a CompressedTableHDU, e.g. with fromBinaryTableHDU(BinaryTableHDU, int, String...), using the specified number of table rows per compressed block, and compression algorithm(s)
  2. Perform the compression via compress()

For example to compress a binary table: 

  BinaryTableHDU table = ...
  
  // 1. Create compressed HDU with the
  CompressedTableHDU compressed = CompressedTableHDU.fromBinaryTableHDU(table, 4, Compression.ZCMPTYPE_RICE_1);
  
  // 2. Perform the compression.
  compressed.compress();

which of course you can compact into a single line as: 

CompressedTableHDU compressed = CompressedTableHDU.fromBinaryTableHDU(table, 4, Compression.ZCMPTYPE_RICE_1).compress();

The two step process (as opposed to a single-step one) was probably chosen because it mimics that of CompressedImageHDU, where further configuration steps may be inserted in-between. After the compression, the compressed table HDU can be handled just like any other HDU, and written to a file or stream, for example.

The reverse process is simply via the asBinaryTableHDU() method. E.g.: 

   CompressedTableHDU compressed = ...
   BinaryTableHDU table = compressed.asBinaryTableHDU();
See Also:

  • Constructor Details

    • CompressedTableHDU

      public CompressedTableHDU(Header hdr, CompressedTableData datum)
      Creates an new compressed table HDU with the specified header and compressed data.
      Parameters:
      hdr - the header
      datum - the compressed table data. The data may not be actually compressed at this point, int which case you may need to call compress() before writing the new compressed HDU to a stream.
      See Also:

  • Method Details

    • useOldStandardVLAIndexing

      public static void useOldStandardVLAIndexing(boolean value)
      Enables or disables using reversed compressed/uncompressed heap indices for when decompressing variable-length arrays (VLAs), as was prescribed by the original FITS 4.0 standard and the Pence et al. 2013 convention.

      The FITS 4.0 standard defines the convention of compressing variable-length columns in binary tables. On page 50 it writes:

      2. Append the VLA descriptors from the uncompressed table (which may be either Q-type or P-type) to the temporary array of VLA descriptors for the compressed table.

      And the original Tiled-table convention by W. Pence et al. also writes:

      [...] we concatenate the array of descriptors from the uncompressed table onto the end of the temporary array of descriptors (to the compressed VLAs in the compressed table) before the 2 combined arrays of descriptors are compressed and written into the heap in the compressed table.
      However, it appears that the commonly used tools fpack / funpack provided in CFITSIO do just the reverse of this presciption. They store the uncopressed pointer before the compressed pointers. Because these tools are widely used we want to support files created or consumed by these tools. As such we allow to deviate from the FITS standard, and swap the order of the stored VLA indices inside compressed tables.
      Parameters:
      value - true if we should use VLA indices in compressed tables that follow the format described in the original Pence et al. 2013 convention, and also the FITS 4.0 standard as of 2024 Mar 1; otherwise false. These original prescriptions define the reverse of what is actually implemented by CFITSIO and its tools fpack and funpack. (Our default is to conform to CFITSIO, and the expected revision of the standard to match).
      Since:
      1.19.1
      See Also:

    • hasOldStandardVLAIndexing

      public static boolean hasOldStandardVLAIndexing()
      Checks if we should use reversed compressed/uncompressed heap indices for when decompressing variable-length arrays (VLAs).
      Returns:
      value true if we will assime VLA indices in compressed tables that follow the format described in the original Pence et al. 2013 convention, and also the FITS 4.0 standard as of 2024 Mar 1; otherwise false. These original prescriptions define the reverse of what is actually implemented by CFITSIO and its tools fpack and funpack. (Our default is to conform to CFITSIO, and the expcted revision of the standard to match).
      Since:
      1.19.1
      See Also:

    • fromBinaryTableHDU

      public static CompressedTableHDU fromBinaryTableHDU(BinaryTableHDU binaryTableHDU, int tileRows, String... columnCompressionAlgorithms) throws FitsException
      Prepare a compressed binary table HDU for the specified binary table. When the tile row size is specified with -1, the value will be set ti the number of rows in the table. The table will be compressed in "rows" that are defined by the tile size. Next step would be to set the compression options into the HDU and then compress it.
      Parameters:
      binaryTableHDU - the binary table to compress
      tileRows - the number of rows that should be compressed per tile.
      columnCompressionAlgorithms - the compression algorithms to use for the columns (optional default compression will be used if a column has no compression specified). You should typically use one or more of the enum values defined in Compression. The FITS standard currently allows only the lossless GZIP_1, GZIP_2, RICE_1, or NOCOMPRESS (e.g. Compression.ZCMPTYPE_GZIP_1 or Compression.ZCMPTYPE_NOCOMPRESS.)
      Returns:
      the prepared compressed binary table HDU.
      Throws:
      IllegalArgumentException - if any of the listed compression algorithms are not approved for use with tables.
      FitsException - if the binary table could not be used to create a compressed binary table.
      See Also:

    • isHeader

      @Deprecated public static boolean isHeader(Header hdr)
      Deprecated.
      (for internal use) Will reduce visibility in the future
      Check that this HDU has a valid header for this type.
      Parameters:
      hdr - header to check
      Returns:
      true if this HDU has a valid header.

    • manufactureData

      @Deprecated public static CompressedTableData manufactureData(Header hdr) throws FitsException
      Deprecated.
      (for internal use) Will reduce visibility in the future
      Parameters:
      hdr - the header that describes the compressed HDU
      Returns:
      a new blank data object created from the header description
      Throws:
      FitsException

    • asBinaryTableHDU

      public BinaryTableHDU asBinaryTableHDU() throws FitsException
      Restores the original binary table HDU by decompressing the data contained in this compresed table HDU.
      Returns:
      The uncompressed binary table HDU.
      Throws:
      FitsException - If there was an issue with the decompression.
      See Also:

    • getTileRows

      public int getTileRows() throws FitsException
      Returns the number of table rows that are compressed in each table tile. This may be useful for figuring out what tiles to decompress, e.g. via asBinaryTableHDU(int, int), when wanting to access select table rows only. This value is stored under the FITS keyword ZTILELEN in the compressed header. Thus, this method simply provides a user-friendly way to access it. Note that the last tile may contain fewer rows than the value indicated by this
      Returns:
      the number of table rows that are compressed into a tile.
      Throws:
      FitsException - if the compressed header does not contain the required ZTILELEN keyword, or it is <= 0.
      Since:
      1.19
      See Also:

    • getTileCount

      public int getTileCount()
      Returns the number of compressed tiles contained in this HDU.
      Returns:
      the number of compressed tiles in this table. It is the same as the NAXIS2 value of the header, which is also returned by TableHDU.getNRows() for this compressed table.
      Since:
      1.19
      See Also:

    • asBinaryTableHDU

      public BinaryTableHDU asBinaryTableHDU(int fromTile, int toTile) throws FitsException, IllegalArgumentException
      Restores a section of the original binary table HDU by decompressing a selected range of compressed table tiles. The returned section will start at row index fromTile * getTileRows() of the full table.
      Parameters:
      fromTile - Java index of first tile to decompress
      toTile - Java index of last tile to decompress
      Returns:
      The uncompressed binary table HDU from the selected compressed tiles.
      Throws:
      IllegalArgumentException - If the tile range is out of bounds
      FitsException - If there was an issue with the decompression.
      See Also:

    • asBinaryTableHDU

      public final BinaryTableHDU asBinaryTableHDU(int tile) throws FitsException, IllegalArgumentException
      Restores a section of the original binary table HDU by decompressing a single compressed table tile. The returned section will start at row index tile * getTileRows() of the full table.
      Parameters:
      tile - Java index of the table tile to decompress
      Returns:
      The uncompressed binary table HDU from the selected compressed tile.
      Throws:
      IllegalArgumentException - If the tile index is out of bounds
      FitsException - If there was an issue with the decompression.
      See Also:

    • getColumnData

      public Object getColumnData(int col) throws FitsException, IllegalArgumentException
      Returns a particular section of a decompressed data column.
      Parameters:
      col - the Java column index
      Returns:
      The uncompressed column data as an array.
      Throws:
      IllegalArgumentException - If the tile range is out of bounds
      FitsException - If there was an issue with the decompression.
      See Also:

    • getColumnData

      public Object getColumnData(int col, int fromTile, int toTile) throws FitsException, IllegalArgumentException
      Returns a particular section of a decompressed data column.
      Parameters:
      col - the Java column index
      fromTile - the Java index of first tile to decompress
      toTile - the Java index of last tile to decompress
      Returns:
      The uncompressed column data segment as an array.
      Throws:
      IllegalArgumentException - If the tile range is out of bounds
      FitsException - If there was an issue with the decompression.
      See Also:

    • compress

      public CompressedTableHDU compress() throws FitsException
      Performs the actual compression with the selected algorithm(s) and options. When creating a compressed table HDU, e.g using the fromBinaryTableHDU(BinaryTableHDU, int, String...) method, the HDU is merely prepared but without actually performing the compression, and this method will have to be called to actually perform the compression. The design would allow for setting options between creation and compressing, but in this case there is really nothing of the sort.
      Returns:
      itself
      Throws:
      FitsException - if the compression could not be performed
      See Also:

    • getTableHeader

      public Header getTableHeader() throws HeaderCardException
      Obtain a header representative of a decompressed TableHDU.
      Returns:
      Header with decompressed cards.
      Throws:
      HeaderCardException - if the card could not be copied
      Since:
      1.18

    • getData

      public CompressedTableData getData()
      Description copied from class: BasicHDU
      Returns the data component of this HDU.
      Overrides:
      getData in class BasicHDU<BinaryTable>
      Returns:
      the associated Data object