Package nom.tam.image.compression.hdu

Class CompressedTableHDU

java.lang.Object

nom.tam.fits.BasicHDU<T>

nom.tam.fits.TableHDU<BinaryTable>

nom.tam.fits.BinaryTableHDU

nom.tam.image.compression.hdu.CompressedTableHDU

All Implemented Interfaces:

FitsElement

public class CompressedTableHDU
extends BinaryTableHDU

A header-data unit (HDU) containing a compressed binary table. A compressed table is still a binary table but with some additional constraints. The original table is divided into groups of rows (tiles) and each tile is compressed on its own. The compressed data is then stored in the 3 data columns of this binary table (compressed, gzipped and uncompressed) depending on the compression type used in the tile. Additional data columns may contain specific compression options for each tile (i.e. compressed table row) individually. Table keywords, which conflict with those in the original table are 'saved' under standard alternative names, so they may be restored with the original table as appropriate.

Compressing a table HDU is typically a two-step process:

1. Create a CompressedTableHDU, e.g. with fromBinaryTableHDU(BinaryTableHDU, int, String...), using the specified number of table rows per compressed block, and compression algorithm(s)
2. Perform the compression via compress()

For example to compress a binary table:

   BinaryTableHDU table = ...
   
   // 1. Create compressed HDU with the
   CompressedTableHDU compressed = CompressedTableHDU.fromBinaryTableHDU(table, 4, Compression.ZCMPTYPE_RICE_1);
   
   // 2. Perform the compression.
   compressed.compress();
 

which of course you can compact into a single line as:

 CompressedTableHDU compressed = CompressedTableHDU.fromBinaryTableHDU(table, 4, Compression.ZCMPTYPE_RICE_1).compress();
 

The two step process (as opposed to a single-step one) was probably chosen because it mimics that of CompressedImageHDU, where further configuration steps may be inserted in-between. After the compression, the compressed table HDU can be handled just like any other HDU, and written to a file or stream, for example.

The reverse process is simply via the asBinaryTableHDU() method. E.g.:

    CompressedTableHDU compressed = ...
    BinaryTableHDU table = compressed.asBinaryTableHDU();
 

See Also:

• CompressedTableData

Field Summary

Fields inherited from class nom.tam.fits.BasicHDU

BITPIX_BYTE, BITPIX_DOUBLE, BITPIX_FLOAT, BITPIX_INT, BITPIX_LONG, BITPIX_SHORT

Constructor Summary

Constructors

Constructor	Description
CompressedTableHDU(Header hdr, CompressedTableData datum)	Creates an new compressed table HDU with the specified header and compressed data.

Method Summary

Modifier and Type	Method	Description
BinaryTableHDU	asBinaryTableHDU()	Restores the original binary table HDU by decompressing the data contained in this compresed table HDU.
final BinaryTableHDU	asBinaryTableHDU(int tile)	Restores a section of the original binary table HDU by decompressing a single compressed table tile.
BinaryTableHDU	asBinaryTableHDU(int fromTile, int toTile)	Restores a section of the original binary table HDU by decompressing a selected range of compressed table tiles.
CompressedTableHDU	compress()	Performs the actual compression with the selected algorithm(s) and options.
static CompressedTableHDU	fromBinaryTableHDU(BinaryTableHDU binaryTableHDU, int tileRows, String... columnCompressionAlgorithms)	Prepare a compressed binary table HDU for the specified binary table.
Object	getColumnData(int col)	Returns a particular section of a decompressed data column.
Object	getColumnData(int col, int fromTile, int toTile)	Returns a particular section of a decompressed data column.
CompressedTableData	getData()	Returns the data component of this HDU.
Header	getTableHeader()	Obtain a header representative of a decompressed TableHDU.
int	getTileCount()	Returns the number of compressed tiles contained in this HDU.
int	getTileRows()	Returns the number of table rows that are compressed in each table tile.
static boolean	hasOldStandardVLAIndexing()	Checks if we should use reversed compressed/uncompressed heap indices for when decompressing variable-length arrays (VLAs).
static boolean	isHeader(Header hdr)	Deprecated. (for internal use) Will reduce visibility in the future
static CompressedTableData	manufactureData(Header hdr)	Deprecated. (for internal use) Will reduce visibility in the future
static void	useOldStandardVLAIndexing(boolean value)	Enables or disables using reversed compressed/uncompressed heap indices for when decompressing variable-length arrays (VLAs), as was prescribed by the original FITS 4.0 standard and the Pence et al. 2013 convention.

Methods inherited from class nom.tam.fits.BinaryTableHDU

addColumn, convertToBits, encapsulate, getColumnDescriptor, info, isData, isHeader, manufactureHeader, setColumnName, setComplexColumn, wrap, write

Methods inherited from class nom.tam.fits.TableHDU

addRow, deleteColumnsIndexOne, deleteColumnsIndexOne, deleteColumnsIndexZero, deleteColumnsIndexZero, deleteRows, deleteRows, findColumn, getColumn, getColumn, getColumnFormat, getColumnMeta, getColumnName, getColumns, getDefaultColumnName, getElement, getNCols, getNRows, getRow, setColumn, setColumn, setColumnMeta, setColumnMeta, setColumnMeta, setColumnMeta, setColumnMeta, setColumnMeta, setColumnMeta, setCurrentColumn, setCurrentColumn, setElement, setRow

Methods inherited from class nom.tam.fits.BasicHDU

addValue, addValue, addValue, addValue, addValue, addValue, addValue, addValue, calcChecksum, card, getAuthor, getAxes, getBitpix, getBitPix, getBlankValue, getBScale, getBUnit, getBZero, getCreationDate, getDummyHDU, getEpoch, getEquinox, getFileOffset, getGroupCount, getHeader, getInstrument, getKernel, getMaximumValue, getMinimumValue, getObject, getObservationDate, getObserver, getOrigin, getParameterCount, getReference, getSize, getStoredChecksum, getStoredDatasum, getTelescope, getTrimmedString, getTrimmedString, read, reset, rewrite, rewriteable, setChecksum, verifyDataIntegrity, verifyIntegrity

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

CompressedTableHDU

public CompressedTableHDU(Header hdr,
 CompressedTableData datum)

Creates an new compressed table HDU with the specified header and compressed data.

Parameters:

hdr - the header

datum - the compressed table data. The data may not be actually compressed at this point, int which case you may need to call compress() before writing the new compressed HDU to a stream.

See Also:

• compress()

Method Details

useOldStandardVLAIndexing

public static void useOldStandardVLAIndexing(boolean value)

Enables or disables using reversed compressed/uncompressed heap indices for when decompressing variable-length arrays (VLAs), as was prescribed by the original FITS 4.0 standard and the Pence et al. 2013 convention.

The FITS 4.0 standard defines the convention of compressing variable-length columns in binary tables. On page 50 it writes:

2. Append the VLA descriptors from the uncompressed table (which may be either Q-type or P-type) to the temporary array of VLA descriptors for the compressed table.

And the original Tiled-table convention by W. Pence et al. also writes:

[...] we concatenate the array of descriptors from the uncompressed table onto the end of the temporary array of descriptors (to the compressed VLAs in the compressed table) before the 2 combined arrays of descriptors are compressed and written into the heap in the compressed table.

However, it appears that the commonly used tools fpack / funpack provided in CFITSIO do just the reverse of this presciption. They store the uncopressed pointer before the compressed pointers. Because these tools are widely used we want to support files created or consumed by these tools. As such we allow to deviate from the FITS standard, and swap the order of the stored VLA indices inside compressed tables.

Parameters:

value - true if we should use VLA indices in compressed tables that follow the format described in the original Pence et al. 2013 convention, and also the FITS 4.0 standard as of 2024 Mar 1; otherwise false. These original prescriptions define the reverse of what is actually implemented by CFITSIO and its tools fpack and funpack. (Our default is to conform to CFITSIO, and the expected revision of the standard to match).

Since:

1.19.1

See Also:

• hasOldStandardVLAIndexing()
• asBinaryTableHDU()

hasOldStandardVLAIndexing

public static boolean hasOldStandardVLAIndexing()

Checks if we should use reversed compressed/uncompressed heap indices for when decompressing variable-length arrays (VLAs).

Returns:

value true if we will assime VLA indices in compressed tables that follow the format described in the original Pence et al. 2013 convention, and also the FITS 4.0 standard as of 2024 Mar 1; otherwise false. These original prescriptions define the reverse of what is actually implemented by CFITSIO and its tools fpack and funpack. (Our default is to conform to CFITSIO, and the expcted revision of the standard to match).

Since:

1.19.1

See Also:

• useOldStandardVLAIndexing(boolean)

fromBinaryTableHDU

public static CompressedTableHDU fromBinaryTableHDU(BinaryTableHDU binaryTableHDU,
 int tileRows,
 String... columnCompressionAlgorithms)
                                             throws FitsException

Prepare a compressed binary table HDU for the specified binary table. When the tile row size is specified with -1, the value will be set ti the number of rows in the table. The table will be compressed in "rows" that are defined by the tile size. Next step would be to set the compression options into the HDU and then compress it.

Parameters:

binaryTableHDU - the binary table to compress

tileRows - the number of rows that should be compressed per tile.

columnCompressionAlgorithms - the compression algorithms to use for the columns (optional default compression will be used if a column has no compression specified). You should typically use one or more of the enum values defined in Compression. The FITS standard currently allows only the lossless GZIP_1, GZIP_2, RICE_1, or NOCOMPRESS (e.g. Compression.ZCMPTYPE_GZIP_1 or Compression.ZCMPTYPE_NOCOMPRESS.)

Returns:

the prepared compressed binary table HDU.

Throws:

IllegalArgumentException - if any of the listed compression algorithms are not approved for use with tables.

FitsException - if the binary table could not be used to create a compressed binary table.

See Also:

• Compression
• Compression.ZCMPTYPE_GZIP_1
• Compression.ZCMPTYPE_GZIP_2
• Compression.ZCMPTYPE_RICE_1
• Compression.ZCMPTYPE_NOCOMPRESS
• asBinaryTableHDU()
• useOldStandardVLAIndexing(boolean)

isHeader

@Deprecated
public static boolean isHeader(Header hdr)

Deprecated.

(for internal use) Will reduce visibility in the future

Check that this HDU has a valid header for this type.

Parameters:

hdr - header to check

Returns:

true if this HDU has a valid header.

manufactureData

@Deprecated
public static CompressedTableData manufactureData(Header hdr)
                                           throws FitsException

Deprecated.

(for internal use) Will reduce visibility in the future

Parameters:

hdr - the header that describes the compressed HDU

Returns:

a new blank data object created from the header description

Throws:

FitsException

asBinaryTableHDU

public BinaryTableHDU asBinaryTableHDU()
                                throws FitsException

Restores the original binary table HDU by decompressing the data contained in this compresed table HDU.

Returns:

The uncompressed binary table HDU.

Throws:

FitsException - If there was an issue with the decompression.

See Also:

• asBinaryTableHDU(int, int)
• fromBinaryTableHDU(BinaryTableHDU, int, String...)
• useOldStandardVLAIndexing(boolean)

getTileRows

public int getTileRows()
                throws FitsException

Returns the number of table rows that are compressed in each table tile. This may be useful for figuring out what tiles to decompress, e.g. via asBinaryTableHDU(int, int), when wanting to access select table rows only. This value is stored under the FITS keyword ZTILELEN in the compressed header. Thus, this method simply provides a user-friendly way to access it. Note that the last tile may contain fewer rows than the value indicated by this

Returns:

the number of table rows that are compressed into a tile.

Throws:

FitsException - if the compressed header does not contain the required ZTILELEN keyword, or it is <= 0.

Since:

1.19

See Also:

• asBinaryTableHDU(int, int)

getTileCount

public int getTileCount()

Returns the number of compressed tiles contained in this HDU.

Returns:

the number of compressed tiles in this table. It is the same as the NAXIS2 value of the header, which is also returned by TableHDU.getNRows() for this compressed table.

Since:

1.19

See Also:

• getTileRows()

asBinaryTableHDU

public BinaryTableHDU asBinaryTableHDU(int fromTile,
 int toTile)
                                throws FitsException,
IllegalArgumentException

Restores a section of the original binary table HDU by decompressing a selected range of compressed table tiles. The returned section will start at row index fromTile * getTileRows() of the full table.

Parameters:

fromTile - Java index of first tile to decompress

toTile - Java index of last tile to decompress

Returns:

The uncompressed binary table HDU from the selected compressed tiles.

Throws:

IllegalArgumentException - If the tile range is out of bounds

FitsException - If there was an issue with the decompression.

See Also:

• getTileRows()
• getTileCount()
• asBinaryTableHDU()
• fromBinaryTableHDU(BinaryTableHDU, int, String...)
• useOldStandardVLAIndexing(boolean)

asBinaryTableHDU

public final BinaryTableHDU asBinaryTableHDU(int tile)
                                      throws FitsException,
IllegalArgumentException

Restores a section of the original binary table HDU by decompressing a single compressed table tile. The returned section will start at row index tile * getTileRows() of the full table.

Parameters:

tile - Java index of the table tile to decompress

Returns:

The uncompressed binary table HDU from the selected compressed tile.

Throws:

IllegalArgumentException - If the tile index is out of bounds

FitsException - If there was an issue with the decompression.

See Also:

• asBinaryTableHDU(int, int)
• getTileRows()
• getTileCount()
• useOldStandardVLAIndexing(boolean)

getColumnData

public Object getColumnData(int col)
                     throws FitsException,
IllegalArgumentException

Returns a particular section of a decompressed data column.

Parameters:

col - the Java column index

Returns:

The uncompressed column data as an array.

Throws:

IllegalArgumentException - If the tile range is out of bounds

FitsException - If there was an issue with the decompression.

See Also:

• getColumnData(int, int, int)
• asBinaryTableHDU()
• fromBinaryTableHDU(BinaryTableHDU, int, String...)

getColumnData

public Object getColumnData(int col,
 int fromTile,
 int toTile)
                     throws FitsException,
IllegalArgumentException

Returns a particular section of a decompressed data column.

Parameters:

col - the Java column index

fromTile - the Java index of first tile to decompress

toTile - the Java index of last tile to decompress

Returns:

The uncompressed column data segment as an array.

Throws:

IllegalArgumentException - If the tile range is out of bounds

FitsException - If there was an issue with the decompression.

See Also:

• getColumnData(int)
• asBinaryTableHDU()
• fromBinaryTableHDU(BinaryTableHDU, int, String...)

compress

public CompressedTableHDU compress()
                            throws FitsException

Performs the actual compression with the selected algorithm(s) and options. When creating a compressed table HDU, e.g using the fromBinaryTableHDU(BinaryTableHDU, int, String...) method, the HDU is merely prepared but without actually performing the compression, and this method will have to be called to actually perform the compression. The design would allow for setting options between creation and compressing, but in this case there is really nothing of the sort.

Returns:

itself

Throws:

FitsException - if the compression could not be performed

See Also:

• fromBinaryTableHDU(BinaryTableHDU, int, String...)
• useOldStandardVLAIndexing(boolean)

getTableHeader

public Header getTableHeader()
                      throws HeaderCardException

Obtain a header representative of a decompressed TableHDU.

Returns:

Header with decompressed cards.

Throws:

HeaderCardException - if the card could not be copied

Since:

1.18

getData

public CompressedTableData getData()

Description copied from class: BasicHDU

Returns the data component of this HDU.

Overrides:

getData in class BasicHDU<BinaryTable>

Returns:

the associated Data object