Package nom.tam.fits.utilities

Class FitsCheckSum

java.lang.Object
nom.tam.fits.utilities.FitsCheckSum
public final class FitsCheckSum extends Object

Helper class for dealing with FITS checksums. It implements the Seaman-Pence 32-bit 1's complement checksum calculation. The implementation accumulates in two 64-bit integer values the low and high-order 16-bits of adjacent 4-byte groups. A carry-over of bits are calculated only at the end of the loop. Given the use of 64-bit accumulators, overflow would occur approximately at 280 billion short values.

This updated version of the class is a little more flexible than the prior incarnation. Specifically it allows incremental updates to data sums, and provides methods for dealing with partial checksums (e.g. from modified segments of the data), which may be used e.g. to calculate delta sums from changed blocks of data. The new implementation provides methods for decoding encoded checksums, and for calculating checksums directly from files without the need to read potentially huge data into RAM first, and for easily accessing the values stored in FITS headers.

See FITS Checksum Proposal

Note this class probably should be in the nom.tam.util package, but somehow it ended up here and now we are stuck with it.

Author:
R J Mather, Tony Johnson, Attila Kovacs
See Also:

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final long
    HDU_CHECKSUM
    The expected checksum for a HDU that already contains a valid CHECKSUM keyword

  • Method Summary

    Modifier and Type
    Method
    Description
    static long
    checksum(byte[] data)
    Computes the checksum for a byte array.
    static long
    checksum(byte[] data, int from, int to)
    Computes the checksum for a segment of a byte array.
    static long
    checksum(ByteBuffer data)
    Computes the checksum from a buffer.
    static long
    checksum(BasicHDU<?> hdu)
    Deprecated.
    Use BasicHDU.verifyIntegrity() instead to verify checksums.
    static long
    checksum(Data data)
    Deprecated.
    Use BasicHDU.verifyIntegrity() instead.
    static long
    checksum(Header header)
    Deprecated.
    Use BasicHDU.verifyIntegrity() instead.
    static long
    checksum(RandomAccess f, long from, long size)
    Computes the checksum directly from a region of a random access file, by buffering moderately sized chunks from the file as necessary.
    static String
    checksumEnc(long c, boolean compl)
    Deprecated.
    Use encode(long, boolean) instead.
    static long
    decode(String encoded)
    Decodes an encoded (and complemented) checksum.
    static long
    decode(String encoded, boolean compl)
    Decodes an encoded checksum, complementing it as required.
    static long
    differenceOf(long total, long part)
    Deprecated.
    Not foolproof, because of the carry over handling in checksums, some additionas are not reversible.
    static String
    encode(long checksum)
    Encodes the complemented checksum.
    static String
    encode(long checksum, boolean compl)
    Encodes the given checksum as is or by its complement.
    static long
    getStoredChecksum(Header header)
    Deprecated.
    Not very useful, since it has no meaning other than ensuring that the checksum of the HDU yields (int) -1 (that is 0xffffffff) after including this value for the CHECKSUM keyword in the header.
    static long
    getStoredDatasum(Header header)
    Returns the DATASUM value stored in a FITS header.
    static void
    setChecksum(BasicHDU<?> hdu)
    Computes and sets the DATASUM and CHECKSUM keywords for a given HDU.
    static void
    setDatasum(Header header, long datasum)
    Sets the DATASUM and CHECKSUM keywords in a FITS header, based on the provided checksum of the data (calculated elsewhere) and the checksum calculated afresh for the header.
    static long
    sumOf(long... parts)
    Calculates the total checksum from partial sums.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • HDU_CHECKSUM

      public static final long HDU_CHECKSUM
      The expected checksum for a HDU that already contains a valid CHECKSUM keyword
      See Also:

  • Method Details

    • checksum

      public static long checksum(byte[] data)
      Computes the checksum for a byte array.
      Parameters:
      data - the byte sequence for which to calculate a chekcsum
      Returns:
      the 32bit checksum in the range from 0 to 2^32-1
      See Also:

    • checksum

      public static long checksum(byte[] data, int from, int to)
      Computes the checksum for a segment of a byte array.
      Parameters:
      data - the byte sequence for which to calculate a chekcsum
      from - Stating index of bytes to include in checksum calculation
      to - Ending index (exclusive) of bytes to include in checksum
      Returns:
      the 32-bit checksum in the range from 0 to 2^32-1
      Since:
      1.17
      See Also:

    • checksum

      public static long checksum(ByteBuffer data)
      Computes the checksum from a buffer. This method can be used to calculate partial checksums for any data that can be wrapped into a buffer one way or another. As such it is suitable for calculating partial sums from segments of data that can be used to update datasums incrementally (e.g. by incrementing the datasum with the difference of the checksum of the new data segment vs the old data segment), or for updating the checksum for new data that has been added to the existing data (e.g. new rows in a binary table) as long as the modified data segment is a multiple of 4 bytes.
      Parameters:
      data - the buffer for which to calculate a (partial) checksum
      Returns:
      the computed 32-bit unsigned checksum as a Java long
      Since:
      1.17
      See Also:

    • checksum

      public static long checksum(Data data) throws FitsException
      Deprecated.
      Use BasicHDU.verifyIntegrity() instead.
      Computes the checksum for a FITS data object, e.g. to be used with setDatasum(Header, long). This call will always calculate the checksum for the data in memory, and as such load deferred mode data into RAM as necessary to perform the calculation. If you rather not load a huge amount of data into RAM, you might consider using checksum(RandomAccess, long, long) instead.
      Parameters:
      data - The FITS data object for which to calculate a checksum
      Returns:
      The checksum of the data
      Throws:
      FitsException - If there was an error serializing the data object
      Since:
      1.17
      See Also:

    • checksum

      public static long checksum(Header header) throws FitsException
      Deprecated.
      Use BasicHDU.verifyIntegrity() instead.
      Computes the checksum for a FITS header. It returns the checksum for the header as is, without attempting to validate the header, or modifying it in any way.
      Parameters:
      header - The FITS header object for which to calculate a checksum
      Returns:
      The checksum of the data
      Throws:
      FitsException - If there was an error serializing the FITS header
      Since:
      1.17
      See Also:

    • checksum

      public static long checksum(BasicHDU<?> hdu) throws FitsException
      Deprecated.
      Use BasicHDU.verifyIntegrity() instead to verify checksums.
      Calculates the FITS checksum for a HDU, e.g to compare agains the value stored under the CHECKSUM header keyword. The
      Parameters:
      hdu - The Fits HDU for which to calculate a checksum, including both the header and data segments.
      Returns:
      The calculated checksum for the given HDU.
      Throws:
      FitsException - if there was an error accessing the contents of the HDU.
      Since:
      1.17
      See Also:

    • checksum

      public static long checksum(RandomAccess f, long from, long size) throws IOException
      Computes the checksum directly from a region of a random access file, by buffering moderately sized chunks from the file as necessary. The file may be very large, up to the full range of 64-bit addresses.
      Parameters:
      f - the random access file, from which to compute a checksum
      from - the starting position in the file, where to start computing the checksum from.
      size - the number of bytes in the file to include in the checksum calculation.
      Returns:
      the checksum for the given segment of the file
      Throws:
      IOException - if there was a problem accessing the file during the computation.
      Since:
      1.17
      See Also:

    • checksumEnc

      @Deprecated public static String checksumEnc(long c, boolean compl)
      Deprecated.
      Use encode(long, boolean) instead.
      Parameters:
      c - The calculated 32-bit (unsigned) checksum
      compl - Whether to complement the raw checksum (as defined by the convention).
      Returns:
      The encoded checksum, suitably encoded for use with the CHECKSUM header

    • encode

      public static String encode(long checksum)
      Encodes the complemented checksum. It is the same as encode(checksum, true).
      Parameters:
      checksum - The calculated 32-bit (unsigned) checksum
      Returns:
      The encoded checksum, suitably encoded for use with the CHECKSUM header
      Since:
      1.17
      See Also:

    • encode

      public static String encode(long checksum, boolean compl)
      Encodes the given checksum as is or by its complement.
      Parameters:
      checksum - The calculated 32-bit (unsigned) checksum
      compl - If true the complement of the specified value will be encoded. Otherwise, the value as is will be encoded. (FITS normally uses the complemenyed value).
      Returns:
      The encoded checksum, suitably encoded for use with the CHECKSUM header
      Since:
      1.17
      See Also:

    • decode

      public static long decode(String encoded) throws IllegalArgumentException
      Decodes an encoded (and complemented) checksum. The same as decode(encoded, true), and the the inverse of encode(long).
      Parameters:
      encoded - The encoded checksum (16 character string)
      Returns:
      The unsigned 32-bit integer complemeted checksum.
      Throws:
      IllegalArgumentException - if the checksum string is invalid (wrong length or contains illegal ASCII characters)
      Since:
      1.17
      See Also:

    • decode

      public static long decode(String encoded, boolean compl) throws IllegalArgumentException
      Decodes an encoded checksum, complementing it as required. It is the inverse of encode(long, boolean).
      Parameters:
      encoded - the encoded checksum (16 character string)
      compl - whether to complement the checksum after decoding. Normally FITS uses complemented 32-bit checksums, so typically this optional argument should be true.
      Returns:
      The unsigned 32-bit integer checksum.
      Throws:
      IllegalArgumentException - if the checksum string is invalid (wrong length or contains illegal ASCII characters)
      Since:
      1.17
      See Also:

    • sumOf

      public static long sumOf(long... parts)
      Calculates the total checksum from partial sums. For example combining checksums from a header and data segment of a HDU, or for composing a data checksum from image tiles.
      Parameters:
      parts - The partial sums that are to be added together.
      Returns:
      The aggregated checksum as a 32-bit unsigned value.
      Since:
      1.17
      See Also:

    • differenceOf

      public static long differenceOf(long total, long part)
      Deprecated.
      Not foolproof, because of the carry over handling in checksums, some additionas are not reversible. E.g. 0xffffffff + 0xffffffff = 0xffffffff, but 0xffffffff - 0xffffffff = 0;
      Subtracts a partial checksum from an aggregated total. One may use it, for example to update the datasum of a large data, when modifying only a small segment of it. Thus, one would first subtract the checksum of the old segment from tha prior datasum, and then add the checksum of the new data segment -- without hacing to recalculate the checksum for the entire data again.
      Parameters:
      total - The total checksum.
      part - The partial checksum to be subtracted from the total.
      Returns:
      The checksum after subtracting the partial sum, as a 32-bit unsigned value.
      Since:
      1.17
      See Also:

    • setDatasum

      public static void setDatasum(Header header, long datasum) throws FitsException
      Sets the DATASUM and CHECKSUM keywords in a FITS header, based on the provided checksum of the data (calculated elsewhere) and the checksum calculated afresh for the header.
      Parameters:
      header - the header in which to store the DATASUM and CHECKSUM values
      datasum - the checksum for the data segment that follows the header in the HDU.
      Throws:
      FitsException - if there was an error serializing the header. Note, the method never throws any other type of exception (including runtime exceptions), which are instead wrapped into a FitsException when they occur.
      Since:
      1.17
      See Also:

    • setChecksum

      public static void setChecksum(BasicHDU<?> hdu) throws FitsException
      Computes and sets the DATASUM and CHECKSUM keywords for a given HDU. This method calculates the sums from scratch, and can be computationally expensive. There are less expensive incremental update methods that can be used if the HDU already had sums recorded earlier, which need to be updated e.g. because there were modifications to the header, or (parts of) the data.
      Parameters:
      hdu - the HDU to be updated.
      Throws:
      FitsException - if there was an error serializing the HDU. Note, the method never throws any other type of exception (including runtime exceptions), which are instead wrapped into a FitsException when they occur.
      See Also:

    • getStoredDatasum

      public static long getStoredDatasum(Header header) throws FitsException
      Returns the DATASUM value stored in a FITS header.
      Parameters:
      header - the FITS header
      Returns:
      The stored datasum value (unsigned 32-bit integer) as a Java long.
      Throws:
      FitsException - if the header does not contain a DATASUM entry.
      Since:
      1.17
      See Also:

    • getStoredChecksum

      public static long getStoredChecksum(Header header) throws FitsException
      Deprecated.
      Not very useful, since it has no meaning other than ensuring that the checksum of the HDU yields (int) -1 (that is 0xffffffff) after including this value for the CHECKSUM keyword in the header. It will be removed in the future.
      Returns the decoded CHECKSUM value stored in a FITS header.
      Parameters:
      header - the FITS header
      Returns:
      The decoded CHECKSUM value (unsigned 32-bit integer) recorded in the header as a Java long.
      Throws:
      FitsException - if the header does not contain a CHECKSUM entry, or it is invalid.
      Since:
      1.17
      See Also: