Package org.apache.hadoop.util

Class CrcUtil

java.lang.Object
org.apache.hadoop.util.CrcUtil
@LimitedPrivate({"Common","HDFS","MapReduce","Yarn"}) @Unstable public final class CrcUtil extends Object
This class provides utilities for working with CRCs.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    CASTAGNOLI_POLYNOMIAL
     
    static final int
    GZIP_POLYNOMIAL
     
    static final int
    MULTIPLICATIVE_IDENTITY
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static int
    compose(int crcA, int crcB, long lengthB, ToIntFunction<Long> mod)
    compose.
    static int
    composeWithMonomial(int crcA, int crcB, int monomial, ToIntFunction<Long> mod)
    composeWithMonomial.
    static int
    getMonomial(long lengthBytes, ToIntFunction<Long> mod)
    Compute x^(lengthBytes * 8) mod mod, where mod is in "reversed" (little-endian) format such that mod & 1 represents x^31 and has an implicit term x^32.
    static byte[]
    intToBytes(int value)
     
    static int
    readInt(byte[] buf, int offset)
    Reads 4-byte big-endian int value from buf starting at offset. buf.length must be greater than or equal to offset + 4.
    static String
    toMultiCrcString(byte[] bytes)
    For use with debug statements; verifies bytes.length on creation, expecting it to be divisible by CRC byte size, and returns a list of hex formatted values.
    static String
    toSingleCrcString(byte[] bytes)
    For use with debug statements; verifies bytes.length on creation, expecting it to represent exactly one CRC, and returns a hex formatted value.
    static void
    writeInt(byte[] buf, int offset, int value)
    Writes big-endian representation of value into buf starting at offset. buf.length must be greater than or equal to offset + 4.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Method Details

    • getMonomial

      public static int getMonomial(long lengthBytes, ToIntFunction<Long> mod)
      Compute x^(lengthBytes * 8) mod mod, where mod is in "reversed" (little-endian) format such that mod & 1 represents x^31 and has an implicit term x^32.
      Parameters:
      lengthBytes - lengthBytes.
      mod - mod.
      Returns:
      monomial.

    • composeWithMonomial

      public static int composeWithMonomial(int crcA, int crcB, int monomial, ToIntFunction<Long> mod)
      composeWithMonomial.
      Parameters:
      crcA - crcA.
      crcB - crcB.
      monomial - Precomputed x^(lengthBInBytes * 8) mod mod
      mod - mod.
      Returns:
      compose with monomial.

    • compose

      public static int compose(int crcA, int crcB, long lengthB, ToIntFunction<Long> mod)
      compose.
      Parameters:
      crcA - crcA.
      crcB - crcB.
      lengthB - length of content corresponding to crcB, in bytes.
      mod - mod.
      Returns:
      compose result.

    • intToBytes

      public static byte[] intToBytes(int value)
      Parameters:
      value - value.
      Returns:
      4-byte array holding the big-endian representation of value.

    • writeInt

      public static void writeInt(byte[] buf, int offset, int value)
      Writes big-endian representation of value into buf starting at offset. buf.length must be greater than or equal to offset + 4.
      Parameters:
      buf - buf size.
      offset - offset.
      value - value.

    • readInt

      public static int readInt(byte[] buf, int offset)
      Reads 4-byte big-endian int value from buf starting at offset. buf.length must be greater than or equal to offset + 4.
      Parameters:
      offset - offset.
      buf - buf.
      Returns:
      int.

    • toSingleCrcString

      public static String toSingleCrcString(byte[] bytes)
      For use with debug statements; verifies bytes.length on creation, expecting it to represent exactly one CRC, and returns a hex formatted value.
      Parameters:
      bytes - bytes.
      Returns:
      a list of hex formatted values.

    • toMultiCrcString

      public static String toMultiCrcString(byte[] bytes)
      For use with debug statements; verifies bytes.length on creation, expecting it to be divisible by CRC byte size, and returns a list of hex formatted values.
      Parameters:
      bytes - bytes.
      Returns:
      a list of hex formatted values.