Package org.apache.hadoop.hdfs.protocol

Class BlockListAsLongs

java.lang.Object
org.apache.hadoop.hdfs.protocol.BlockListAsLongs
All Implemented Interfaces:
Iterable<BlockListAsLongs.BlockReportReplica>
@Private @Evolving public abstract class BlockListAsLongs extends Object implements Iterable<BlockListAsLongs.BlockReportReplica>

  • Field Details

  • Constructor Details

    • BlockListAsLongs

      public BlockListAsLongs()

  • Method Details

    • decodeBuffer

      public static BlockListAsLongs decodeBuffer(int numBlocks, org.apache.hadoop.thirdparty.protobuf.ByteString blocksBuf, int maxDataLength)
      Prepare an instance to in-place decode the given ByteString buffer.
      Parameters:
      numBlocks - - blocks in the buffer
      blocksBuf - - ByteString encoded varints
      maxDataLength - - maximum allowable data size in protobuf message
      Returns:
      BlockListAsLongs

    • decodeBuffers

      @VisibleForTesting public static BlockListAsLongs decodeBuffers(int numBlocks, List<org.apache.hadoop.thirdparty.protobuf.ByteString> blocksBufs)
      Prepare an instance to in-place decode the given ByteString buffers.
      Parameters:
      numBlocks - - blocks in the buffers
      blocksBufs - - list of ByteString encoded varints
      Returns:
      BlockListAsLongs

    • decodeBuffers

      public static BlockListAsLongs decodeBuffers(int numBlocks, List<org.apache.hadoop.thirdparty.protobuf.ByteString> blocksBufs, int maxDataLength)
      Prepare an instance to in-place decode the given ByteString buffers.
      Parameters:
      numBlocks - - blocks in the buffers
      blocksBufs - - list of ByteString encoded varints
      maxDataLength - - maximum allowable data size in protobuf message
      Returns:
      BlockListAsLongs

    • decodeLongs

      public static BlockListAsLongs decodeLongs(List<Long> blocksList)
      Prepare an instance to in-place decode the given list of Longs. Note it's much more efficient to decode ByteString buffers and only exists for compatibility.
      Parameters:
      blocksList - - list of longs
      Returns:
      BlockListAsLongs

    • decodeLongs

      public static BlockListAsLongs decodeLongs(List<Long> blocksList, int maxDataLength)
      Prepare an instance to in-place decode the given list of Longs. Note it's much more efficient to decode ByteString buffers and only exists for compatibility.
      Parameters:
      blocksList - - list of longs
      maxDataLength - - maximum allowable data size in protobuf message
      Returns:
      BlockListAsLongs

    • encode

      @VisibleForTesting public static BlockListAsLongs encode(Collection<? extends Replica> replicas)
      Prepare an instance to encode the collection of replicas into an efficient ByteString.
      Parameters:
      replicas - - replicas to encode
      Returns:
      BlockListAsLongs

    • readFrom

      public static BlockListAsLongs readFrom(InputStream is, int maxDataLength) throws IOException
      Throws:
      IOException

    • writeTo

      public void writeTo(OutputStream os) throws IOException
      Throws:
      IOException

    • builder

      @VisibleForTesting public static BlockListAsLongs.Builder builder()

    • builder

      public static BlockListAsLongs.Builder builder(int maxDataLength)

    • getNumberOfBlocks

      public abstract int getNumberOfBlocks()
      The number of blocks
      Returns:
      - the number of blocks

    • getBlocksBuffer

      public abstract org.apache.hadoop.thirdparty.protobuf.ByteString getBlocksBuffer()
      Very efficient encoding of the block report into a ByteString to avoid the overhead of protobuf repeating fields. Primitive repeating fields require re-allocs of an ArrayList<Long> and the associated (un)boxing overhead which puts pressure on GC. The structure of the buffer is as follows: - each replica is represented by 4 longs: blockId, block length, genstamp, replica state
      Returns:
      ByteString encoded block report

    • getBlocksBuffers

      public List<org.apache.hadoop.thirdparty.protobuf.ByteString> getBlocksBuffers()
      List of ByteStrings that encode this block report
      Returns:
      ByteStrings

    • getBlockListAsLongs

      public abstract long[] getBlockListAsLongs()
      Convert block report to old-style list of longs. Only used to re-encode the block report when the DN detects an older NN. This is inefficient, but in practice a DN is unlikely to be upgraded first The structure of the array is as follows: 0: the length of the finalized replica list; 1: the length of the under-construction replica list; - followed by finalized replica list where each replica is represented by 3 longs: one for the blockId, one for the block length, and one for the generation stamp; - followed by the invalid replica represented with three -1s; - followed by the under-construction replica list where each replica is represented by 4 longs: three for the block id, length, generation stamp, and the fourth for the replica state.
      Returns:
      list of longs

    • iterator

      public abstract Iterator<BlockListAsLongs.BlockReportReplica> iterator()
      Returns a singleton iterator over blocks in the block report. Do not add the returned blocks to a collection.
      Specified by:
      iterator in interface Iterable<BlockListAsLongs.BlockReportReplica>
      Returns:
      Iterator