Package org.apache.hadoop.hdfs.server.protocol

Interface DatanodeProtocol

All Known Subinterfaces:
NamenodeProtocols
All Known Implementing Classes:
DatanodeProtocolClientSideTranslatorPB, NameNodeRpcServer
@Private public interface DatanodeProtocol
Protocol that a DFS datanode uses to communicate with the NameNode. It's used to upload current load information and block reports. The only way a NameNode can communicate with a DataNode is by returning values from these functions.

  • Field Details

    • versionID

      static final long versionID
      This class is used by both the Namenode (client) and BackupNode (server) to insulate from the protocol serialization. If you are adding/changing DN's interface then you need to change both this class and ALSO related protocol buffer wire protocol definition in DatanodeProtocol.proto. For more details on protocol buffer wire protocol, please see .../org/apache/hadoop/hdfs/protocolPB/overview.html
      See Also:

    • NOTIFY

      static final int NOTIFY
      See Also:

    • DISK_ERROR

      static final int DISK_ERROR
      See Also:

    • INVALID_BLOCK

      static final int INVALID_BLOCK
      See Also:

    • FATAL_DISK_ERROR

      static final int FATAL_DISK_ERROR
      See Also:

    • DNA_UNKNOWN

      static final int DNA_UNKNOWN
      Determines actions that data node should perform when receiving a datanode command.
      See Also:

    • DNA_TRANSFER

      static final int DNA_TRANSFER
      See Also:

    • DNA_INVALIDATE

      static final int DNA_INVALIDATE
      See Also:

    • DNA_SHUTDOWN

      static final int DNA_SHUTDOWN
      See Also:

    • DNA_REGISTER

      static final int DNA_REGISTER
      See Also:

    • DNA_FINALIZE

      static final int DNA_FINALIZE
      See Also:

    • DNA_RECOVERBLOCK

      static final int DNA_RECOVERBLOCK
      See Also:

    • DNA_ACCESSKEYUPDATE

      static final int DNA_ACCESSKEYUPDATE
      See Also:

    • DNA_BALANCERBANDWIDTHUPDATE

      static final int DNA_BALANCERBANDWIDTHUPDATE
      See Also:

    • DNA_CACHE

      static final int DNA_CACHE
      See Also:

    • DNA_UNCACHE

      static final int DNA_UNCACHE
      See Also:

    • DNA_ERASURE_CODING_RECONSTRUCTION

      static final int DNA_ERASURE_CODING_RECONSTRUCTION
      See Also:

    • DNA_BLOCK_STORAGE_MOVEMENT

      static final int DNA_BLOCK_STORAGE_MOVEMENT
      See Also:

    • DNA_DROP_SPS_WORK_COMMAND

      static final int DNA_DROP_SPS_WORK_COMMAND
      See Also:

  • Method Details

    • registerDatanode

      DatanodeRegistration registerDatanode(DatanodeRegistration registration) throws IOException
      Register Datanode.
      Parameters:
      registration - datanode registration information
      Returns:
      the given DatanodeRegistration with updated registration information
      Throws:
      IOException
      See Also:
      • FSNamesystem.registerDatanode(DatanodeRegistration)

    • sendHeartbeat

      HeartbeatResponse sendHeartbeat(DatanodeRegistration registration, org.apache.hadoop.hdfs.server.protocol.StorageReport[] reports, long dnCacheCapacity, long dnCacheUsed, int xmitsInProgress, int xceiverCount, int failedVolumes, VolumeFailureSummary volumeFailureSummary, boolean requestFullBlockReportLease, @Nonnull org.apache.hadoop.hdfs.server.protocol.SlowPeerReports slowPeers, @Nonnull org.apache.hadoop.hdfs.server.protocol.SlowDiskReports slowDisks) throws IOException
      sendHeartbeat() tells the NameNode that the DataNode is still alive and well. Includes some status info, too. It also gives the NameNode a chance to return an array of "DatanodeCommand" objects in HeartbeatResponse. A DatanodeCommand tells the DataNode to invalidate local block(s), or to copy them to other DataNodes, etc.
      Parameters:
      registration - datanode registration information.
      reports - utilization report per storage.
      dnCacheCapacity - the total cache capacity of the datanode (in bytes).
      dnCacheUsed - the amount of cache used by the datanode (in bytes).
      xmitsInProgress - number of transfers from this datanode to others.
      xceiverCount - number of active transceiver threads.
      failedVolumes - number of failed volumes.
      volumeFailureSummary - info about volume failures.
      requestFullBlockReportLease - whether to request a full block report lease.
      slowPeers - Details of peer DataNodes that were detected as being slow to respond to packet writes. Empty report if no slow peers were detected by the DataNode.
      slowDisks - Details of disks on DataNodes that were detected as being slow. Empty report if no slow disks were detected.
      Throws:
      IOException - on error.

    • blockReport

      DatanodeCommand blockReport(DatanodeRegistration registration, String poolId, StorageBlockReport[] reports, BlockReportContext context) throws IOException
      blockReport() tells the NameNode about all the locally-stored blocks. The NameNode returns an array of Blocks that have become obsolete and should be deleted. This function is meant to upload *all* the locally-stored blocks. It's invoked upon startup and then infrequently afterwards.
      Parameters:
      registration - datanode registration
      poolId - the block pool ID for the blocks
      reports - report of blocks per storage Each finalized block is represented as 3 longs. Each under- construction replica is represented as 4 longs. This is done instead of Block[] to reduce memory used by block reports.
      reports - report of blocks per storage
      context - Context information for this block report.
      Returns:
      - the next command for DN to process.
      Throws:
      IOException

    • cacheReport

      DatanodeCommand cacheReport(DatanodeRegistration registration, String poolId, List<Long> blockIds) throws IOException
      Communicates the complete list of locally cached blocks to the NameNode. This method is similar to blockReport(DatanodeRegistration, String, StorageBlockReport[], BlockReportContext), which is used to communicated blocks stored on disk.
      Parameters:
      registration - The datanode registration.
      poolId - The block pool ID for the blocks.
      blockIds - A list of block IDs.
      Returns:
      The DatanodeCommand.
      Throws:
      IOException

    • blockReceivedAndDeleted

      void blockReceivedAndDeleted(DatanodeRegistration registration, String poolId, StorageReceivedDeletedBlocks[] rcvdAndDeletedBlocks) throws IOException
      blockReceivedAndDeleted() allows the DataNode to tell the NameNode about recently-received and -deleted block data. For the case of received blocks, a hint for preferred replica to be deleted when there is any excessive blocks is provided. For example, whenever client code writes a new Block here, or another DataNode copies a Block to this DataNode, it will call blockReceived().
      Throws:
      IOException

    • errorReport

      void errorReport(DatanodeRegistration registration, int errorCode, String msg) throws IOException
      errorReport() tells the NameNode about something that has gone awry. Useful for debugging.
      Throws:
      IOException

    • versionRequest

      NamespaceInfo versionRequest() throws IOException
      Throws:
      IOException

    • reportBadBlocks

      void reportBadBlocks(org.apache.hadoop.hdfs.protocol.LocatedBlock[] blocks) throws IOException
      same as ClientProtocol.reportBadBlocks(LocatedBlock[]) }
      Throws:
      IOException

    • commitBlockSynchronization

      void commitBlockSynchronization(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, long newgenerationstamp, long newlength, boolean closeFile, boolean deleteblock, org.apache.hadoop.hdfs.protocol.DatanodeID[] newtargets, String[] newtargetstorages) throws IOException
      Commit block synchronization in lease recovery
      Throws:
      IOException