Package org.apache.hadoop.yarn.server.resourcemanager.rmnode

Class RMNodeImpl

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl
All Implemented Interfaces:
org.apache.hadoop.yarn.event.EventHandler<RMNodeEvent>, RMNode
@Private @Unstable public class RMNodeImpl extends Object implements RMNode, org.apache.hadoop.yarn.event.EventHandler<RMNodeEvent>
This class is used to keep track of all the applications/containers running on a node.

  • Constructor Details

    • RMNodeImpl

      public RMNodeImpl(org.apache.hadoop.yarn.api.records.NodeId nodeId, RMContext context, String hostName, int cmPort, int httpPort, org.apache.hadoop.net.Node node, org.apache.hadoop.yarn.api.records.Resource capability, String nodeManagerVersion)

    • RMNodeImpl

      public RMNodeImpl(org.apache.hadoop.yarn.api.records.NodeId nodeId, RMContext context, String hostName, int cmPort, int httpPort, org.apache.hadoop.net.Node node, org.apache.hadoop.yarn.api.records.Resource capability, String nodeManagerVersion, org.apache.hadoop.yarn.api.records.Resource physResource)

  • Method Details

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getHostName

      public String getHostName()
      Description copied from interface: RMNode
      the hostname of this node
      Specified by:
      getHostName in interface RMNode
      Returns:
      hostname of this node

    • getCommandPort

      public int getCommandPort()
      Description copied from interface: RMNode
      the command port for this node
      Specified by:
      getCommandPort in interface RMNode
      Returns:
      command port for this node

    • getHttpPort

      public int getHttpPort()
      Description copied from interface: RMNode
      the http port for this node
      Specified by:
      getHttpPort in interface RMNode
      Returns:
      http port for this node

    • setHttpPort

      public void setHttpPort(int port)

    • getNodeID

      public org.apache.hadoop.yarn.api.records.NodeId getNodeID()
      Description copied from interface: RMNode
      the node id of of this node.
      Specified by:
      getNodeID in interface RMNode
      Returns:
      the node id of this node.

    • getNodeAddress

      public String getNodeAddress()
      Description copied from interface: RMNode
      the ContainerManager address for this node.
      Specified by:
      getNodeAddress in interface RMNode
      Returns:
      the ContainerManager address for this node.

    • getHttpAddress

      public String getHttpAddress()
      Description copied from interface: RMNode
      the http-Address for this node.
      Specified by:
      getHttpAddress in interface RMNode
      Returns:
      the http-url address for this node

    • getTotalCapability

      public org.apache.hadoop.yarn.api.records.Resource getTotalCapability()
      Description copied from interface: RMNode
      the total available resource.
      Specified by:
      getTotalCapability in interface RMNode
      Returns:
      the total available resource.

    • getAllocatedContainerResource

      public org.apache.hadoop.yarn.api.records.Resource getAllocatedContainerResource()
      Description copied from interface: RMNode
      The total allocated resources to containers. This will include the sum of Guaranteed and Opportunistic containers queued + running + paused on the node.
      Specified by:
      getAllocatedContainerResource in interface RMNode
      Returns:
      the total allocated resources, including all Guaranteed and Opportunistic containers in queued, running and paused states.

    • isUpdatedCapability

      public boolean isUpdatedCapability()
      Description copied from interface: RMNode
      If the total available resources has been updated.
      Specified by:
      isUpdatedCapability in interface RMNode
      Returns:
      If the capability has been updated.

    • resetUpdatedCapability

      public void resetUpdatedCapability()
      Description copied from interface: RMNode
      Mark that the updated event has been processed.
      Specified by:
      resetUpdatedCapability in interface RMNode

    • getRackName

      public String getRackName()
      Description copied from interface: RMNode
      The rack name for this node manager.
      Specified by:
      getRackName in interface RMNode
      Returns:
      the rack name.

    • getNode

      public org.apache.hadoop.net.Node getNode()
      Description copied from interface: RMNode
      the Node information for this node.
      Specified by:
      getNode in interface RMNode
      Returns:
      Node information for this node.

    • getHealthReport

      public String getHealthReport()
      Description copied from interface: RMNode
      the latest health report received from this node.
      Specified by:
      getHealthReport in interface RMNode
      Returns:
      the latest health report received from this node.

    • setHealthReport

      public void setHealthReport(String healthReport)

    • setLastHealthReportTime

      public void setLastHealthReportTime(long lastHealthReportTime)

    • getLastHealthReportTime

      public long getLastHealthReportTime()
      Description copied from interface: RMNode
      the time of the latest health report received from this node.
      Specified by:
      getLastHealthReportTime in interface RMNode
      Returns:
      the time of the latest health report received from this node.

    • getNodeManagerVersion

      public String getNodeManagerVersion()
      Description copied from interface: RMNode
      the node manager version of the node received as part of the registration with the resource manager.
      Specified by:
      getNodeManagerVersion in interface RMNode
      Returns:
      node manager version.

    • getAggregatedContainersUtilization

      public org.apache.hadoop.yarn.api.records.ResourceUtilization getAggregatedContainersUtilization()
      Description copied from interface: RMNode
      the aggregated resource utilization of the containers.
      Specified by:
      getAggregatedContainersUtilization in interface RMNode
      Returns:
      the aggregated resource utilization of the containers.

    • setAggregatedContainersUtilization

      public void setAggregatedContainersUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization containersUtilization)

    • getNodeUtilization

      public org.apache.hadoop.yarn.api.records.ResourceUtilization getNodeUtilization()
      Description copied from interface: RMNode
      the total resource utilization of the node.
      Specified by:
      getNodeUtilization in interface RMNode
      Returns:
      the total resource utilization of the node.

    • setNodeUtilization

      public void setNodeUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization nodeUtilization)

    • getPhysicalResource

      public org.apache.hadoop.yarn.api.records.Resource getPhysicalResource()
      Description copied from interface: RMNode
      the physical resources in the node.
      Specified by:
      getPhysicalResource in interface RMNode
      Returns:
      the physical resources in the node.

    • setPhysicalResource

      public void setPhysicalResource(org.apache.hadoop.yarn.api.records.Resource physicalResource)

    • getState

      public org.apache.hadoop.yarn.api.records.NodeState getState()
      Specified by:
      getState in interface RMNode

    • getAppsToCleanup

      public List<org.apache.hadoop.yarn.api.records.ApplicationId> getAppsToCleanup()
      Specified by:
      getAppsToCleanup in interface RMNode

    • getRunningApps

      public List<org.apache.hadoop.yarn.api.records.ApplicationId> getRunningApps()
      Specified by:
      getRunningApps in interface RMNode

    • getContainersToCleanUp

      public List<org.apache.hadoop.yarn.api.records.ContainerId> getContainersToCleanUp()
      Specified by:
      getContainersToCleanUp in interface RMNode

    • setAndUpdateNodeHeartbeatResponse

      public void setAndUpdateNodeHeartbeatResponse(org.apache.hadoop.yarn.server.api.protocolrecords.NodeHeartbeatResponse response)
      Description copied from interface: RMNode
      Update a NodeHeartbeatResponse with the list of containers and applications to clean up for this node, and the containers to be updated.
      Specified by:
      setAndUpdateNodeHeartbeatResponse in interface RMNode
      Parameters:
      response - the NodeHeartbeatResponse to update

    • getToBeUpdatedContainers

      @VisibleForTesting public Collection<org.apache.hadoop.yarn.api.records.Container> getToBeUpdatedContainers()

    • getLastNodeHeartBeatResponse

      public org.apache.hadoop.yarn.server.api.protocolrecords.NodeHeartbeatResponse getLastNodeHeartBeatResponse()
      Specified by:
      getLastNodeHeartBeatResponse in interface RMNode

    • resetLastNodeHeartBeatResponse

      public void resetLastNodeHeartBeatResponse()
      Description copied from interface: RMNode
      Reset lastNodeHeartbeatResponse's ID to 0.
      Specified by:
      resetLastNodeHeartBeatResponse in interface RMNode

    • calculateHeartBeatInterval

      public long calculateHeartBeatInterval(long defaultInterval, long minInterval, long maxInterval, float speedupFactor, float slowdownFactor)
      Specified by:
      calculateHeartBeatInterval in interface RMNode

    • handle

      public void handle(RMNodeEvent event)
      Specified by:
      handle in interface org.apache.hadoop.yarn.event.EventHandler<RMNodeEvent>

    • deactivateNode

      public static void deactivateNode(RMNodeImpl rmNode, org.apache.hadoop.yarn.api.records.NodeState finalState)
      Put a node in deactivated (decommissioned or shutdown) status.
      Parameters:
      rmNode - RMNode.
      finalState - NodeState.

    • reportNodeRunning

      public static void reportNodeRunning(RMNodeImpl rmNode, List<org.apache.hadoop.yarn.server.api.protocolrecords.NMContainerStatus> containers)
      Report node is RUNNING.
      Parameters:
      rmNode - RMNode.
      containers - NMContainerStatus List.

    • reportNodeUnusable

      public static void reportNodeUnusable(RMNodeImpl rmNode, org.apache.hadoop.yarn.api.records.NodeState finalState)
      Report node is UNUSABLE and update metrics.
      Parameters:
      rmNode - RMNode.
      finalState - NodeState.

    • pullContainerUpdates

      public List<UpdatedContainerInfo> pullContainerUpdates()
      Description copied from interface: RMNode
      Get and clear the list of containerUpdates accumulated across NM heartbeats.
      Specified by:
      pullContainerUpdates in interface RMNode
      Returns:
      containerUpdates accumulated across NM heartbeats.

    • setNextHeartBeat

      @VisibleForTesting public void setNextHeartBeat(boolean nextHeartBeat)

    • getQueueSize

      @VisibleForTesting public int getQueueSize()

    • getUpdatedExistContainers

      @VisibleForTesting public Map<org.apache.hadoop.yarn.api.records.ContainerId,org.apache.hadoop.yarn.api.records.ContainerStatus> getUpdatedExistContainers()

    • getLaunchedContainers

      @VisibleForTesting public Set<org.apache.hadoop.yarn.api.records.ContainerId> getLaunchedContainers()

    • getCompletedContainers

      @VisibleForTesting public Set<org.apache.hadoop.yarn.api.records.ContainerId> getCompletedContainers()

    • getNodeLabels

      public Set<String> getNodeLabels()
      Description copied from interface: RMNode
      Get set of labels in this node
      Specified by:
      getNodeLabels in interface RMNode
      Returns:
      labels in this node

    • pullNewlyIncreasedContainers

      public List<org.apache.hadoop.yarn.api.records.Container> pullNewlyIncreasedContainers()
      Specified by:
      pullNewlyIncreasedContainers in interface RMNode

    • getOriginalTotalCapability

      public org.apache.hadoop.yarn.api.records.Resource getOriginalTotalCapability()

    • getOpportunisticContainersStatus

      public org.apache.hadoop.yarn.server.api.records.OpportunisticContainersStatus getOpportunisticContainersStatus()
      Specified by:
      getOpportunisticContainersStatus in interface RMNode

    • setOpportunisticContainersStatus

      public void setOpportunisticContainersStatus(org.apache.hadoop.yarn.server.api.records.OpportunisticContainersStatus opportunisticContainersStatus)

    • getUntrackedTimeStamp

      public long getUntrackedTimeStamp()
      Specified by:
      getUntrackedTimeStamp in interface RMNode

    • setUntrackedTimeStamp

      public void setUntrackedTimeStamp(long ts)
      Specified by:
      setUntrackedTimeStamp in interface RMNode

    • getDecommissioningTimeout

      public Integer getDecommissioningTimeout()
      Specified by:
      getDecommissioningTimeout in interface RMNode

    • getAllocationTagsWithCount

      public Map<String,Long> getAllocationTagsWithCount()
      Description copied from interface: RMNode
      Get the allocation tags and their counts associated with this node.
      Specified by:
      getAllocationTagsWithCount in interface RMNode
      Returns:
      a map of each allocation tag and its count.

    • getRMContext

      public RMContext getRMContext()
      Specified by:
      getRMContext in interface RMNode
      Returns:
      the RM context associated with this RM node.

    • getAllNodeAttributes

      public Set<org.apache.hadoop.yarn.api.records.NodeAttribute> getAllNodeAttributes()
      Specified by:
      getAllNodeAttributes in interface RMNode
      Returns:
      all node attributes as a Set.

    • getContainersToBeRemovedFromNM

      @VisibleForTesting public Set<org.apache.hadoop.yarn.api.records.ContainerId> getContainersToBeRemovedFromNM()