Package org.apache.hadoop.yarn.server.resourcemanager.rmapp

Class RMAppImpl

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl
All Implemented Interfaces:
org.apache.hadoop.yarn.event.EventHandler<RMAppEvent>, Recoverable, RMApp
public class RMAppImpl extends Object implements RMApp, Recoverable

  • Constructor Details

    • RMAppImpl

      public RMAppImpl(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, RMContext rmContext, org.apache.hadoop.conf.Configuration config, String name, String user, String queue, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext, YarnScheduler scheduler, ApplicationMasterService masterService, long submitTime, String applicationType, Set<String> applicationTags, List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs)

    • RMAppImpl

      public RMAppImpl(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, RMContext rmContext, org.apache.hadoop.conf.Configuration config, String name, String user, String queue, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext, YarnScheduler scheduler, ApplicationMasterService masterService, long submitTime, String applicationType, Set<String> applicationTags, List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs, ApplicationPlacementContext placementContext, long startTime)

    • RMAppImpl

      public RMAppImpl(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, RMContext rmContext, org.apache.hadoop.conf.Configuration config, String name, org.apache.hadoop.security.UserGroupInformation userUgi, String queue, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext submissionContext, YarnScheduler scheduler, ApplicationMasterService masterService, long submitTime, String applicationType, Set<String> applicationTags, List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs, ApplicationPlacementContext placementContext, long startTime)

  • Method Details

    • startTimelineCollector

      public void startTimelineCollector()
      Starts the application level timeline collector for this app. This should be used only if the timeline service v.2 is enabled.

    • stopTimelineCollector

      public void stopTimelineCollector()
      Stops the application level timeline collector for this app. This should be used only if the timeline service v.2 is enabled.

    • getApplicationId

      public org.apache.hadoop.yarn.api.records.ApplicationId getApplicationId()
      Description copied from interface: RMApp
      The application id for this RMApp.
      Specified by:
      getApplicationId in interface RMApp
      Returns:
      the ApplicationId for this RMApp.

    • getApplicationSubmissionContext

      public org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext getApplicationSubmissionContext()
      Description copied from interface: RMApp
      The application submission context for this RMApp
      Specified by:
      getApplicationSubmissionContext in interface RMApp
      Returns:
      the ApplicationSubmissionContext for this RMApp

    • getFinalApplicationStatus

      public org.apache.hadoop.yarn.api.records.FinalApplicationStatus getFinalApplicationStatus()
      Description copied from interface: RMApp
      The final finish state of the AM when unregistering as in FinishApplicationMasterRequest.setFinalApplicationStatus(FinalApplicationStatus).
      Specified by:
      getFinalApplicationStatus in interface RMApp
      Returns:
      the final finish state of the AM as set in FinishApplicationMasterRequest.setFinalApplicationStatus(FinalApplicationStatus).

    • getState

      public RMAppState getState()
      Description copied from interface: RMApp
      The current state of the RMApp.
      Specified by:
      getState in interface RMApp
      Returns:
      the current state RMAppState for this application.

    • getUser

      public String getUser()
      Description copied from interface: RMApp
      The user who submitted this application.
      Specified by:
      getUser in interface RMApp
      Returns:
      the user who submitted the application.

    • getProgress

      public float getProgress()
      Description copied from interface: RMApp
      Progress of application.
      Specified by:
      getProgress in interface RMApp
      Returns:
      the progress of the RMApp.

    • getRMAppAttempt

      public RMAppAttempt getRMAppAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)
      Description copied from interface: RMApp
      RMApp can have multiple application attempts RMAppAttempt. This method returns the RMAppAttempt corresponding to ApplicationAttemptId.
      Specified by:
      getRMAppAttempt in interface RMApp
      Parameters:
      appAttemptId - the application attempt id
      Returns:
      the RMAppAttempt corresponding to the ApplicationAttemptId.

    • getQueue

      public String getQueue()
      Description copied from interface: RMApp
      Each Application is submitted to a queue decided by ApplicationSubmissionContext.setQueue(String). This method returns the queue to which an application was submitted.
      Specified by:
      getQueue in interface RMApp
      Returns:
      the queue to which the application was submitted to.

    • setQueue

      public void setQueue(String queue)
      Description copied from interface: RMApp
      Reflects a change in the application's queue from the one specified in the ApplicationSubmissionContext.
      Specified by:
      setQueue in interface RMApp
      Parameters:
      queue - the new queue name

    • getCollectorData

      public org.apache.hadoop.yarn.server.api.records.AppCollectorData getCollectorData()
      Description copied from interface: RMApp
      The timeline collector information for the application. It should be used only if the timeline service v.2 is enabled.
      Specified by:
      getCollectorData in interface RMApp
      Returns:
      the data for the application's collector, including collector address, RM ID, version and collector token. Return null if the timeline service v.2 is not enabled.

    • setCollectorData

      public void setCollectorData(org.apache.hadoop.yarn.server.api.records.AppCollectorData incomingData)

    • getCollectorInfo

      public org.apache.hadoop.yarn.api.records.CollectorInfo getCollectorInfo()
      Description copied from interface: RMApp
      The timeline collector information to be sent to AM. It should be used only if the timeline service v.2 is enabled.
      Specified by:
      getCollectorInfo in interface RMApp
      Returns:
      collector info, including collector address and collector token. Return null if the timeline service v.2 is not enabled.

    • removeCollectorData

      public void removeCollectorData()

    • getName

      public String getName()
      Description copied from interface: RMApp
      The name of the application as set in ApplicationSubmissionContext.setApplicationName(String).
      Specified by:
      getName in interface RMApp
      Returns:
      the name of the application.

    • getCurrentAppAttempt

      public RMAppAttempt getCurrentAppAttempt()
      Description copied from interface: RMApp
      RMApp can have multiple application attempts RMAppAttempt. This method returns the current RMAppAttempt.
      Specified by:
      getCurrentAppAttempt in interface RMApp
      Returns:
      the current RMAppAttempt

    • getAppAttempts

      public Map<org.apache.hadoop.yarn.api.records.ApplicationAttemptId,RMAppAttempt> getAppAttempts()
      Description copied from interface: RMApp
      RMApp can have multiple application attempts RMAppAttempt. This method returns the all RMAppAttempts for the RMApp.
      Specified by:
      getAppAttempts in interface RMApp
      Returns:
      all RMAppAttempts for the RMApp.

    • pullRMNodeUpdates

      public int pullRMNodeUpdates(Map<RMNode,org.apache.hadoop.yarn.api.records.NodeUpdateType> upNodes)
      Description copied from interface: RMApp
      To receive the collection of all RMNodes whose updates have been received by the RMApp. Updates can be node becoming lost or becoming healthy etc. The method clears the information from the RMApp. So each call to this method gives the delta from the previous call.
      Specified by:
      pullRMNodeUpdates in interface RMApp
      Parameters:
      upNodes - Map into which the updates are transferred, with each node updates as the key, and the NodeUpdateType for that update as the corresponding value.
      Returns:
      the number of nodes added to the Map

    • createAndGetApplicationReport

      public org.apache.hadoop.yarn.api.records.ApplicationReport createAndGetApplicationReport(String clientUserName, boolean allowAccess)
      Description copied from interface: RMApp
      To get the status of an application in the RM, this method can be used. If full access is not allowed then the following fields in the report will be stubbed:
      • host - set to "N/A"
      • RPC port - set to -1
      • client token - set to "N/A"
      • diagnostics - set to "N/A"
      • tracking URL - set to "N/A"
      • original tracking URL - set to "N/A"
      • resource usage report - all values are -1
      Specified by:
      createAndGetApplicationReport in interface RMApp
      Parameters:
      clientUserName - the user name of the client requesting the report
      allowAccess - whether to allow full access to the report
      Returns:
      the ApplicationReport detailing the status of the application.

    • getFinishTime

      public long getFinishTime()
      Description copied from interface: RMApp
      The finish time of the RMApp
      Specified by:
      getFinishTime in interface RMApp
      Returns:
      the finish time of the application.,

    • getStartTime

      public long getStartTime()
      Description copied from interface: RMApp
      the start time of the application.
      Specified by:
      getStartTime in interface RMApp
      Returns:
      the start time of the application.

    • getLaunchTime

      public long getLaunchTime()
      Description copied from interface: RMApp
      The launch time of the application. Since getStartTime() returns what is essentially submit time, this new field is to prevent potential backwards compatibility issues.
      Specified by:
      getLaunchTime in interface RMApp
      Returns:
      the launch time of the application.

    • getSubmitTime

      public long getSubmitTime()
      Description copied from interface: RMApp
      the submit time of the application.
      Specified by:
      getSubmitTime in interface RMApp
      Returns:
      the submit time of the application.

    • getTrackingUrl

      public String getTrackingUrl()
      Description copied from interface: RMApp
      The tracking url for the application master.
      Specified by:
      getTrackingUrl in interface RMApp
      Returns:
      the tracking url for the application master.

    • getOriginalTrackingUrl

      public String getOriginalTrackingUrl()
      Description copied from interface: RMApp
      The original tracking url for the application master.
      Specified by:
      getOriginalTrackingUrl in interface RMApp
      Returns:
      the original tracking url for the application master.

    • getDiagnostics

      public StringBuilder getDiagnostics()
      Description copied from interface: RMApp
      the diagnostics information for the application master.
      Specified by:
      getDiagnostics in interface RMApp
      Returns:
      the diagnostics information for the application master.

    • getMaxAppAttempts

      public int getMaxAppAttempts()
      Description copied from interface: RMApp
      The number of max attempts of the application.
      Specified by:
      getMaxAppAttempts in interface RMApp
      Returns:
      the number of max attempts of the application.

    • handle

      public void handle(RMAppEvent event)
      Specified by:
      handle in interface org.apache.hadoop.yarn.event.EventHandler<RMAppEvent>

    • recover

      public void recover(RMStateStore.RMState state)
      Specified by:
      recover in interface Recoverable

    • getNumFailedAppAttempts

      public int getNumFailedAppAttempts()

    • getApplicationType

      public String getApplicationType()
      Description copied from interface: RMApp
      Returns the application type
      Specified by:
      getApplicationType in interface RMApp
      Returns:
      the application type.

    • getApplicationTags

      public Set<String> getApplicationTags()
      Description copied from interface: RMApp
      Get tags for the application
      Specified by:
      getApplicationTags in interface RMApp
      Returns:
      tags corresponding to the application

    • isAppFinalStateStored

      public boolean isAppFinalStateStored()
      Description copied from interface: RMApp
      Check whether this application's state has been saved to the state store.
      Specified by:
      isAppFinalStateStored in interface RMApp
      Returns:
      the flag indicating whether the applications's state is stored.

    • createApplicationState

      public org.apache.hadoop.yarn.api.records.YarnApplicationState createApplicationState()
      Description copied from interface: RMApp
      Create the external user-facing state of ApplicationMaster from the current state of the RMApp.
      Specified by:
      createApplicationState in interface RMApp
      Returns:
      the external user-facing state of ApplicationMaster.

    • isAppInFinalState

      public static boolean isAppInFinalState(RMApp rmApp)

    • isAppInCompletedStates

      public boolean isAppInCompletedStates()
      Description copied from interface: RMApp
      To verify whether app has reached in its completing/completed states.
      Specified by:
      isAppInCompletedStates in interface RMApp
      Returns:
      True/False to confirm whether app is in final states

    • getApplicationPlacementContext

      public ApplicationPlacementContext getApplicationPlacementContext()
      Description copied from interface: RMApp
      Get the application -> queue placement context
      Specified by:
      getApplicationPlacementContext in interface RMApp
      Returns:
      ApplicationPlacementContext

    • getRecoveredFinalState

      public RMAppState getRecoveredFinalState()

    • getRanNodes

      public Set<org.apache.hadoop.yarn.api.records.NodeId> getRanNodes()
      Description copied from interface: RMApp
      Nodes on which the containers for this RMApp ran.
      Specified by:
      getRanNodes in interface RMApp
      Returns:
      the set of nodes that ran any containers from this RMApp Add more node on which containers for this RMApp ran

    • getRMAppMetrics

      public RMAppMetrics getRMAppMetrics()
      Description copied from interface: RMApp
      Get RMAppMetrics of the RMApp.
      Specified by:
      getRMAppMetrics in interface RMApp
      Returns:
      metrics

    • setSystemClock

      @Private @VisibleForTesting public void setSystemClock(org.apache.hadoop.yarn.util.Clock clock)

    • getReservationId

      public org.apache.hadoop.yarn.api.records.ReservationId getReservationId()
      Specified by:
      getReservationId in interface RMApp

    • getAMResourceRequests

      public List<org.apache.hadoop.yarn.api.records.ResourceRequest> getAMResourceRequests()
      Specified by:
      getAMResourceRequests in interface RMApp

    • getLogAggregationReportsForApp

      public Map<org.apache.hadoop.yarn.api.records.NodeId,org.apache.hadoop.yarn.server.api.protocolrecords.LogAggregationReport> getLogAggregationReportsForApp()
      Specified by:
      getLogAggregationReportsForApp in interface RMApp

    • aggregateLogReport

      public void aggregateLogReport(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.server.api.protocolrecords.LogAggregationReport report)

    • getLogAggregationFailureMessagesForNM

      public String getLogAggregationFailureMessagesForNM(org.apache.hadoop.yarn.api.records.NodeId nodeId)

    • getLogAggregationStatusForAppReport

      public org.apache.hadoop.yarn.api.records.LogAggregationStatus getLogAggregationStatusForAppReport()
      Specified by:
      getLogAggregationStatusForAppReport in interface RMApp

    • getAppNodeLabelExpression

      public String getAppNodeLabelExpression()
      Specified by:
      getAppNodeLabelExpression in interface RMApp

    • getAmNodeLabelExpression

      public String getAmNodeLabelExpression()
      Description copied from interface: RMApp
      Return the node label expression of the AM container.
      Specified by:
      getAmNodeLabelExpression in interface RMApp
      Returns:
      the node label expression.

    • getCallerContext

      public org.apache.hadoop.ipc.CallerContext getCallerContext()
      Specified by:
      getCallerContext in interface RMApp

    • getNextAttemptId

      @Private @VisibleForTesting public int getNextAttemptId()

    • getApplicationTimeouts

      public Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long> getApplicationTimeouts()
      Specified by:
      getApplicationTimeouts in interface RMApp

    • updateApplicationTimeout

      public void updateApplicationTimeout(Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long> updateTimeout)

    • getApplicationPriority

      public org.apache.hadoop.yarn.api.records.Priority getApplicationPriority()
      Description copied from interface: RMApp
      Get priority of the application.
      Specified by:
      getApplicationPriority in interface RMApp
      Returns:
      priority

    • setApplicationPriority

      public void setApplicationPriority(org.apache.hadoop.yarn.api.records.Priority applicationPriority)

    • getApplicationSchedulingEnvs

      public Map<String,String> getApplicationSchedulingEnvs()
      Description copied from interface: RMApp
      Get the application scheduling environment variables.
      Specified by:
      getApplicationSchedulingEnvs in interface RMApp
      Returns:
      Map of envs related to application scheduling preferences.

    • onInvalidStateTransition

      protected void onInvalidStateTransition(RMAppEventType rmAppEventType, RMAppState state)
      catch the InvalidStateTransition.
      Parameters:
      state - RMAppState.
      rmAppEventType - RMAppEventType.

    • getRealUser

      public String getRealUser()
      Specified by:
      getRealUser in interface RMApp