Package org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo

Class FifoScheduler

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>, Recoverable, ResourceScheduler, YarnScheduler
@LimitedPrivate("yarn") @Evolving public class FifoScheduler extends AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode> implements org.apache.hadoop.conf.Configurable

  • Constructor Details

    • FifoScheduler

      public FifoScheduler()

  • Method Details

    • serviceInit

      public void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
      Overrides:
      serviceInit in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
      Throws:
      Exception

    • serviceStart

      public void serviceStart() throws Exception
      Overrides:
      serviceStart in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
      Throws:
      Exception

    • serviceStop

      public void serviceStop() throws Exception
      Overrides:
      serviceStop in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
      Throws:
      Exception

    • setConf

      public void setConf(org.apache.hadoop.conf.Configuration conf)
      Specified by:
      setConf in interface org.apache.hadoop.conf.Configurable

    • getConf

      public org.apache.hadoop.conf.Configuration getConf()
      Specified by:
      getConf in interface org.apache.hadoop.conf.Configurable

    • getNumClusterNodes

      public int getNumClusterNodes()
      Description copied from interface: YarnScheduler
      Get the number of nodes available in the cluster.
      Specified by:
      getNumClusterNodes in interface YarnScheduler
      Returns:
      the number of available nodes.

    • setRMContext

      public void setRMContext(RMContext rmContext)
      Description copied from interface: ResourceScheduler
      Set RMContext for ResourceScheduler. This method should be called immediately after instantiating a scheduler once.
      Specified by:
      setRMContext in interface ResourceScheduler
      Parameters:
      rmContext - created by ResourceManager

    • reinitialize

      public void reinitialize(org.apache.hadoop.conf.Configuration conf, RMContext rmContext) throws IOException
      Description copied from interface: ResourceScheduler
      Re-initialize the ResourceScheduler.
      Specified by:
      reinitialize in interface ResourceScheduler
      Overrides:
      reinitialize in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
      Parameters:
      conf - configuration
      rmContext - RMContext.
      Throws:
      IOException - an I/O exception has occurred.

    • allocate

      public Allocation allocate(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask, List<org.apache.hadoop.yarn.api.records.SchedulingRequest> schedulingRequests, List<org.apache.hadoop.yarn.api.records.ContainerId> release, List<String> blacklistAdditions, List<String> blacklistRemovals, ContainerUpdates updateRequests)
      Description copied from interface: YarnScheduler
      The main API between the ApplicationMaster and the Scheduler. The ApplicationMaster may request/update container resources, number of containers, node/rack preference for allocations etc. to the Scheduler.
      Specified by:
      allocate in interface YarnScheduler
      Parameters:
      applicationAttemptId - the id of the application attempt.
      ask - the request made by an application to obtain various allocations like host/rack, resource, number of containers, relaxLocality etc., see ResourceRequest.
      schedulingRequests - similar to ask, but with added ability to specify allocation tags etc., see SchedulingRequest.
      release - the list of containers to be released.
      blacklistAdditions - places (node/rack) to be added to the blacklist.
      blacklistRemovals - places (node/rack) to be removed from the blacklist.
      updateRequests - container promotion/demotion updates.
      Returns:
      the Allocation for the application.

    • addApplication

      @VisibleForTesting public void addApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String queue, String user, boolean isAppRecovering, boolean unmanagedAM)

    • addApplicationAttempt

      @VisibleForTesting public void addApplicationAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId, boolean transferStateFromPreviousAttempt, boolean isAttemptRecovering)

    • handle

      public void handle(SchedulerEvent event)
      Specified by:
      handle in interface org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>

    • completedContainerInternal

      @Lock(FifoScheduler.class) protected void completedContainerInternal(RMContainer rmContainer, org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus, RMContainerEventType event)
      Specified by:
      completedContainerInternal in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>

    • getQueueInfo

      public org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(String queueName, boolean includeChildQueues, boolean recursive)
      Description copied from interface: YarnScheduler
      Get queue information.
      Specified by:
      getQueueInfo in interface YarnScheduler
      Parameters:
      queueName - queue name
      includeChildQueues - include child queues?
      recursive - get children queues?
      Returns:
      queue information

    • getQueueUserAclInfo

      public List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueUserAclInfo()
      Description copied from interface: YarnScheduler
      Get acls for queues for current user.
      Specified by:
      getQueueUserAclInfo in interface YarnScheduler
      Returns:
      acls for queues for current user

    • getResourceCalculator

      public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
      Specified by:
      getResourceCalculator in interface YarnScheduler

    • recover

      public void recover(RMStateStore.RMState state)
      Specified by:
      recover in interface Recoverable

    • getRMContainer

      public RMContainer getRMContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
      Description copied from interface: YarnScheduler
      Get the container for the given containerId.
      Specified by:
      getRMContainer in interface YarnScheduler
      Overrides:
      getRMContainer in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
      Parameters:
      containerId - the given containerId.
      Returns:
      the container for the given containerId.

    • getRootQueueMetrics

      public QueueMetrics getRootQueueMetrics()
      Description copied from interface: YarnScheduler
      Get the root queue for the scheduler.
      Specified by:
      getRootQueueMetrics in interface YarnScheduler
      Returns:
      the root queue for the scheduler.

    • checkAccess

      public boolean checkAccess(org.apache.hadoop.security.UserGroupInformation callerUGI, org.apache.hadoop.yarn.api.records.QueueACL acl, String queueName)
      Description copied from interface: YarnScheduler
      Check if the user has permission to perform the operation. If the user has QueueACL.ADMINISTER_QUEUE permission, this user can view/modify the applications in this queue.
      Specified by:
      checkAccess in interface YarnScheduler
      Parameters:
      callerUGI - caller UserGroupInformation.
      acl - queue ACL.
      queueName - queue Name.
      Returns:
      true if the user has the permission, false otherwise

    • getAppsInQueue

      public List<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> getAppsInQueue(String queueName)
      Description copied from interface: YarnScheduler
      Gets the apps under a given queue
      Specified by:
      getAppsInQueue in interface YarnScheduler
      Parameters:
      queueName - the name of the queue.
      Returns:
      a collection of app attempt ids in the given queue.

    • getUsedResource

      public org.apache.hadoop.yarn.api.records.Resource getUsedResource()

    • nodeUpdate

      protected void nodeUpdate(RMNode nm)
      Description copied from class: AbstractYarnScheduler
      Process a heartbeat update from a node.
      Overrides:
      nodeUpdate in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
      Parameters:
      nm - The RMNode corresponding to the NodeManager

    • killContainer

      @VisibleForTesting public void killContainer(RMContainer container)
      Description copied from class: AbstractYarnScheduler
      Kill a RMContainer. This is meant to be called in tests only to simulate AM container failures.
      Specified by:
      killContainer in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>
      Parameters:
      container - the container to kill

    • recoverContainersOnNode

      public void recoverContainersOnNode(List<org.apache.hadoop.yarn.server.api.protocolrecords.NMContainerStatus> containerReports, RMNode nm)
      Overrides:
      recoverContainersOnNode in class AbstractYarnScheduler<FifoAppAttempt,FiCaSchedulerNode>