Package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity

Class AbstractParentQueue

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractCSQueue
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractParentQueue
All Implemented Interfaces:
CSQueue, Queue, SchedulerQueue<CSQueue>
Direct Known Subclasses:
AbstractManagedParentQueue, ParentQueue
public abstract class AbstractParentQueue extends AbstractCSQueue

  • Field Details

    • childQueues

      protected final List<CSQueue> childQueues

  • Constructor Details

  • Method Details

    • setupQueueConfigs

      protected void setupQueueConfigs(org.apache.hadoop.yarn.api.records.Resource clusterResource) throws IOException
      Description copied from class: AbstractCSQueue
      Initialize queue properties that are based on configuration.
      Overrides:
      setupQueueConfigs in class AbstractCSQueue
      Parameters:
      clusterResource - overall resource of the cluster
      Throws:
      IOException - if configuration is set in a way that is inconsistent

    • setDynamicQueueACLProperties

      protected void setDynamicQueueACLProperties()
      Overrides:
      setDynamicQueueACLProperties in class AbstractCSQueue

    • getCapacityConfigurationTypeForQueues

      public AbstractParentQueue.QueueCapacityType getCapacityConfigurationTypeForQueues(Collection<CSQueue> queues) throws IOException
      Throws:
      IOException

    • getQueueInfo

      public org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(boolean includeChildQueues, boolean recursive)
      Description copied from interface: Queue
      Get queue information
      Parameters:
      includeChildQueues - include child queues?
      recursive - recursively get child queue information?
      Returns:
      queue information

    • getQueueUserAclInfo

      public List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueUserAclInfo(org.apache.hadoop.security.UserGroupInformation user)
      Description copied from interface: Queue
      Get queue ACLs for given user.
      Parameters:
      user - username
      Returns:
      queue ACLs for user

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • createNewQueue

      public CSQueue createNewQueue(String childQueuePath, boolean isLeaf) throws SchedulerDynamicEditException
      Throws:
      SchedulerDynamicEditException

    • removeChildQueue

      public void removeChildQueue(CSQueue queue) throws SchedulerDynamicEditException
      Throws:
      SchedulerDynamicEditException

    • isEligibleForAutoQueueCreation

      public boolean isEligibleForAutoQueueCreation()
      Check whether this queue supports adding additional child queues dynamically.
      Returns:
      true, if queue is eligible to create additional queues dynamically, false otherwise

    • isEligibleForLegacyAutoQueueCreation

      public boolean isEligibleForLegacyAutoQueueCreation()
      Check whether this queue supports legacy(v1) dynamic child queue creation.
      Returns:
      true if queue is eligible to create child queues dynamically using the legacy system, false otherwise

    • reinitialize

      public void reinitialize(CSQueue newlyParsedQueue, org.apache.hadoop.yarn.api.records.Resource clusterResource) throws IOException
      Description copied from interface: CSQueue
      Reinitialize the queue.
      Parameters:
      newlyParsedQueue - new queue to re-initalize from
      clusterResource - resources in the cluster
      Throws:
      IOException - an I/O exception has occurred.

    • submitApplication

      public void submitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String user, String queue) throws org.apache.hadoop.security.AccessControlException
      Description copied from interface: CSQueue
      Submit a new application to the queue.
      Parameters:
      applicationId - the applicationId of the application being submitted
      user - user who submitted the application
      queue - queue to which the application is submitted
      Throws:
      org.apache.hadoop.security.AccessControlException - if any acl violation is there.

    • validateSubmitApplication

      public void validateSubmitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String userName, String queue) throws org.apache.hadoop.security.AccessControlException
      Description copied from interface: CSQueue
      Validate submitApplication api so that moveApplication do a pre-check.
      Specified by:
      validateSubmitApplication in interface CSQueue
      Overrides:
      validateSubmitApplication in class AbstractCSQueue
      Parameters:
      applicationId - Application ID
      userName - User Name
      queue - Queue Name
      Throws:
      org.apache.hadoop.security.AccessControlException - if any acl violation is there.

    • submitApplicationAttempt

      public void submitApplicationAttempt(FiCaSchedulerApp application, String userName)
      Description copied from interface: CSQueue
      Submit an application attempt to the queue.
      Parameters:
      application - application whose attempt is being submitted.
      userName - userName who submitted the application.

    • submitApplicationAttempt

      public void submitApplicationAttempt(FiCaSchedulerApp application, String userName, boolean isMoveApp)
      Description copied from interface: CSQueue
      Submit an application attempt to the queue.
      Parameters:
      application - application whose attempt is being submitted
      userName - user who submitted the application attempt
      isMoveApp - is application being moved across the queue

    • finishApplicationAttempt

      public void finishApplicationAttempt(FiCaSchedulerApp application, String queue)
      Description copied from interface: CSQueue
      An application attempt submitted to this queue has finished.
      Parameters:
      application - application attempt.
      queue - queue.

    • finishApplication

      public void finishApplication(org.apache.hadoop.yarn.api.records.ApplicationId application, String user)
      Description copied from interface: CSQueue
      An application submitted to this queue has finished.
      Parameters:
      application - applicationId.
      user - user who submitted the application

    • assignContainers

      public CSAssignment assignContainers(org.apache.hadoop.yarn.api.records.Resource clusterResource, CandidateNodeSet<FiCaSchedulerNode> candidates, ResourceLimits resourceLimits, SchedulingMode schedulingMode)
      Description copied from interface: CSQueue
      Assign containers to applications in the queue or it's children (if any).
      Parameters:
      clusterResource - the resource of the cluster.
      candidates - CandidateNodeSet the nodes that are considered for the current placement.
      resourceLimits - how much overall resource of this queue can use.
      schedulingMode - Type of exclusive check when assign container on a NodeManager, see SchedulingMode.
      Returns:
      the assignment

    • getResourceLimitsOfChild

      public ResourceLimits getResourceLimitsOfChild(CSQueue child, org.apache.hadoop.yarn.api.records.Resource clusterResource, ResourceLimits parentLimits, String nodePartition, boolean netLimit)

    • completedContainer

      public void completedContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerApp application, FiCaSchedulerNode node, RMContainer rmContainer, org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus, RMContainerEventType event, CSQueue completedChildQueue, boolean sortQueues)
      Description copied from interface: CSQueue
      A container assigned to the queue has completed.
      Parameters:
      clusterResource - the resource of the cluster
      application - application to which the container was assigned
      node - node on which the container completed
      rmContainer - completed container, null if it was just a reservation
      containerStatus - ContainerStatus for the completed container
      event - event to be sent to the container
      completedChildQueue - CSQueue to reinsert in childQueues
      sortQueues - indicates whether it should re-sort the queues

    • refreshAfterResourceCalculation

      public void refreshAfterResourceCalculation(org.apache.hadoop.yarn.api.records.Resource clusterResource, ResourceLimits resourceLimits)

    • updateClusterResource

      public void updateClusterResource(org.apache.hadoop.yarn.api.records.Resource clusterResource, ResourceLimits resourceLimits)
      Description copied from interface: CSQueue
      Update the cluster resource for queues as we add/remove nodes
      Parameters:
      clusterResource - the current cluster resource
      resourceLimits - the current ResourceLimits

    • updateClusterResourceLegacyMode

      public void updateClusterResourceLegacyMode(org.apache.hadoop.yarn.api.records.Resource clusterResource, ResourceLimits resourceLimits)

    • hasChildQueues

      public boolean hasChildQueues()

    • getChildQueues

      public List<CSQueue> getChildQueues()
      Description copied from interface: CSQueue
      Get child queues
      Returns:
      child queues

    • getChildQueuesByTryLock

      public List<CSQueue> getChildQueuesByTryLock()
      Description copied from interface: CSQueue
      Get child queues By tryLock.
      Returns:
      child queues

    • recoverContainer

      public void recoverContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, SchedulerApplicationAttempt attempt, RMContainer rmContainer)
      Description copied from interface: Queue
      Recover the state of the queue for a given container.
      Parameters:
      clusterResource - the resource of the cluster
      attempt - the application for which the container was allocated
      rmContainer - the container that was recovered.

    • getAbstractUsersManager

      public ActiveUsersManager getAbstractUsersManager()
      Description copied from interface: CSQueue
      Get the AbstractUsersManager for the queue.
      Returns:
      the AbstractUsersManager for the queue

    • collectSchedulerApplications

      public void collectSchedulerApplications(Collection<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> apps)
      Description copied from interface: CSQueue
      Adds all applications in the queue and its subqueues to the given collection.
      Parameters:
      apps - the collection to add the applications to

    • attachContainer

      public void attachContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerApp application, RMContainer rmContainer)
      Description copied from interface: CSQueue
      Attach a container to this queue
      Parameters:
      clusterResource - the current cluster resource
      application - application to which the container was assigned
      rmContainer - the container to attach

    • detachContainer

      public void detachContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerApp application, RMContainer rmContainer)
      Description copied from interface: CSQueue
      Detach a container from this queue
      Parameters:
      clusterResource - the current cluster resource
      application - application to which the container was assigned
      rmContainer - the container to detach

    • getNumApplications

      public int getNumApplications()
      Description copied from interface: CSQueue
      Get the number of applications in the queue.
      Returns:
      number of applications

    • apply

      public void apply(org.apache.hadoop.yarn.api.records.Resource cluster, ResourceCommitRequest<FiCaSchedulerApp,FiCaSchedulerNode> request)

    • stopQueue

      public void stopQueue()
      Description copied from interface: SchedulerQueue
      Stop the queue.

    • getQueueOrderingPolicy

      public QueueOrderingPolicy getQueueOrderingPolicy()

    • isEligibleForAutoDeletion

      public boolean isEligibleForAutoDeletion()
      Description copied from class: AbstractCSQueue
      Checks whether this queue is a dynamic queue and could be deleted.
      Overrides:
      isEligibleForAutoDeletion in class AbstractCSQueue
      Returns:
      true if the dynamic queue could be deleted, false otherwise

    • getAutoCreatedQueueTemplate

      public AutoCreatedQueueTemplate getAutoCreatedQueueTemplate()