Package org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity

Class PreemptableResourceCalculator

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.AbstractPreemptableResourceCalculator
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator
public class PreemptableResourceCalculator extends AbstractPreemptableResourceCalculator
Calculate how much resources need to be preempted for each queue, will be used by PreemptionCandidatesSelector

  • Constructor Details

    • PreemptableResourceCalculator

      public PreemptableResourceCalculator(CapacitySchedulerPreemptionContext preemptionContext, boolean isReservedPreemptionCandidatesSelector, boolean allowQueuesBalanceAfterAllQueuesSatisfied)
      PreemptableResourceCalculator constructor.
      Parameters:
      preemptionContext - context.
      isReservedPreemptionCandidatesSelector - this will be set by different implementation of candidate selectors, please refer to TempQueuePerPartition#offer for details.
      allowQueuesBalanceAfterAllQueuesSatisfied - Should resources be preempted from an over-served queue when the requesting queues are all at or over their guarantees? An example is, there're 10 queues under root, guaranteed resource of them are all 10%. Assume there're two queues are using resources, queueA uses 10% queueB uses 90%. For all queues are guaranteed, but it's not fair for queueA. We wanna make this behavior can be configured. By default it is not allowed.

  • Method Details

    • computeIdealResourceDistribution

      protected void computeIdealResourceDistribution(org.apache.hadoop.yarn.util.resource.ResourceCalculator rc, List<TempQueuePerPartition> queues, org.apache.hadoop.yarn.api.records.Resource totalPreemptionAllowed, org.apache.hadoop.yarn.api.records.Resource tot_guarant)
      This method computes (for a single level in the tree, passed as a List<TempQueue>) the ideal assignment of resources. This is done recursively to allocate capacity fairly across all queues with pending demands. It terminates when no resources are left to assign, or when all demand is satisfied.
      Parameters:
      rc - resource calculator
      queues - a list of cloned queues to be assigned capacity to (this is an out param)
      totalPreemptionAllowed - total amount of preemption we allow
      tot_guarant - the amount of capacity assigned to this pool of queues

    • recursivelyComputeIdealAssignment

      protected void recursivelyComputeIdealAssignment(TempQueuePerPartition root, org.apache.hadoop.yarn.api.records.Resource totalPreemptionAllowed)
      This method recursively computes the ideal assignment of resources to each level of the hierarchy. This ensures that leafs that are over-capacity but with parents within capacity will not be preemptionCandidates. Preemptions are allowed within each subtree according to local over/under capacity.
      Parameters:
      root - the root of the cloned queue hierachy
      totalPreemptionAllowed - maximum amount of preemption allowed

    • computeIdealAllocation

      public void computeIdealAllocation(org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource totalPreemptionAllowed)