Package org.apache.hadoop.mapreduce.v2.hs

Class CachedHistoryStorage

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage
All Implemented Interfaces:
Closeable, AutoCloseable, HistoryStorage, org.apache.hadoop.service.Service
public class CachedHistoryStorage extends org.apache.hadoop.service.AbstractService implements HistoryStorage
Manages an in memory cache of parsed Job History files.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Constructor Summary

    Constructors
    Constructor
    Description
    CachedHistoryStorage()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    Map<org.apache.hadoop.mapreduce.v2.api.records.JobId,org.apache.hadoop.mapreduce.v2.app.job.Job>
    getAllPartialJobs()
    Get all of the cached jobs.
    org.apache.hadoop.mapreduce.v2.app.job.Job
    getFullJob(org.apache.hadoop.mapreduce.v2.api.records.JobId jobId)
    Get a fully parsed job.
    int
    getLoadedTasksCacheSize()
     
    JobsInfo
    getPartialJobs(Long offset, Long count, String user, String queue, Long sBegin, Long sEnd, Long fBegin, Long fEnd, org.apache.hadoop.mapreduce.v2.api.records.JobState jobState)
    Look for a set of partial jobs.
    static JobsInfo
    getPartialJobs(Collection<org.apache.hadoop.mapreduce.v2.app.job.Job> jobs, Long offset, Long count, String user, String queue, Long sBegin, Long sEnd, Long fBegin, Long fEnd, org.apache.hadoop.mapreduce.v2.api.records.JobState jobState)
     
    boolean
    getUseLoadedTasksCache()
     
    void
    refreshLoadedJobCache()
     
    void
    serviceInit(org.apache.hadoop.conf.Configuration conf)
     
    void
    setHistoryFileManager(HistoryFileManager hsManager)
    Give the Storage a reference to a class that can be used to interact with history files.

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceStart, serviceStop, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • CachedHistoryStorage

      public CachedHistoryStorage()

  • Method Details

    • setHistoryFileManager

      public void setHistoryFileManager(HistoryFileManager hsManager)
      Description copied from interface: HistoryStorage
      Give the Storage a reference to a class that can be used to interact with history files.
      Specified by:
      setHistoryFileManager in interface HistoryStorage
      Parameters:
      hsManager - the class that is used to interact with history files.

    • serviceInit

      public void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
      Overrides:
      serviceInit in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • refreshLoadedJobCache

      public void refreshLoadedJobCache()

    • getFullJob

      public org.apache.hadoop.mapreduce.v2.app.job.Job getFullJob(org.apache.hadoop.mapreduce.v2.api.records.JobId jobId)
      Description copied from interface: HistoryStorage
      Get a fully parsed job.
      Specified by:
      getFullJob in interface HistoryStorage
      Parameters:
      jobId - the id of the job
      Returns:
      the job, or null if it is not found.

    • getAllPartialJobs

      public Map<org.apache.hadoop.mapreduce.v2.api.records.JobId,org.apache.hadoop.mapreduce.v2.app.job.Job> getAllPartialJobs()
      Description copied from interface: HistoryStorage
      Get all of the cached jobs. This only returns partial jobs and is here for legacy reasons.
      Specified by:
      getAllPartialJobs in interface HistoryStorage
      Returns:
      all of the cached jobs

    • getPartialJobs

      public JobsInfo getPartialJobs(Long offset, Long count, String user, String queue, Long sBegin, Long sEnd, Long fBegin, Long fEnd, org.apache.hadoop.mapreduce.v2.api.records.JobState jobState)
      Description copied from interface: HistoryStorage
      Look for a set of partial jobs.
      Specified by:
      getPartialJobs in interface HistoryStorage
      Parameters:
      offset - the offset into the list of jobs.
      count - the maximum number of jobs to return.
      user - only return jobs for the given user.
      queue - only return jobs for in the given queue.
      sBegin - only return Jobs that started on or after the given time.
      sEnd - only return Jobs that started on or before the given time.
      fBegin - only return Jobs that ended on or after the given time.
      fEnd - only return Jobs that ended on or before the given time.
      jobState - only return Jobs that are in the given job state.
      Returns:
      The list of filtered jobs.

    • getPartialJobs

      public static JobsInfo getPartialJobs(Collection<org.apache.hadoop.mapreduce.v2.app.job.Job> jobs, Long offset, Long count, String user, String queue, Long sBegin, Long sEnd, Long fBegin, Long fEnd, org.apache.hadoop.mapreduce.v2.api.records.JobState jobState)

    • getUseLoadedTasksCache

      @VisibleForTesting public boolean getUseLoadedTasksCache()

    • getLoadedTasksCacheSize

      @VisibleForTesting public int getLoadedTasksCacheSize()