Package org.apache.hadoop.mapreduce.v2.app

Class JobEndNotifier

java.lang.Object
org.apache.hadoop.mapreduce.v2.app.JobEndNotifier
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
public class JobEndNotifier extends Object implements org.apache.hadoop.conf.Configurable

This class handles job end notification. Submitters of jobs can choose to be notified of the end of a job by supplying a URL to which a connection will be established.

  • The URL connection is fire and forget by default.
  • User can specify number of retry attempts and a time interval at which to attempt retries
  • Cluster administrators can set final parameters to set maximum number of tries (0 would disable job end notification) and max time interval and a proxy if needed
  • The URL may contain sentinels which will be replaced by jobId and jobStatus (eg. SUCCEEDED/KILLED/FAILED)

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected int
    numTries
     
    protected String
    proxyConf
     
    protected Proxy
    proxyToUse
     
    protected int
    timeout
     
    protected URL
    urlToNotify
     
    protected String
    userUrl
     
    protected int
    waitInterval
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    JobEndNotifier()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.hadoop.conf.Configuration
    getConf()
     
    void
    notify(org.apache.hadoop.mapreduce.v2.api.records.JobReport jobReport)
    Notify a server of the completion of a submitted job.
    protected boolean
    notifyURLOnce()
    Notify the URL just once.
    void
    setConf(org.apache.hadoop.conf.Configuration conf)
    Parse the URL that needs to be notified of the end of the job, along with the number of retries in case of failure, the amount of time to wait between retries and proxy settings

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • userUrl

      protected String userUrl

    • proxyConf

      protected String proxyConf

    • numTries

      protected int numTries

    • waitInterval

      protected int waitInterval

    • timeout

      protected int timeout

    • urlToNotify

      protected URL urlToNotify

    • proxyToUse

      protected Proxy proxyToUse

  • Constructor Details

    • JobEndNotifier

      public JobEndNotifier()

  • Method Details

    • setConf

      public void setConf(org.apache.hadoop.conf.Configuration conf)
      Parse the URL that needs to be notified of the end of the job, along with the number of retries in case of failure, the amount of time to wait between retries and proxy settings
      Specified by:
      setConf in interface org.apache.hadoop.conf.Configurable
      Parameters:
      conf - the configuration

    • getConf

      public org.apache.hadoop.conf.Configuration getConf()
      Specified by:
      getConf in interface org.apache.hadoop.conf.Configurable

    • notifyURLOnce

      protected boolean notifyURLOnce()
      Notify the URL just once. Use best effort.

    • notify

      public void notify(org.apache.hadoop.mapreduce.v2.api.records.JobReport jobReport) throws InterruptedException
      Notify a server of the completion of a submitted job. The user must have configured MRJobConfig.MR_JOB_END_NOTIFICATION_URL
      Parameters:
      jobReport - JobReport used to read JobId and JobStatus
      Throws:
      InterruptedException