Package org.apache.hadoop.mapred

Class LocalContainerLauncher

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.mapred.LocalContainerLauncher
All Implemented Interfaces:
Closeable, AutoCloseable, ContainerLauncher, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.event.EventHandler<ContainerLauncherEvent>
public class LocalContainerLauncher extends org.apache.hadoop.service.AbstractService implements ContainerLauncher
Runs the container task locally in a thread. Since all (sub)tasks share the same local directory, they must be executed sequentially in order to avoid creating/deleting the same files/dirs.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher

    ContainerLauncher.EventType

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Constructor Summary

    Constructors
    Constructor
    Description
    LocalContainerLauncher(AppContext context, org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical)
     
    LocalContainerLauncher(AppContext context, org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical, ClassLoader jobClassLoader)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    handle(ContainerLauncherEvent event)
     
    protected static org.apache.hadoop.mapred.MapOutputFile
    renameMapOutputForReduce(org.apache.hadoop.mapred.JobConf conf, org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId mapId, org.apache.hadoop.mapred.MapOutputFile subMapOutputFile)
    Within the _local_ filesystem (not HDFS), all activity takes place within a subdir inside one of the LOCAL_DIRS (${local.dir}/usercache/$user/appcache/$appId/$contId/), and all sub-MapTasks create the same filename ("file.out").
    void
    serviceStart()
     
    void
    serviceStop()
     
    void
    setEncryptedSpillKey(byte[] encryptedSpillKey)
     

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • LocalContainerLauncher

      public LocalContainerLauncher(AppContext context, org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical)

    • LocalContainerLauncher

      public LocalContainerLauncher(AppContext context, org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical, ClassLoader jobClassLoader)

  • Method Details

    • serviceStart

      public void serviceStart() throws Exception
      Overrides:
      serviceStart in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • serviceStop

      public void serviceStop() throws Exception
      Overrides:
      serviceStop in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • handle

      public void handle(ContainerLauncherEvent event)
      Specified by:
      handle in interface org.apache.hadoop.yarn.event.EventHandler<ContainerLauncherEvent>

    • setEncryptedSpillKey

      public void setEncryptedSpillKey(byte[] encryptedSpillKey)

    • renameMapOutputForReduce

      @VisibleForTesting protected static org.apache.hadoop.mapred.MapOutputFile renameMapOutputForReduce(org.apache.hadoop.mapred.JobConf conf, org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId mapId, org.apache.hadoop.mapred.MapOutputFile subMapOutputFile) throws IOException
      Within the _local_ filesystem (not HDFS), all activity takes place within a subdir inside one of the LOCAL_DIRS (${local.dir}/usercache/$user/appcache/$appId/$contId/), and all sub-MapTasks create the same filename ("file.out"). Rename that to something unique (e.g., "map_0.out") to avoid possible collisions. Longer-term, we'll modify [something] to use TaskAttemptID-based filenames instead of "file.out". (All of this is entirely internal, so there are no particular compatibility issues.)
      Throws:
      IOException