Package org.apache.hadoop.yarn.server.nodemanager.amrmproxy

Class AMRMProxyService

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.service.CompositeService
org.apache.hadoop.yarn.server.nodemanager.amrmproxy.AMRMProxyService
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.api.ApplicationMasterProtocol
public class AMRMProxyService extends org.apache.hadoop.service.CompositeService implements org.apache.hadoop.yarn.api.ApplicationMasterProtocol
AMRMProxyService is a service that runs on each node manager that can be used to intercept and inspect messages from application master to the cluster resource manager. It listens to messages from the application master and creates a request intercepting pipeline instance for each application. The pipeline is a chain of interceptor instances that can inspect and modify the request/response as needed.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    AMRMProxyService.RequestInterceptorChainWrapper
    Private structure for encapsulating RequestInterceptor and ApplicationAttemptId instances.

    Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

    org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Field Summary

    Fields inherited from class org.apache.hadoop.service.CompositeService

    STOP_ONLY_STARTED_SERVICES

  • Constructor Summary

    Constructors
    Constructor
    Description
    AMRMProxyService(Context nmContext, org.apache.hadoop.yarn.event.AsyncDispatcher dispatcher)
    Creates an instance of the service.

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse
    allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request)
    This is called by the AMs started on this node to send heart beat to RM.
    protected RequestInterceptor
    createRequestInterceptorChain()
    This method creates and returns reference of the first interceptor in the chain of request interceptor instances.
    org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse
    finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request)
    This is called by the AMs started on this node to unregister from the RM.
    InetSocketAddress
    getBindAddress()
     
    protected Map<org.apache.hadoop.yarn.api.records.ApplicationId,AMRMProxyService.RequestInterceptorChainWrapper>
    getPipelines()
    Gets the Request interceptor chains for all the applications.
    AMRMProxyTokenSecretManager
    getSecretManager()
     
    protected void
    initializePipeline(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, String user, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> localToken, Map<String,byte[]> recoveredDataMap, boolean isRecovery, org.apache.hadoop.security.Credentials credentials)
    Initializes the request interceptor pipeline for the specified application.
    void
    processApplicationStartRequest(org.apache.hadoop.yarn.api.protocolrecords.StartContainerRequest request)
    Callback from the ContainerManager implementation for initializing the application request processing pipeline.
    void
    recover()
    Recover from NM state store.
    org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse
    registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request)
    This is called by the AMs started on this node to register with the RM.
    protected void
    serviceInit(org.apache.hadoop.conf.Configuration conf)
     
    protected void
    serviceStart()
     
    protected void
    serviceStop()
     
    protected void
    stopApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)
    Shuts down the request processing pipeline for the specified application attempt id.

    Methods inherited from class org.apache.hadoop.service.CompositeService

    addIfService, addService, getServices, removeService

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • AMRMProxyService

      public AMRMProxyService(Context nmContext, org.apache.hadoop.yarn.event.AsyncDispatcher dispatcher)
      Creates an instance of the service.
      Parameters:
      nmContext - NM context
      dispatcher - NM dispatcher

  • Method Details

    • serviceInit

      protected void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
      Overrides:
      serviceInit in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • serviceStart

      protected void serviceStart() throws Exception
      Overrides:
      serviceStart in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • serviceStop

      protected void serviceStop() throws Exception
      Overrides:
      serviceStop in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • recover

      public void recover() throws IOException
      Recover from NM state store. Called after serviceInit before serviceStart.
      Throws:
      IOException - if recover fails

    • registerApplicationMaster

      public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      This is called by the AMs started on this node to register with the RM. This method does the initial authorization and then forwards the request to the application instance specific interceptor chain.
      Specified by:
      registerApplicationMaster in interface org.apache.hadoop.yarn.api.ApplicationMasterProtocol
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • finishApplicationMaster

      public org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      This is called by the AMs started on this node to unregister from the RM. This method does the initial authorization and then forwards the request to the application instance specific interceptor chain.
      Specified by:
      finishApplicationMaster in interface org.apache.hadoop.yarn.api.ApplicationMasterProtocol
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • allocate

      public org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      This is called by the AMs started on this node to send heart beat to RM. This method does the initial authorization and then forwards the request to the application instance specific pipeline, which is a chain of request interceptor objects. One application request processing pipeline is created per AM instance.
      Specified by:
      allocate in interface org.apache.hadoop.yarn.api.ApplicationMasterProtocol
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • processApplicationStartRequest

      public void processApplicationStartRequest(org.apache.hadoop.yarn.api.protocolrecords.StartContainerRequest request) throws IOException, org.apache.hadoop.yarn.exceptions.YarnException
      Callback from the ContainerManager implementation for initializing the application request processing pipeline.
      Parameters:
      request - - encapsulates information for starting an AM
      Throws:
      IOException - if fails
      org.apache.hadoop.yarn.exceptions.YarnException - if fails

    • initializePipeline

      protected void initializePipeline(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, String user, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> localToken, Map<String,byte[]> recoveredDataMap, boolean isRecovery, org.apache.hadoop.security.Credentials credentials)
      Initializes the request interceptor pipeline for the specified application.
      Parameters:
      applicationAttemptId - attempt id
      user - user name
      amrmToken - amrmToken issued by RM
      localToken - amrmToken issued by AMRMProxy
      recoveredDataMap - the recovered states for AMRMProxy from NMSS
      isRecovery - whether this is to recover a previously existing pipeline

    • stopApplication

      protected void stopApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)
      Shuts down the request processing pipeline for the specified application attempt id.
      Parameters:
      applicationId - application id

    • getPipelines

      protected Map<org.apache.hadoop.yarn.api.records.ApplicationId,AMRMProxyService.RequestInterceptorChainWrapper> getPipelines()
      Gets the Request interceptor chains for all the applications.
      Returns:
      the request interceptor chains.

    • createRequestInterceptorChain

      protected RequestInterceptor createRequestInterceptorChain()
      This method creates and returns reference of the first interceptor in the chain of request interceptor instances.
      Returns:
      the reference of the first interceptor in the chain

    • getBindAddress

      @Private public InetSocketAddress getBindAddress()

    • getSecretManager

      @Private public AMRMProxyTokenSecretManager getSecretManager()