Package org.apache.hadoop.hdfs.server.federation.router

Class RouterRpcClient

java.lang.Object
org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient
Direct Known Subclasses:
RouterAsyncRpcClient
public class RouterRpcClient extends Object
A client proxy for Router to NN communication using the NN ClientProtocol.

Provides routers to invoke remote ClientProtocol methods and handle retries/failover.

  • invokeSingle Make a single request to a single namespace
  • invokeSequential Make a sequential series of requests to multiple ordered namespaces until a condition is met.
  • invokeConcurrent Make concurrent requests to multiple namespaces and return all of the results.
Also maintains a cached pool of connections to NNs. Connections are managed by the ConnectionManager and are unique to each user + NN. The size of the connection pool can be configured. Larger pools allow for more simultaneous requests to a single NN from a single user.

  • Constructor Details

    • RouterRpcClient

      public RouterRpcClient(org.apache.hadoop.conf.Configuration conf, Router router, ActiveNamenodeResolver resolver, RouterRpcMonitor monitor, RouterStateIdContext routerStateIdContext)
      Create a router RPC client to manage remote procedure calls to NNs.
      Parameters:
      conf - Hdfs Configuration.
      router - A router using this RPC client.
      resolver - A NN resolver to determine the currently active NN in HA.
      monitor - Optional performance monitor.
      routerStateIdContext - the router state context object to hold the state ids for all namespaces.

  • Method Details

    • initConcurrentCallExecutorService

      protected void initConcurrentCallExecutorService(org.apache.hadoop.conf.Configuration conf)

    • getNamenodeResolver

      public ActiveNamenodeResolver getNamenodeResolver()
      Get the active namenode resolver used by this client.
      Returns:
      Active namenode resolver.

    • getExecutorService

      @VisibleForTesting public ThreadPoolExecutor getExecutorService()
      Get the executor service used by invoking concurrent calls.
      Returns:
      the executor service.

    • shutdown

      public void shutdown()
      Shutdown the client.

    • getNumConnections

      public int getNumConnections()
      Total number of available sockets between the router and NNs.
      Returns:
      Number of namenode clients.

    • getNumActiveConnections

      public int getNumActiveConnections()
      Total number of available sockets between the router and NNs.
      Returns:
      Number of namenode clients.

    • getNumIdleConnections

      public int getNumIdleConnections()
      Total number of idle sockets between the router and NNs.
      Returns:
      Number of namenode clients.

    • getNumActiveConnectionsRecently

      public int getNumActiveConnectionsRecently()
      Total number of active sockets between the router and NNs.
      Returns:
      Number of recently active namenode clients.

    • getNumConnectionPools

      public int getNumConnectionPools()
      Total number of open connection pools to a NN. Each connection pool. represents one user + one NN.
      Returns:
      Number of connection pools.

    • getNumCreatingConnections

      public int getNumCreatingConnections()
      Number of connections between the router and NNs being created sockets.
      Returns:
      Number of connections waiting to be created.

    • getJSON

      public String getJSON()
      JSON representation of the connection pool.
      Returns:
      String representation of the JSON.

    • getAsyncCallerPoolJson

      public String getAsyncCallerPoolJson()
      JSON representation of the async caller thread pool.
      Returns:
      String representation of the JSON.

    • getRejectedPermitsPerNsJSON

      public String getRejectedPermitsPerNsJSON()
      JSON representation of the rejected permits for each nameservice.
      Returns:
      String representation of the rejected permits for each nameservice.

    • getAcceptedPermitsPerNsJSON

      public String getAcceptedPermitsPerNsJSON()
      JSON representation of the accepted permits for each nameservice.
      Returns:
      String representation of the accepted permits for each nameservice.

    • getConnection

      protected ConnectionContext getConnection(org.apache.hadoop.security.UserGroupInformation ugi, String nsId, String rpcAddress, Class<?> proto) throws IOException
      Get ClientProtocol proxy client for a NameNode. Each combination of user + NN must use a unique proxy client. Previously created clients are cached and stored in a connection pool by the ConnectionManager.
      Parameters:
      ugi - User group information.
      nsId - Nameservice identifier.
      rpcAddress - RPC server address of the NN.
      proto - Protocol of the connection.
      Returns:
      ConnectionContext containing a ClientProtocol proxy client for the NN + current user.
      Throws:
      IOException - If we cannot get a connection to the NameNode.

    • shouldRetry

      protected org.apache.hadoop.io.retry.RetryPolicy.RetryAction.RetryDecision shouldRetry(IOException ioe, int retryCount, String nsId, FederationNamenodeContext namenode, boolean listObserverFirst) throws IOException
      If we should retry the RPC call.
      Parameters:
      ioe - IOException reported.
      retryCount - Number of retries.
      nsId - Nameservice ID.
      namenode - namenode context.
      listObserverFirst - Observer read case, observer NN will be ranked first.
      Returns:
      Retry decision.
      Throws:
      IOException - An IO Error occurred.

    • invokeMethod

      @VisibleForTesting public Object invokeMethod(org.apache.hadoop.security.UserGroupInformation ugi, List<? extends FederationNamenodeContext> namenodes, boolean useObserver, Class<?> protocol, Method method, Object... params) throws ConnectException, org.apache.hadoop.ipc.StandbyException, IOException
      Invokes a method against the ClientProtocol proxy server. If a standby exception is generated by the call to the client, retries using the alternate server.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Parameters:
      ugi - User group information.
      namenodes - A prioritized list of namenodes within the same nameservice.
      useObserver - Whether to use observer namenodes.
      protocol - the protocol of the connection.
      method - Remote ClientProtocol method to invoke.
      params - Variable list of parameters matching the method.
      Returns:
      The result of invoking the method.
      Throws:
      ConnectException - If it cannot connect to any Namenode.
      org.apache.hadoop.ipc.StandbyException - If all Namenodes are in Standby.
      IOException - If it cannot invoke the method.

    • handlerAllNamenodeFail

      protected Object handlerAllNamenodeFail(List<? extends FederationNamenodeContext> namenodes, Method method, Map<FederationNamenodeContext,IOException> ioes, Object[] params) throws IOException
      All namenodes cannot successfully process the RPC request, throw corresponding exceptions according to the exception type of each namenode.
      Parameters:
      namenodes - A prioritized list of namenodes within the same nameservice.
      method - Remote ClientProtocol method to invoke.
      ioes - The exception type of each namenode.
      params - Variable list of parameters matching the method.
      Returns:
      null
      Throws:
      IOException - Corresponding IOException according to the exception type of each namenode.

    • postProcessResult

      protected void postProcessResult(Method method, RouterRpcClient.ExecutionStatus status, FederationNamenodeContext namenode, String nsId, org.apache.hadoop.hdfs.NameNodeProxiesClient.ProxyAndInfo<?> client) throws IOException
      The RPC request is successfully processed by the NameNode, the NameNode status in the router cache is updated according to the ExecutionStatus.
      Parameters:
      method - Remote method to invoke.
      status - Current execution status.
      namenode - The namenode that successfully processed this RPC request.
      nsId - Nameservice ID.
      client - Connection client.
      Throws:
      IOException - If the state store cannot be accessed.

    • handleInvokeMethodIOException

      protected void handleInvokeMethodIOException(FederationNamenodeContext namenode, IOException ioe, RouterRpcClient.ExecutionStatus status, boolean useObserver) throws IOException
      The RPC request to the NameNode throws an exception, handle it according to the type of exception.
      Parameters:
      namenode - The namenode that processed this RPC request.
      ioe - The exception thrown by this RPC request.
      status - The current execution status.
      useObserver - Whether to use observer namenodes.
      Throws:
      IOException - If it cannot invoke the method.

    • addClientInfoToCallerContext

      protected void addClientInfoToCallerContext(org.apache.hadoop.security.UserGroupInformation ugi)
      For tracking some information about the actual client. It adds trace info "clientIp:ip", "clientPort:port", "clientId:id", "clientCallId:callId" and "realUser:userName" in the caller context, removing the old values if they were already present.
      Parameters:
      ugi - User group information.

    • invoke

      protected Object invoke(FederationNamenodeContext namenode, Boolean listObserverFirst, int retryCount, Method method, Object obj, Object... params) throws IOException
      Invokes a method on the designated object. Catches exceptions specific to the invocation.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Parameters:
      namenode - namenode context.
      listObserverFirst - Observer read case, observer NN will be ranked first.
      retryCount - Current retry times
      method - Method to invoke
      obj - Target object for the method
      params - Variable parameters
      Returns:
      Response from the remote server
      Throws:
      IOException - If error occurs.

    • handlerInvokeException

      protected Object handlerInvokeException(FederationNamenodeContext namenode, Boolean listObserverFirst, int retryCount, Method method, Object obj, Throwable e, Object[] params) throws IOException
      Handle the exception when an RPC request to the NameNode throws an exception.
      Parameters:
      namenode - namenode context.
      listObserverFirst - Observer read case, observer NN will be ranked first.
      retryCount - Current retry times
      method - Method to invoke
      obj - Target object for the method
      e - The exception thrown by the current invocation.
      params - Variable parameters
      Returns:
      Response from the remote server
      Throws:
      IOException - If error occurs.

    • isUnavailableException

      public static boolean isUnavailableException(IOException ioe)
      Check if the exception comes from an unavailable subcluster.
      Parameters:
      ioe - IOException to check.
      Returns:
      If the exception comes from an unavailable subcluster.

    • getCleanException

      protected static IOException getCleanException(IOException ioe)
      Get a clean copy of the exception. Sometimes the exceptions returned by the server contain the full stack trace in the message.
      Parameters:
      ioe - Exception to clean up.
      Returns:
      Copy of the original exception with a clean message.

    • invokeSingle

      public <T> T invokeSingle(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, RemoteMethod method, List<RemoteLocation> locations, Class<T> clazz) throws IOException
      Invokes a ClientProtocol method. Determines the target nameservice via a provided block.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote method return.
      Parameters:
      block - Block used to determine appropriate nameservice.
      method - The remote method and parameters to invoke.
      locations - The remote locations will be used.
      clazz - Class for the return type.
      Returns:
      The result of invoking the method.
      Throws:
      IOException - If the invoke generated an error.

    • invokeSingleBlockPool

      public Object invokeSingleBlockPool(String bpId, RemoteMethod method) throws IOException
      Invokes a ClientProtocol method. Determines the target nameservice using the block pool id.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Parameters:
      bpId - Block pool identifier.
      method - The remote method and parameters to invoke.
      Returns:
      The result of invoking the method.
      Throws:
      IOException - If the invoke generated an error.

    • invokeSingle

      public Object invokeSingle(String nsId, RemoteMethod method) throws IOException
      Invokes a ClientProtocol method against the specified namespace.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Parameters:
      nsId - Target namespace for the method.
      method - The remote method and parameters to invoke.
      Returns:
      The result of invoking the method.
      Throws:
      IOException - If the invoke generated an error.

    • invokeSingle

      public <T> T invokeSingle(String nsId, RemoteMethod method, Class<T> clazz) throws IOException
      Invokes a remote method against the specified namespace.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote method return.
      Parameters:
      nsId - Target namespace for the method.
      method - The remote method and parameters to invoke.
      clazz - Class for the return type.
      Returns:
      The result of invoking the method.
      Throws:
      IOException - If the invoke generated an error.

    • invokeSingle

      public <T> T invokeSingle(org.apache.hadoop.hdfs.protocol.ExtendedBlock extendedBlock, RemoteMethod method, Class<T> clazz) throws IOException
      Invokes a remote method against the specified extendedBlock.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote method return.
      Parameters:
      extendedBlock - Target extendedBlock for the method.
      method - The remote method and parameters to invoke.
      clazz - Class for the return type.
      Returns:
      The result of invoking the method.
      Throws:
      IOException - If the invoke generated an error.

    • invokeSingle

      public <T> T invokeSingle(RemoteLocationContext location, RemoteMethod remoteMethod, Class<T> clazz) throws IOException
      Invokes a single proxy call for a single location.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote method return.
      Parameters:
      location - RemoteLocation to invoke.
      remoteMethod - The remote method and parameters to invoke.
      clazz - Class for the return type.
      Returns:
      The result of invoking the method if successful.
      Throws:
      IOException - If the invoke generated an error.

    • invokeSequential

      public <T> T invokeSequential(List<? extends RemoteLocationContext> locations, RemoteMethod remoteMethod) throws IOException
      Invokes sequential proxy calls to different locations. Continues to invoke calls until a call returns without throwing a remote exception.
      Type Parameters:
      T - The type of the remote method return.
      Parameters:
      locations - List of locations/nameservices to call concurrently.
      remoteMethod - The remote method and parameters to invoke.
      Returns:
      The result of the first successful call, or if no calls are successful, the result of the last RPC call executed.
      Throws:
      IOException - if the success condition is not met and one of the RPC calls generated a remote exception.

    • invokeSequential

      public <T> T invokeSequential(List<? extends RemoteLocationContext> locations, RemoteMethod remoteMethod, Class<T> expectedResultClass, Object expectedResultValue) throws IOException
      Invokes sequential proxy calls to different locations. Continues to invoke calls until the success condition is met, or until all locations have been attempted. The success condition may be specified by:
      • An expected result class
      • An expected result value
      If no expected result class/values are specified, the success condition is a call that does not throw a remote exception.
      Type Parameters:
      T - The type of the remote method return.
      Parameters:
      locations - List of locations/nameservices to call concurrently.
      remoteMethod - The remote method and parameters to invoke.
      expectedResultClass - In order to be considered a positive result, the return type must be of this class.
      expectedResultValue - In order to be considered a positive result, the return value must equal the value of this object.
      Returns:
      The result of the first successful call, or if no calls are successful, the result of the first RPC call executed.
      Throws:
      IOException - if the success condition is not met, return the first remote exception generated.

    • invokeSequential

      public <R extends RemoteLocationContext, T> RemoteResult invokeSequential(RemoteMethod remoteMethod, List<R> locations, Class<T> expectedResultClass, Object expectedResultValue) throws IOException
      Invokes sequential proxy calls to different locations. Continues to invoke calls until the success condition is met, or until all locations have been attempted. The success condition may be specified by:
      • An expected result class
      • An expected result value
      If no expected result class/values are specified, the success condition is a call that does not throw a remote exception. This returns RemoteResult, which contains the invoked location as well as the result.
      Type Parameters:
      R - The type of the remote location.
      T - The type of the remote method return.
      Parameters:
      remoteMethod - The remote method and parameters to invoke.
      locations - List of locations/nameservices to call concurrently.
      expectedResultClass - In order to be considered a positive result, the return type must be of this class.
      expectedResultValue - In order to be considered a positive result, the return value must equal the value of this object.
      Returns:
      The result of the first successful call, or if no calls are successful, the result of the first RPC call executed, along with the invoked location in form of RemoteResult.
      Throws:
      IOException - if the success condition is not met, return the first remote exception generated.

    • processException

      protected IOException processException(IOException ioe, RemoteLocationContext loc)
      Exception messages might contain local subcluster paths. This method generates a new exception with the proper message.
      Parameters:
      ioe - Original IOException.
      loc - Location we are processing.
      Returns:
      Exception processed for federation.

    • isExpectedClass

      protected static boolean isExpectedClass(Class<?> expectedClass, Object clazz)
      Checks if a result matches the required result class.
      Parameters:
      expectedClass - Required result class, null to skip the check.
      clazz - The result to check.
      Returns:
      True if the result is an instance of the required class or if the expected class is null.

    • isExpectedValue

      protected static boolean isExpectedValue(Object expectedValue, Object value)
      Checks if a result matches the expected value.
      Parameters:
      expectedValue - The expected value, null to skip the check.
      value - The result to check.
      Returns:
      True if the result is equals to the expected value or if the expected value is null.

    • invokeAll

      public <T extends RemoteLocationContext> boolean invokeAll(Collection<T> locations, RemoteMethod method) throws IOException
      Invoke method in all locations and return success if any succeeds.
      Type Parameters:
      T - The type of the remote location.
      Parameters:
      locations - List of remote locations to call concurrently.
      method - The remote method and parameters to invoke.
      Returns:
      If the call succeeds in any location.
      Throws:
      IOException - If any of the calls return an exception.

    • invokeConcurrent

      public <T extends RemoteLocationContext, R> void invokeConcurrent(Collection<T> locations, RemoteMethod method) throws IOException
      Invoke multiple concurrent proxy calls to different clients. Returns an array of results.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      locations - List of remote locations to call concurrently.
      method - The remote method and parameters to invoke.
      Throws:
      IOException - If all the calls throw an exception.

    • invokeConcurrent

      public <T extends RemoteLocationContext, R> Map<T,R> invokeConcurrent(Collection<T> locations, RemoteMethod method, Class<R> clazz) throws IOException
      Invoke multiple concurrent proxy calls to different clients. Returns an array of results.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      locations - List of remote locations to call concurrently.
      method - The remote method and parameters to invoke.
      clazz - Type of the remote return type.
      Returns:
      Result of invoking the method per subcluster: nsId to result.
      Throws:
      IOException - If all the calls throw an exception.

    • invokeConcurrent

      public <T extends RemoteLocationContext, R> void invokeConcurrent(Collection<T> locations, RemoteMethod method, boolean requireResponse, boolean standby) throws IOException
      Invoke multiple concurrent proxy calls to different clients. Returns an array of results.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      locations - List of remote locations to call concurrently.
      method - The remote method and parameters to invoke.
      requireResponse - If true an exception will be thrown if all calls do not complete. If false exceptions are ignored and all data results successfully received are returned.
      standby - If the requests should go to the standby namenodes too.
      Throws:
      IOException - If all the calls throw an exception.

    • invokeConcurrent

      public <T extends RemoteLocationContext, R> Map<T,R> invokeConcurrent(Collection<T> locations, RemoteMethod method, boolean requireResponse, boolean standby, Class<R> clazz) throws IOException
      Invokes multiple concurrent proxy calls to different clients. Returns an array of results.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      locations - List of remote locations to call concurrently.
      method - The remote method and parameters to invoke.
      requireResponse - If true an exception will be thrown if all calls do not complete. If false exceptions are ignored and all data results successfully received are returned.
      standby - If the requests should go to the standby namenodes too.
      clazz - Type of the remote return type.
      Returns:
      Result of invoking the method per subcluster: nsId to result.
      Throws:
      IOException - If requiredResponse=true and any of the calls throw an exception.

    • invokeConcurrent

      public <T extends RemoteLocationContext, R> Map<T,R> invokeConcurrent(Collection<T> locations, RemoteMethod method, boolean requireResponse, boolean standby, long timeOutMs, Class<R> clazz) throws IOException
      Invokes multiple concurrent proxy calls to different clients. Returns an array of results.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      locations - List of remote locations to call concurrently.
      method - The remote method and parameters to invoke.
      requireResponse - If true an exception will be thrown if all calls do not complete. If false exceptions are ignored and all data results successfully received are returned.
      standby - If the requests should go to the standby namenodes too.
      timeOutMs - Timeout for each individual call.
      clazz - Type of the remote return type.
      Returns:
      Result of invoking the method per subcluster: nsId to result.
      Throws:
      IOException - If requiredResponse=true and any of the calls throw an exception.

    • postProcessResult

      protected static <T extends RemoteLocationContext, R> Map<T,R> postProcessResult(boolean requireResponse, List<RemoteResult<T,R>> results) throws IOException
      Post-process the results returned by invokeConcurrent(Collection, RemoteMethod, boolean, long, Class).
      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      requireResponse - If true an exception will be thrown if all calls do not complete. If false exceptions are ignored and all data results successfully received are returned.
      results - Result of invoking the method per subcluster (list of results), This includes the exception for each remote location.
      Returns:
      Result of invoking the method per subcluster: nsId to result.
      Throws:
      IOException - If requiredResponse=true and any of the calls throw an exception.

    • invokeConcurrent

      public <T extends RemoteLocationContext, R> List<RemoteResult<T,R>> invokeConcurrent(Collection<T> locations, RemoteMethod method, boolean standby, long timeOutMs, Class<R> clazz) throws IOException
      Invokes multiple concurrent proxy calls to different clients. Returns an array of results.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return
      Parameters:
      locations - List of remote locations to call concurrently.
      method - The remote method and parameters to invoke.
      standby - If the requests should go to the standby namenodes too.
      timeOutMs - Timeout for each individual call.
      clazz - Type of the remote return type.
      Returns:
      Result of invoking the method per subcluster (list of results). This includes the exception for each remote location.
      Throws:
      IOException - If there are errors invoking the method.

    • getRemoteResults

      protected <T extends RemoteLocationContext, R> List<RemoteResult<T,R>> getRemoteResults(RemoteMethod method, long timeOutMs, RouterRpcFairnessPolicyController controller, List<T> orderedLocations, List<Callable<Object>> callables) throws IOException
      Invokes multiple concurrent proxy calls to different clients. Returns an array of results.
      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      method - The remote method and parameters to invoke.
      timeOutMs - Timeout for each individual call.
      controller - Fairness manager to control handlers assigned per NS.
      orderedLocations - List of remote locations to call concurrently.
      callables - Invoke method for each NameNode.
      Returns:
      Result of invoking the method per subcluster (list of results), This includes the exception for each remote location.
      Throws:
      IOException - If there are errors invoking the method.

    • processFutures

      protected <T extends RemoteLocationContext, R> List<RemoteResult<T,R>> processFutures(RemoteMethod method, Method m, List<T> orderedLocations, List<Future<Object>> futures) throws InterruptedException
      Handle all futures during the invokeConcurrent call process.
      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      method - The remote method and parameters to invoke.
      m - The method to invoke.
      orderedLocations - List of remote locations to call concurrently.
      futures - all futures during the invokeConcurrent call process.
      Returns:
      Result of invoking the method per subcluster (list of results), This includes the exception for each remote location.
      Throws:
      InterruptedException - if the current thread was interrupted while waiting.

    • invokeSingle

      public <T extends RemoteLocationContext, R> List<RemoteResult<T,R>> invokeSingle(T location, RemoteMethod method) throws IOException
      Invokes a ClientProtocol method against the specified namespace.

      Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

      Type Parameters:
      T - The type of the remote location.
      R - The type of the remote method return.
      Parameters:
      location - RemoteLocation to invoke.
      method - The remote method and parameters to invoke.
      Returns:
      Result of invoking the method per subcluster (list of results), This includes the exception for each remote location.
      Throws:
      IOException - If there are errors invoking the method.

    • transferThreadLocalContext

      protected void transferThreadLocalContext(org.apache.hadoop.ipc.Server.Call originCall, org.apache.hadoop.ipc.CallerContext originContext)
      Transfer origin thread local context which is necessary to current worker thread when invoking method concurrently by executor service.
      Parameters:
      originCall - origin Call required for getting remote client ip.
      originContext - origin CallerContext which should be transferred to server side.

    • acquirePermit

      protected void acquirePermit(String nsId, org.apache.hadoop.security.UserGroupInformation ugi, String methodName, RouterRpcFairnessPolicyController controller) throws IOException
      Acquire permit to continue processing the request for specific nsId.
      Parameters:
      nsId - Identifier of the block pool.
      ugi - UserGroupIdentifier associated with the user.
      methodName - The name of remote method that needs to be invoked.
      controller - fairness policy controller to acquire permit from
      Throws:
      IOException - If permit could not be acquired for the nsId.

    • releasePermit

      protected void releasePermit(String nsId, org.apache.hadoop.security.UserGroupInformation ugi, RemoteMethod m, RouterRpcFairnessPolicyController controller)
      Release permit for specific nsId after processing against downstream nsId is completed.
      Parameters:
      nsId - Identifier of the block pool.
      ugi - UserGroupIdentifier associated with the user.
      m - Remote method that needs to be invoked.
      controller - fairness policy controller to release permit from

    • getRouterRpcFairnessPolicyController

      public RouterRpcFairnessPolicyController getRouterRpcFairnessPolicyController()

    • incrRejectedPermitForNs

      protected void incrRejectedPermitForNs(String ns)

    • getRejectedPermitForNs

      public Long getRejectedPermitForNs(String ns)

    • incrAcceptedPermitForNs

      protected void incrAcceptedPermitForNs(String ns)

    • getAcceptedPermitForNs

      public Long getAcceptedPermitForNs(String ns)

    • refreshFairnessPolicyController

      public String refreshFairnessPolicyController(org.apache.hadoop.conf.Configuration conf)
      Refreshes/changes the fairness policy controller implementation if possible and returns the controller class name.
      Parameters:
      conf - Configuration
      Returns:
      New controller class name if successfully refreshed, else old controller class name

    • getOrderedNamenodes

      public List<? extends FederationNamenodeContext> getOrderedNamenodes(String nsId, boolean isObserverRead) throws IOException
      Get a prioritized list of NNs that share the same nameservice ID (in the same namespace). In observer read case, OBSERVER NNs will be first in the list. Otherwise, ACTIVE NNs will be first in the list.
      Parameters:
      nsId - The nameservice ID for the namespace.
      isObserverRead - Read on observer namenode.
      Returns:
      A prioritized list of NNs to use for communication.
      Throws:
      IOException - If a NN cannot be located for the nameservice ID.

    • isObserverReadEligible

      protected boolean isObserverReadEligible(String nsId, Method method)

    • isNamespaceObserverReadEligible

      public boolean isNamespaceObserverReadEligible(String nsId)
      Check if a namespace is eligible for observer reads.
      Parameters:
      nsId - namespaceID
      Returns:
      whether the 'namespace' has observer reads enabled.

    • shouldRotateCache

      protected boolean shouldRotateCache(IOException ioe)
      Determine whether router rotated cache is required when NoNamenodesAvailableException occurs.
      Parameters:
      ioe - cause of the NoNamenodesAvailableException.
      Returns:
      true if NoNamenodesAvailableException occurs due to unavailable exception, otherwise false.