Package org.apache.hadoop.yarn.server.resourcemanager.recovery

Class FileSystemRMStateStore

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore
org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service
@Private @Unstable public class FileSystemRMStateStore extends RMStateStore

  • Field Details

    • LOG

      public static final org.slf4j.Logger LOG

    • ROOT_DIR_NAME

      protected static final String ROOT_DIR_NAME
      See Also:

    • CURRENT_VERSION_INFO

      protected static final org.apache.hadoop.yarn.server.records.Version CURRENT_VERSION_INFO

    • AMRMTOKEN_SECRET_MANAGER_NODE

      protected static final String AMRMTOKEN_SECRET_MANAGER_NODE
      See Also:

    • fs

      protected org.apache.hadoop.fs.FileSystem fs

    • fsConf

      @VisibleForTesting protected org.apache.hadoop.conf.Configuration fsConf

  • Constructor Details

    • FileSystemRMStateStore

      public FileSystemRMStateStore()

  • Method Details

    • initInternal

      public void initInternal(org.apache.hadoop.conf.Configuration conf) throws Exception
      Description copied from class: RMStateStore
      Derived classes initialize themselves using this method.
      Specified by:
      initInternal in class RMStateStore
      Parameters:
      conf - Configuration.
      Throws:
      Exception - error occur.

    • startInternal

      protected void startInternal() throws Exception
      Description copied from class: RMStateStore
      Derived classes start themselves using this method. The base class is started and the event dispatcher is ready to use at this point.
      Specified by:
      startInternal in class RMStateStore
      Throws:
      Exception - error occur.

    • closeInternal

      protected void closeInternal() throws Exception
      Description copied from class: RMStateStore
      Derived classes close themselves using this method. The base class will be closed and the event dispatcher will be shutdown after this.
      Specified by:
      closeInternal in class RMStateStore
      Throws:
      Exception - error occur.

    • getCurrentVersion

      protected org.apache.hadoop.yarn.server.records.Version getCurrentVersion()
      Description copied from class: RMStateStore
      Get the current version of the underlying state store.
      Specified by:
      getCurrentVersion in class RMStateStore
      Returns:
      current version.

    • loadVersion

      protected org.apache.hadoop.yarn.server.records.Version loadVersion() throws Exception
      Description copied from class: RMStateStore
      Derived class use this method to load the version information from state store.
      Specified by:
      loadVersion in class RMStateStore
      Returns:
      current version.
      Throws:
      Exception - error occur.

    • storeVersion

      protected void storeVersion() throws Exception
      Description copied from class: RMStateStore
      Derived class use this method to store the version information.
      Specified by:
      storeVersion in class RMStateStore
      Throws:
      Exception - error occur.

    • getAndIncrementEpoch

      public long getAndIncrementEpoch() throws Exception
      Description copied from class: RMStateStore
      Get the current epoch of RM and increment the value.
      Specified by:
      getAndIncrementEpoch in class RMStateStore
      Returns:
      current epoch.
      Throws:
      Exception - error occur.

    • loadState

      public RMStateStore.RMState loadState() throws Exception
      Description copied from class: RMStateStore
      Blocking API The derived class must recover state from the store and return a new RMState object populated with that state This must not be called on the dispatcher thread.
      Specified by:
      loadState in class RMStateStore
      Returns:
      RMState.
      Throws:
      Exception - error occur.

    • storeApplicationStateInternal

      public void storeApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId appId, ApplicationStateData appStateDataPB) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to store the state of an application.
      Specified by:
      storeApplicationStateInternal in class RMStateStore
      Parameters:
      appId - application Id.
      appStateDataPB - application StateData.
      Throws:
      Exception - error occur.

    • updateApplicationStateInternal

      public void updateApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId appId, ApplicationStateData appStateDataPB) throws Exception
      Specified by:
      updateApplicationStateInternal in class RMStateStore
      Throws:
      Exception

    • storeApplicationAttemptStateInternal

      public void storeApplicationAttemptStateInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId, ApplicationAttemptStateData attemptStateDataPB) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to store the state of an application attempt.
      Specified by:
      storeApplicationAttemptStateInternal in class RMStateStore
      Parameters:
      appAttemptId - Application AttemptId.
      attemptStateDataPB - Application AttemptStateData.
      Throws:
      Exception - error occur.

    • updateApplicationAttemptStateInternal

      public void updateApplicationAttemptStateInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId, ApplicationAttemptStateData attemptStateDataPB) throws Exception
      Specified by:
      updateApplicationAttemptStateInternal in class RMStateStore
      Throws:
      Exception

    • removeApplicationAttemptInternal

      public void removeApplicationAttemptInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to remove the state of specified attempt.
      Specified by:
      removeApplicationAttemptInternal in class RMStateStore
      Parameters:
      appAttemptId - application attempt id.
      Throws:
      Exception - exception occurs.

    • removeApplicationStateInternal

      public void removeApplicationStateInternal(ApplicationStateData appState) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to remove the state of an application and its attempts.
      Specified by:
      removeApplicationStateInternal in class RMStateStore
      Parameters:
      appState - ApplicationStateData.
      Throws:
      Exception - error occurs.

    • storeRMDelegationTokenState

      public void storeRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier identifier, Long renewDate) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to store the state of RMDelegationToken and sequence number.
      Specified by:
      storeRMDelegationTokenState in class RMStateStore
      Parameters:
      identifier - RMDelegationTokenIdentifier.
      renewDate - token renew date.
      Throws:
      Exception - error occur.

    • removeRMDelegationTokenState

      public void removeRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier identifier) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to remove the state of RMDelegationToken.
      Specified by:
      removeRMDelegationTokenState in class RMStateStore
      Parameters:
      identifier - RMDelegationTokenIdentifier.
      Throws:
      Exception - error occurs.

    • updateRMDelegationTokenState

      protected void updateRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier rmDTIdentifier, Long renewDate) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to update the state of RMDelegationToken and sequence number.
      Specified by:
      updateRMDelegationTokenState in class RMStateStore
      Parameters:
      rmDTIdentifier - RMDelegationTokenIdentifier.
      renewDate - token renew date.
      Throws:
      Exception - error occurs.

    • storeRMDTMasterKeyState

      public void storeRMDTMasterKeyState(org.apache.hadoop.security.token.delegation.DelegationKey masterKey) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to store the state of DelegationToken Master Key.
      Specified by:
      storeRMDTMasterKeyState in class RMStateStore
      Parameters:
      masterKey - DelegationToken Master Key.
      Throws:
      Exception - error occur.

    • removeRMDTMasterKeyState

      public void removeRMDTMasterKeyState(org.apache.hadoop.security.token.delegation.DelegationKey masterKey) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to remove the state of DelegationToken Master Key.
      Specified by:
      removeRMDTMasterKeyState in class RMStateStore
      Parameters:
      masterKey - DelegationKey.
      Throws:
      Exception - exception occurs.

    • deleteStore

      public void deleteStore() throws Exception
      Description copied from class: RMStateStore
      Derived classes must implement this method to delete the state store.
      Specified by:
      deleteStore in class RMStateStore
      Throws:
      Exception - exception occurs.

    • removeApplication

      public void removeApplication(org.apache.hadoop.yarn.api.records.ApplicationId removeAppId) throws Exception
      Description copied from class: RMStateStore
      Derived classes must implement this method to remove application from the state store.
      Specified by:
      removeApplication in class RMStateStore
      Parameters:
      removeAppId - application Id.
      Throws:
      Exception - exception occurs.

    • storeProxyCACertState

      protected void storeProxyCACertState(X509Certificate caCert, PrivateKey caPrivateKey) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to store the CA Certificate and Private Key.
      Specified by:
      storeProxyCACertState in class RMStateStore
      Parameters:
      caCert - X509Certificate.
      caPrivateKey - PrivateKey.
      Throws:
      Exception - error occurs.

    • getAppDir

      @VisibleForTesting protected org.apache.hadoop.fs.Path getAppDir(org.apache.hadoop.yarn.api.records.ApplicationId appId)

    • getAppAttemptDir

      @VisibleForTesting protected org.apache.hadoop.fs.Path getAppAttemptDir(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttId)

    • writeFile

      protected void writeFile(org.apache.hadoop.fs.Path outputPath, byte[] data, boolean makeUnreadableByAdmin) throws Exception
      Throws:
      Exception

    • updateFile

      protected void updateFile(org.apache.hadoop.fs.Path outputPath, byte[] data, boolean makeUnreadableByAdmin) throws Exception
      Throws:
      Exception

    • replaceFile

      protected void replaceFile(org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path dstPath) throws Exception
      Throws:
      Exception

    • storeOrUpdateAMRMTokenSecretManagerState

      public void storeOrUpdateAMRMTokenSecretManagerState(AMRMTokenSecretManagerState amrmTokenSecretManagerState, boolean isUpdate) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to store or update the state of AMRMToken Master Key.
      Specified by:
      storeOrUpdateAMRMTokenSecretManagerState in class RMStateStore
      Parameters:
      amrmTokenSecretManagerState - amrmTokenSecretManagerState.
      isUpdate - true, update; otherwise not update.
      Throws:
      Exception - exception occurs.

    • storeReservationState

      protected void storeReservationState(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto reservationAllocation, String planName, String reservationIdName) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to store the state of a reservation allocation.
      Specified by:
      storeReservationState in class RMStateStore
      Parameters:
      reservationAllocation - reservation Allocation.
      planName - plan Name.
      reservationIdName - reservationId Name.
      Throws:
      Exception - error occurs.

    • removeReservationState

      protected void removeReservationState(String planName, String reservationIdName) throws Exception
      Description copied from class: RMStateStore
      Blocking API Derived classes must implement this method to remove the state of a reservation allocation.
      Specified by:
      removeReservationState in class RMStateStore
      Parameters:
      planName - plan Name.
      reservationIdName - reservationId Name.
      Throws:
      Exception - exception occurs.

    • getNumRetries

      @VisibleForTesting public int getNumRetries()

    • getRetryInterval

      @VisibleForTesting public long getRetryInterval()