Package org.apache.hadoop.yarn.server.resourcemanager.recovery

Class LeveldbRMStateStore

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore

org.apache.hadoop.yarn.server.resourcemanager.recovery.LeveldbRMStateStore

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service

public class LeveldbRMStateStore
extends RMStateStore

Changes from 1.0 to 1.1, Addition of ReservationSystem state.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore

RMStateStore.ProxyCAState, RMStateStore.RMDTSecretManagerState, RMStateStore.RMState, RMStateStore.RMStateStoreState

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field	Description
static final org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore

AM_CLIENT_TOKEN_MASTER_KEY_NAME, AM_RM_TOKEN_SERVICE, AMRMTOKEN_SECRET_MANAGER_ROOT, baseEpoch, DELEGATION_KEY_PREFIX, DELEGATION_TOKEN_PREFIX, DELEGATION_TOKEN_SEQUENCE_NUMBER_PREFIX, EPOCH_NODE, PROXY_CA_CERT_NODE, PROXY_CA_PRIVATE_KEY_NODE, PROXY_CA_ROOT, RESERVATION_SYSTEM_ROOT, resourceManager, RM_APP_ROOT, RM_DELEGATION_TOKENS_ROOT_ZNODE_NAME, RM_DT_SECRET_MANAGER_ROOT, rmStateStoreEventHandler, VERSION_NODE

Constructor Summary

Constructors

Constructor	Description
LeveldbRMStateStore()

Method Summary

Modifier and Type	Method	Description
protected void	closeInternal()	Derived classes close themselves using this method.
void	deleteStore()	Derived classes must implement this method to delete the state store.
long	getAndIncrementEpoch()	Get the current epoch of RM and increment the value.
protected org.apache.hadoop.yarn.server.records.Version	getCurrentVersion()	Get the current version of the underlying state store.
protected void	initInternal(org.apache.hadoop.conf.Configuration conf)	Derived classes initialize themselves using this method.
RMStateStore.RMState	loadState()	Blocking API The derived class must recover state from the store and return a new RMState object populated with that state This must not be called on the dispatcher thread.
protected org.apache.hadoop.yarn.server.records.Version	loadVersion()	Derived class use this method to load the version information from state store.
void	removeApplication(org.apache.hadoop.yarn.api.records.ApplicationId removeAppId)	Derived classes must implement this method to remove application from the state store.
void	removeApplicationAttemptInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)	Blocking API Derived classes must implement this method to remove the state of specified attempt.
protected void	removeApplicationStateInternal(ApplicationStateData appState)	Blocking API Derived classes must implement this method to remove the state of an application and its attempts.
protected void	removeReservationState(String planName, String reservationIdName)	Blocking API Derived classes must implement this method to remove the state of a reservation allocation.
protected void	removeRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier tokenId)	Blocking API Derived classes must implement this method to remove the state of RMDelegationToken.
protected void	removeRMDTMasterKeyState(org.apache.hadoop.security.token.delegation.DelegationKey masterKey)	Blocking API Derived classes must implement this method to remove the state of DelegationToken Master Key.
protected void	setDbManager(DBManager dbManager)
protected void	startInternal()	Derived classes start themselves using this method.
protected void	storeApplicationAttemptStateInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId, ApplicationAttemptStateData attemptStateData)	Blocking API Derived classes must implement this method to store the state of an application attempt.
protected void	storeApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId appId, ApplicationStateData appStateData)	Blocking API Derived classes must implement this method to store the state of an application.
void	storeOrUpdateAMRMTokenSecretManagerState(AMRMTokenSecretManagerState state, boolean isUpdate)	Blocking API Derived classes must implement this method to store or update the state of AMRMToken Master Key.
protected void	storeProxyCACertState(X509Certificate caCert, PrivateKey caPrivateKey)	Blocking API Derived classes must implement this method to store the CA Certificate and Private Key.
protected void	storeReservationState(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto reservationAllocation, String planName, String reservationIdName)	Blocking API Derived classes must implement this method to store the state of a reservation allocation.
protected void	storeRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier tokenId, Long renewDate)	Blocking API Derived classes must implement this method to store the state of RMDelegationToken and sequence number.
protected void	storeRMDTMasterKeyState(org.apache.hadoop.security.token.delegation.DelegationKey masterKey)	Blocking API Derived classes must implement this method to store the state of DelegationToken Master Key.
protected void	storeVersion()	Derived class use this method to store the version information.
protected void	storeVersion(org.apache.hadoop.yarn.server.records.Version version)
protected void	updateApplicationAttemptStateInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId, ApplicationAttemptStateData attemptStateData)
protected void	updateApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId appId, ApplicationStateData appStateData)
protected void	updateRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier tokenId, Long renewDate)	Blocking API Derived classes must implement this method to update the state of RMDelegationToken and sequence number.

Methods inherited from class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore

checkVersion, getCredentialsFromAppAttempt, getRMStateStoreEventHandler, getRMStateStoreState, handleStoreEvent, isFencedState, nextEpoch, notifyStoreOperationFailed, removeApplication, removeApplicationAttempt, removeReservation, removeRMDelegationToken, removeRMDTMasterKey, serviceInit, serviceStart, serviceStop, setResourceManager, setRMDispatcher, storeNewApplication, storeNewApplicationAttempt, storeNewReservation, storeOrUpdateAMRMTokenSecretManager, storeProxyCACert, storeRMDelegationToken, storeRMDTMasterKey, updateApplicationAttemptState, updateApplicationState, updateApplicationState, updateApplicationStateSynchronously, updateFencedState, updateRMDelegationToken

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

LOG

public static final org.slf4j.Logger LOG

Constructor Details

LeveldbRMStateStore

public LeveldbRMStateStore()

Method Details

initInternal

protected void initInternal(org.apache.hadoop.conf.Configuration conf)

Description copied from class: RMStateStore

Derived classes initialize themselves using this method.

Specified by:

initInternal in class RMStateStore

Parameters:

conf - Configuration.

startInternal

protected void startInternal()
                      throws Exception

Description copied from class: RMStateStore

Derived classes start themselves using this method. The base class is started and the event dispatcher is ready to use at this point.

Specified by:

startInternal in class RMStateStore

Throws:

Exception - error occur.

closeInternal

protected void closeInternal()
                      throws Exception

Description copied from class: RMStateStore

Derived classes close themselves using this method. The base class will be closed and the event dispatcher will be shutdown after this.

Specified by:

closeInternal in class RMStateStore

Throws:

Exception - error occur.

loadVersion

protected org.apache.hadoop.yarn.server.records.Version loadVersion()
                                                             throws Exception

Description copied from class: RMStateStore

Derived class use this method to load the version information from state store.

Specified by:

loadVersion in class RMStateStore

Returns:

current version.

Throws:

Exception - error occur.

storeVersion

protected void storeVersion()
                     throws Exception

Description copied from class: RMStateStore

Derived class use this method to store the version information.

Specified by:

storeVersion in class RMStateStore

Throws:

Exception - error occur.

storeVersion

protected void storeVersion(org.apache.hadoop.yarn.server.records.Version version)

getCurrentVersion

protected org.apache.hadoop.yarn.server.records.Version getCurrentVersion()

Description copied from class: RMStateStore

Get the current version of the underlying state store.

Specified by:

getCurrentVersion in class RMStateStore

Returns:

current version.

getAndIncrementEpoch

public long getAndIncrementEpoch()
                          throws Exception

Description copied from class: RMStateStore

Get the current epoch of RM and increment the value.

Specified by:

getAndIncrementEpoch in class RMStateStore

Returns:

current epoch.

Throws:

Exception - error occur.

loadState

public RMStateStore.RMState loadState()
                               throws Exception

Description copied from class: RMStateStore

Blocking API The derived class must recover state from the store and return a new RMState object populated with that state This must not be called on the dispatcher thread.

Specified by:

loadState in class RMStateStore

Returns:

RMState.

Throws:

Exception - error occur.

storeApplicationStateInternal

protected void storeApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId appId,
 ApplicationStateData appStateData)
                                      throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to store the state of an application.

Specified by:

storeApplicationStateInternal in class RMStateStore

Parameters:

appId - application Id.

appStateData - application StateData.

Throws:

IOException

updateApplicationStateInternal

protected void updateApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId appId,
 ApplicationStateData appStateData)
                                       throws IOException

Specified by:

updateApplicationStateInternal in class RMStateStore

Throws:

IOException

storeApplicationAttemptStateInternal

protected void storeApplicationAttemptStateInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId,
 ApplicationAttemptStateData attemptStateData)
                                             throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to store the state of an application attempt.

Specified by:

storeApplicationAttemptStateInternal in class RMStateStore

Parameters:

attemptId - Application AttemptId.

attemptStateData - Application AttemptStateData.

Throws:

IOException

updateApplicationAttemptStateInternal

protected void updateApplicationAttemptStateInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId,
 ApplicationAttemptStateData attemptStateData)
                                              throws IOException

Specified by:

updateApplicationAttemptStateInternal in class RMStateStore

Throws:

IOException

removeApplicationAttemptInternal

public void removeApplicationAttemptInternal(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)
                                      throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to remove the state of specified attempt.

Specified by:

removeApplicationAttemptInternal in class RMStateStore

Parameters:

attemptId - application attempt id.

Throws:

IOException

removeApplicationStateInternal

protected void removeApplicationStateInternal(ApplicationStateData appState)
                                       throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to remove the state of an application and its attempts.

Specified by:

removeApplicationStateInternal in class RMStateStore

Parameters:

appState - ApplicationStateData.

Throws:

IOException

storeReservationState

protected void storeReservationState(org.apache.hadoop.yarn.proto.YarnProtos.ReservationAllocationStateProto reservationAllocation,
 String planName,
 String reservationIdName)
                              throws Exception

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to store the state of a reservation allocation.

Specified by:

storeReservationState in class RMStateStore

Parameters:

reservationAllocation - reservation Allocation.

planName - plan Name.

reservationIdName - reservationId Name.

Throws:

Exception - error occurs.

removeReservationState

protected void removeReservationState(String planName,
 String reservationIdName)
                               throws Exception

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to remove the state of a reservation allocation.

Specified by:

removeReservationState in class RMStateStore

Parameters:

planName - plan Name.

reservationIdName - reservationId Name.

Throws:

Exception - exception occurs.

storeRMDelegationTokenState

protected void storeRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier tokenId,
 Long renewDate)
                                    throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to store the state of RMDelegationToken and sequence number.

Specified by:

storeRMDelegationTokenState in class RMStateStore

Parameters:

tokenId - RMDelegationTokenIdentifier.

renewDate - token renew date.

Throws:

IOException

updateRMDelegationTokenState

protected void updateRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier tokenId,
 Long renewDate)
                                     throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to update the state of RMDelegationToken and sequence number.

Specified by:

updateRMDelegationTokenState in class RMStateStore

Parameters:

tokenId - RMDelegationTokenIdentifier.

renewDate - token renew date.

Throws:

IOException

removeRMDelegationTokenState

protected void removeRMDelegationTokenState(org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier tokenId)
                                     throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to remove the state of RMDelegationToken.

Specified by:

removeRMDelegationTokenState in class RMStateStore

Parameters:

tokenId - RMDelegationTokenIdentifier.

Throws:

IOException

storeRMDTMasterKeyState

protected void storeRMDTMasterKeyState(org.apache.hadoop.security.token.delegation.DelegationKey masterKey)
                                throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to store the state of DelegationToken Master Key.

Specified by:

storeRMDTMasterKeyState in class RMStateStore

Parameters:

masterKey - DelegationToken Master Key.

Throws:

IOException

removeRMDTMasterKeyState

protected void removeRMDTMasterKeyState(org.apache.hadoop.security.token.delegation.DelegationKey masterKey)
                                 throws IOException

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to remove the state of DelegationToken Master Key.

Specified by:

removeRMDTMasterKeyState in class RMStateStore

Parameters:

masterKey - DelegationKey.

Throws:

IOException

storeOrUpdateAMRMTokenSecretManagerState

public void storeOrUpdateAMRMTokenSecretManagerState(AMRMTokenSecretManagerState state,
 boolean isUpdate)

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to store or update the state of AMRMToken Master Key.

Specified by:

storeOrUpdateAMRMTokenSecretManagerState in class RMStateStore

Parameters:

state - amrmTokenSecretManagerState.

isUpdate - true, update; otherwise not update.

storeProxyCACertState

protected void storeProxyCACertState(X509Certificate caCert,
 PrivateKey caPrivateKey)
                              throws Exception

Description copied from class: RMStateStore

Blocking API Derived classes must implement this method to store the CA Certificate and Private Key.

Specified by:

storeProxyCACertState in class RMStateStore

Parameters:

caCert - X509Certificate.

caPrivateKey - PrivateKey.

Throws:

Exception - error occurs.

deleteStore

public void deleteStore()
                 throws IOException

Description copied from class: RMStateStore

Derived classes must implement this method to delete the state store.

Specified by:

deleteStore in class RMStateStore

Throws:

IOException

removeApplication

public void removeApplication(org.apache.hadoop.yarn.api.records.ApplicationId removeAppId)
                       throws IOException

Description copied from class: RMStateStore

Derived classes must implement this method to remove application from the state store.

Specified by:

removeApplication in class RMStateStore

Parameters:

removeAppId - application Id.

Throws:

IOException

setDbManager

@VisibleForTesting
protected void setDbManager(DBManager dbManager)