Package org.apache.hadoop.hdfs.server.namenode

Class ErasureCodingPolicyManager

java.lang.Object
org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager
@LimitedPrivate("HDFS") public final class ErasureCodingPolicyManager extends Object
This manages erasure coding policies predefined and activated in the system. It loads customized policies and syncs with persisted ones in NameNode image. This class is instantiated by the FSNamesystem.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static org.slf4j.Logger
    LOG
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
    addPolicy(org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy policy)
    Add an erasure coding policy.
    static boolean
    checkStoragePolicySuitableForECStripedMode(byte storagePolicyID)
     
    void
    clear()
    Clear and clean up.
    boolean
    disablePolicy(String name)
    Disable an erasure coding policy by policyName.
    boolean
    enablePolicy(String name)
    Enable an erasure coding policy by policyName.
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
    getByID(byte id)
    Get a ErasureCodingPolicy by policy ID, including system policy and user defined policy.
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
    getByName(String name)
    Get a ErasureCodingPolicy by policy name, including system policy and user defined policy.
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy[]
    getCopyOfEnabledPolicies()
     
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy[]
    getEnabledPolicies()
    Get the set of enabled policies.
    String
    getEnabledPoliciesMetric()
     
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
    getEnabledPolicyByName(String name)
    Get enabled policy by policy name.
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
    getErasureCodingPolicyByName(String name)
    Get a ErasureCodingPolicy by policy name, including system policy, user defined policy and Replication policy.
    static ErasureCodingPolicyManager
    getInstance()
     
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo[]
    getPersistedPolicies()
    Get all system defined policies and user defined policies as it is written out in the fsimage.
    org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo[]
    getPolicies()
    Get all system defined policies and user defined policies.
    List<org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy>
    getRemovedPolicies()
     
    void
    init(org.apache.hadoop.conf.Configuration conf)
     
    void
    loadPolicies(List<org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo> ecPolicies, org.apache.hadoop.conf.Configuration conf)
    Reload erasure coding policies from fsImage.
    void
    removePolicy(String name)
    Remove an User erasure coding policy by policyName.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • LOG

      public static org.slf4j.Logger LOG

  • Method Details

    • getInstance

      public static ErasureCodingPolicyManager getInstance()

    • init

      public void init(org.apache.hadoop.conf.Configuration conf) throws IOException
      Throws:
      IOException

    • getEnabledPolicies

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy[] getEnabledPolicies()
      Get the set of enabled policies.
      Returns:
      all policies

    • getEnabledPolicyByName

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy getEnabledPolicyByName(String name)
      Get enabled policy by policy name.

    • checkStoragePolicySuitableForECStripedMode

      public static boolean checkStoragePolicySuitableForECStripedMode(byte storagePolicyID)
      Returns:
      if the specified storage policy ID is suitable for striped EC files.

    • getPolicies

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo[] getPolicies()
      Get all system defined policies and user defined policies.
      Returns:
      all policies

    • getPersistedPolicies

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo[] getPersistedPolicies()
      Get all system defined policies and user defined policies as it is written out in the fsimage. The difference between persisted policies and all policies is that if a default policy is only enabled at startup, it will appear as disabled in the persisted policy list and in the fsimage.
      Returns:
      persisted policies

    • getCopyOfEnabledPolicies

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy[] getCopyOfEnabledPolicies()

    • getByID

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy getByID(byte id)
      Get a ErasureCodingPolicy by policy ID, including system policy and user defined policy.
      Returns:
      ecPolicy, or null if not found

    • getByName

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy getByName(String name)
      Get a ErasureCodingPolicy by policy name, including system policy and user defined policy.
      Returns:
      ecPolicy, or null if not found

    • getErasureCodingPolicyByName

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy getErasureCodingPolicyByName(String name)
      Get a ErasureCodingPolicy by policy name, including system policy, user defined policy and Replication policy.
      Returns:
      ecPolicy, or null if not found

    • clear

      public void clear()
      Clear and clean up.

    • addPolicy

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy addPolicy(org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy policy)
      Add an erasure coding policy.
      Returns:
      the added policy

    • removePolicy

      public void removePolicy(String name)
      Remove an User erasure coding policy by policyName.

    • getRemovedPolicies

      @VisibleForTesting public List<org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy> getRemovedPolicies()

    • disablePolicy

      public boolean disablePolicy(String name)
      Disable an erasure coding policy by policyName.

    • enablePolicy

      public boolean enablePolicy(String name)
      Enable an erasure coding policy by policyName.

    • loadPolicies

      public void loadPolicies(List<org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo> ecPolicies, org.apache.hadoop.conf.Configuration conf) throws IOException
      Reload erasure coding policies from fsImage.
      Parameters:
      ecPolicies - contains ErasureCodingPolicy list
      Throws:
      IOException

    • getEnabledPoliciesMetric

      public String getEnabledPoliciesMetric()