Package org.apache.hadoop.hdfs.server.namenode

Class FSDirectory

java.lang.Object
org.apache.hadoop.hdfs.server.namenode.FSDirectory
All Implemented Interfaces:
Closeable, AutoCloseable
@Private public class FSDirectory extends Object implements Closeable
Both FSDirectory and FSNamesystem manage the state of the namespace. FSDirectory is a pure in-memory data structure, all of whose operations happen entirely in memory. In contrast, FSNamesystem persists the operations to the disk.
See Also:

  • Field Details

    • DOT_RESERVED_STRING

      public static final String DOT_RESERVED_STRING
      See Also:

    • DOT_RESERVED_PATH_PREFIX

      public static final String DOT_RESERVED_PATH_PREFIX
      See Also:

    • DOT_RESERVED

      public static final byte[] DOT_RESERVED

    • DOT_INODES_STRING

      public static final String DOT_INODES_STRING
      See Also:

    • DOT_INODES

      public static final byte[] DOT_INODES

    • DOT_RESERVED_STATUS

      public static final org.apache.hadoop.hdfs.protocol.HdfsFileStatus DOT_RESERVED_STATUS

    • DOT_SNAPSHOT_DIR_STATUS

      public static final org.apache.hadoop.hdfs.protocol.HdfsFileStatus DOT_SNAPSHOT_DIR_STATUS

    • ezManager

      @VisibleForTesting public final EncryptionZoneManager ezManager

  • Method Details

    • setINodeAttributeProvider

      public void setINodeAttributeProvider(@Nullable INodeAttributeProvider provider)

    • getListLimit

      public int getListLimit()

    • isImageLoaded

      public boolean isImageLoaded()
      Indicates whether the image loading is complete or not.
      Returns:
      true if image loading is complete, false otherwise

    • getProtectedDirectories

      public SortedSet<String> getProtectedDirectories()

    • isProtectedSubDirectoriesEnable

      public boolean isProtectedSubDirectoriesEnable()

    • setMaxDirItems

      public void setMaxDirItems(int newVal)

    • getMaxDirItems

      public int getMaxDirItems()

    • getRoot

      public INodeDirectory getRoot()
      Returns:
      the root directory inode.

    • getBlockStoragePolicySuite

      public BlockStoragePolicySuite getBlockStoragePolicySuite()

    • isPosixAclInheritanceEnabled

      @VisibleForTesting public boolean isPosixAclInheritanceEnabled()

    • setPosixAclInheritanceEnabled

      @VisibleForTesting public void setPosixAclInheritanceEnabled(boolean posixAclInheritanceEnabled)

    • close

      public void close() throws IOException
      Shutdown the filestore
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • resolvePath

      @VisibleForTesting public INodesInPath resolvePath(FSPermissionChecker pc, String src, FSDirectory.DirOp dirOp) throws org.apache.hadoop.fs.UnresolvedLinkException, FileNotFoundException, org.apache.hadoop.security.AccessControlException, org.apache.hadoop.fs.ParentNotDirectoryException
      Resolves a given path into an INodesInPath. All ancestor inodes that exist are validated as traversable directories. Symlinks in the ancestry will generate an UnresolvedLinkException. The returned IIP will be an accessible path that also passed additional sanity checks based on how the path will be used as specified by the DirOp. READ: Expands reserved paths and performs permission checks during traversal. WRITE: In addition to READ checks, ensures the path is not a snapshot path. CREATE: In addition to WRITE checks, ensures path does not contain illegal character sequences.
      Parameters:
      pc - A permission checker for traversal checks. Pass null for no permission checks.
      src - The path to resolve.
      dirOp - The FSDirectory.DirOp that controls additional checks.
      Returns:
      if the path indicates an inode, return path after replacing up to <inodeid> with the corresponding path of the inode, else the path in src as is. If the path refers to a path in the "raw" directory, return the non-raw pathname.
      Throws:
      FileNotFoundException
      org.apache.hadoop.security.AccessControlException
      org.apache.hadoop.fs.ParentNotDirectoryException
      org.apache.hadoop.fs.UnresolvedLinkException

    • unprotectedResolvePath

      public INodesInPath unprotectedResolvePath(String src) throws FileNotFoundException
      This method should only be used from internal paths and not those provided directly by a user. It resolves a given path into an INodesInPath in a similar way to resolvePath(...), only traversal and permissions are not checked.
      Parameters:
      src - The path to resolve.
      Returns:
      if the path indicates an inode, return path after replacing up to <inodeid> with the corresponding path of the inode, else the path in src as is. If the path refers to a path in the "raw" directory, return the non-raw pathname.
      Throws:
      FileNotFoundException

    • isNonEmptyDirectory

      public boolean isNonEmptyDirectory(INodesInPath inodesInPath)
      Returns:
      true if the path is a non-empty directory; otherwise, return false.

    • updateCount

      public void updateCount(INodesInPath iip, INode.QuotaDelta quotaDelta, boolean check) throws org.apache.hadoop.hdfs.protocol.QuotaExceededException
      Throws:
      org.apache.hadoop.hdfs.protocol.QuotaExceededException

    • updateSpaceForCompleteBlock

      public void updateSpaceForCompleteBlock(BlockInfo completeBlk, INodesInPath inodes) throws IOException
      Update the cached quota space for a block that is being completed. Must only be called once, as the block is being completed.
      Parameters:
      completeBlk - - Completed block for which to update space
      inodes - - INodes in path to file containing completeBlk; if null this will be resolved internally
      Throws:
      IOException

    • getStorageTypeDeltas

      public EnumCounters<org.apache.hadoop.fs.StorageType> getStorageTypeDeltas(byte storagePolicyID, long dsDelta, short oldRep, short newRep)

    • addLastINode

      @VisibleForTesting public INodesInPath addLastINode(INodesInPath existing, INode inode, org.apache.hadoop.fs.permission.FsPermission modes, boolean checkQuota, Optional<QuotaCounts> quotaCount) throws org.apache.hadoop.hdfs.protocol.QuotaExceededException
      Add a child to the end of the path specified by INodesInPath.
      Parameters:
      existing - the INodesInPath containing all the ancestral INodes
      inode - the new INode to add
      modes - create modes
      checkQuota - whether to check quota
      Returns:
      an INodesInPath instance containing the new INode
      Throws:
      org.apache.hadoop.hdfs.protocol.QuotaExceededException

    • removeLastINode

      @VisibleForTesting public long removeLastINode(INodesInPath iip)
      Remove the last inode in the path from the namespace. Note: the caller needs to update the ancestors' quota count.
      Returns:
      -1 for failing to remove; 0 for removing a reference whose referred inode has other reference nodes; 1 otherwise.

    • getYieldCount

      @VisibleForTesting public long getYieldCount()

    • getINodeMap

      public INodeMap getINodeMap()

    • addToInodeMap

      public final void addToInodeMap(INode inode)
      This method is always called with writeLock of FSDirectory held.

    • addRootDirToEncryptionZone

      public final void addRootDirToEncryptionZone(XAttrFeature xaf)
      This is to handle encryption zone for rootDir when loading from fsimage, and should only be called during NN restart.

    • removeFromInodeMap

      public final void removeFromInodeMap(List<? extends INode> inodes)
      This method is always called with writeLock of FSDirectory held.

    • getInode

      public INode getInode(long id)
      Get the inode from inodeMap based on its inode id.
      Parameters:
      id - The given id
      Returns:
      The inode associated with the given id

    • getPathComponents

      public static byte[][] getPathComponents(INode inode)
      Given an INode get all the path complents leading to it from the root. If an Inode corresponding to C is given in /A/B/C, the returned patch components will be {root, A, B, C}. Note that this method cannot handle scenarios where the inode is in a snapshot.

    • isReservedName

      public static boolean isReservedName(INode inode)
      Check if a given inode name is reserved

    • isReservedName

      public static boolean isReservedName(String src)
      Check if a given path is reserved

    • isExactReservedName

      public static boolean isExactReservedName(String src)

    • isExactReservedName

      public static boolean isExactReservedName(byte[][] components)

    • getINodesInPath

      public INodesInPath getINodesInPath(String src, FSDirectory.DirOp dirOp) throws org.apache.hadoop.fs.UnresolvedLinkException, org.apache.hadoop.security.AccessControlException, org.apache.hadoop.fs.ParentNotDirectoryException
      Resolves the given path into inodes. Reserved paths are not handled and permissions are not verified. Client supplied paths should be resolved via resolvePath(FSPermissionChecker, String, DirOp). This method should only be used by internal methods.
      Returns:
      the INodesInPath containing all inodes in the path.
      Throws:
      org.apache.hadoop.fs.UnresolvedLinkException
      org.apache.hadoop.fs.ParentNotDirectoryException
      org.apache.hadoop.security.AccessControlException

    • getINodesInPath

      public INodesInPath getINodesInPath(byte[][] components, FSDirectory.DirOp dirOp) throws org.apache.hadoop.fs.UnresolvedLinkException, org.apache.hadoop.security.AccessControlException, org.apache.hadoop.fs.ParentNotDirectoryException
      Throws:
      org.apache.hadoop.fs.UnresolvedLinkException
      org.apache.hadoop.security.AccessControlException
      org.apache.hadoop.fs.ParentNotDirectoryException

    • getINode

      @VisibleForTesting public INode getINode(String src) throws org.apache.hadoop.fs.UnresolvedLinkException, org.apache.hadoop.security.AccessControlException, org.apache.hadoop.fs.ParentNotDirectoryException
      Get INode associated with the file / directory. See getINode(String, DirOp)
      Throws:
      org.apache.hadoop.fs.UnresolvedLinkException
      org.apache.hadoop.security.AccessControlException
      org.apache.hadoop.fs.ParentNotDirectoryException

    • getINode4Write

      @VisibleForTesting public INode getINode4Write(String src) throws org.apache.hadoop.fs.UnresolvedLinkException, org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.ParentNotDirectoryException
      Get INode associated with the file / directory. See getINode(String, DirOp)
      Throws:
      org.apache.hadoop.fs.UnresolvedLinkException
      org.apache.hadoop.security.AccessControlException
      FileNotFoundException
      org.apache.hadoop.fs.ParentNotDirectoryException

    • getINode

      public INode getINode(String src, FSDirectory.DirOp dirOp) throws org.apache.hadoop.fs.UnresolvedLinkException, org.apache.hadoop.security.AccessControlException, org.apache.hadoop.fs.ParentNotDirectoryException
      Get INode associated with the file / directory.
      Throws:
      org.apache.hadoop.fs.UnresolvedLinkException
      org.apache.hadoop.security.AccessControlException
      org.apache.hadoop.fs.ParentNotDirectoryException

    • getLastInodeId

      public long getLastInodeId()
      Returns:
      the last inode ID.

    • resetLastInodeIdWithoutChecking

      @VisibleForTesting public void resetLastInodeIdWithoutChecking(long newValue)
      Should only be used for tests to reset to any value.
      Parameters:
      newValue - new value to set to