Package org.apache.hadoop.fs
Class Hdfs
java.lang.Object
org.apache.hadoop.fs.AbstractFileSystem
org.apache.hadoop.fs.Hdfs
- All Implemented Interfaces:
org.apache.hadoop.fs.PathCapabilities
@Private
@Evolving
public class Hdfs
extends org.apache.hadoop.fs.AbstractFileSystem
-
Field Summary
Fields inherited from class org.apache.hadoop.fs.AbstractFileSystem
statistics -
Method Summary
Modifier and TypeMethodDescriptionvoidaccess(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode) voidcancelDelegationToken(org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier> token) Deprecated.Use Token.cancel instead.createInternal(org.apache.hadoop.fs.Path f, EnumSet<org.apache.hadoop.fs.CreateFlag> createFlag, org.apache.hadoop.fs.permission.FsPermission absolutePermission, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt, boolean createParent) org.apache.hadoop.fs.PathcreateSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) voidcreateSymlink(org.apache.hadoop.fs.Path target, org.apache.hadoop.fs.Path link, boolean createParent) booleandelete(org.apache.hadoop.fs.Path f, boolean recursive) voiddeleteSnapshot(org.apache.hadoop.fs.Path snapshotDir, String snapshotName) org.apache.hadoop.fs.permission.AclStatusgetAclStatus(org.apache.hadoop.fs.Path path) Collection<? extends org.apache.hadoop.fs.BlockStoragePolicySpi>List<org.apache.hadoop.security.token.Token<?>>getDelegationTokens(String renewer) org.apache.hadoop.fs.BlockLocation[]getFileBlockLocations(org.apache.hadoop.fs.Path p, long start, long len) The returned BlockLocation will have different formats for replicated and erasure coded file.org.apache.hadoop.fs.FileChecksumgetFileChecksum(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatusgetFileLinkStatus(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatusgetFileStatus(org.apache.hadoop.fs.Path f) If the given path is a symlink, the path will be resolved to a target path and it will get the resolved path's FileStatus object.org.apache.hadoop.fs.FsStatusorg.apache.hadoop.fs.PathgetLinkTarget(org.apache.hadoop.fs.Path p) org.apache.hadoop.fs.FsServerDefaultsDeprecated.org.apache.hadoop.fs.FsServerDefaultsgetServerDefaults(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.BlockStoragePolicySpigetStoragePolicy(org.apache.hadoop.fs.Path src) intbyte[]getXAttrs(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path>listCorruptFileBlocks(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus>listLocatedStatus(org.apache.hadoop.fs.Path p) The BlockLocation of returned LocatedFileStatus will have different formats for replicated and erasure coded file.org.apache.hadoop.fs.FileStatus[]listStatus(org.apache.hadoop.fs.Path f) If any of the the immediate children of the given path f is a symlink, the returned FileStatus object of that children would be represented as a symlink.org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus>listStatusIterator(org.apache.hadoop.fs.Path f) listXAttrs(org.apache.hadoop.fs.Path path) voidmkdir(org.apache.hadoop.fs.Path dir, org.apache.hadoop.fs.permission.FsPermission permission, boolean createParent) voidmodifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) voidmsync()Synchronize client metadata state with Active NameNode.open(org.apache.hadoop.fs.Path f, int bufferSize) voidremoveAcl(org.apache.hadoop.fs.Path path) voidremoveAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) voidremoveDefaultAcl(org.apache.hadoop.fs.Path path) voidremoveXAttr(org.apache.hadoop.fs.Path path, String name) voidrenameInternal(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) voidrenameInternal(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, boolean overwrite) voidrenameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName) longrenewDelegationToken(org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier> token) Deprecated.Use Token.renew instead.voidsatisfyStoragePolicy(org.apache.hadoop.fs.Path path) voidvoidvoidsetPermission(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission) booleansetReplication(org.apache.hadoop.fs.Path f, short replication) voidsetStoragePolicy(org.apache.hadoop.fs.Path path, String policyName) voidsetTimes(org.apache.hadoop.fs.Path f, long mtime, long atime) voidsetVerifyChecksum(boolean verifyChecksum) voidsetXAttr(org.apache.hadoop.fs.Path path, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag) booleanbooleantruncate(org.apache.hadoop.fs.Path f, long newLength) voidunsetStoragePolicy(org.apache.hadoop.fs.Path src) Methods inherited from class org.apache.hadoop.fs.AbstractFileSystem
checkPath, checkScheme, clearStatistics, create, createFileSystem, createMultipartUploader, equals, get, getAllStatistics, getEnclosingRoot, getFsStatus, getHomeDirectory, getInitialWorkingDirectory, getStatistics, getStatistics, getUri, getUriPath, hashCode, hasPathCapability, isValidName, makeQualified, methodNotSupported, open, openFileWithOptions, printStatistics, rename, resolvePath, setXAttr
-
Method Details
-
getUriDefaultPort
public int getUriDefaultPort()- Specified by:
getUriDefaultPortin classorg.apache.hadoop.fs.AbstractFileSystem
-
createInternal
public HdfsDataOutputStream createInternal(org.apache.hadoop.fs.Path f, EnumSet<org.apache.hadoop.fs.CreateFlag> createFlag, org.apache.hadoop.fs.permission.FsPermission absolutePermission, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt, boolean createParent) throws IOException - Specified by:
createInternalin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
delete
public boolean delete(org.apache.hadoop.fs.Path f, boolean recursive) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
deletein classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
getFileBlockLocations
public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.Path p, long start, long len) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException The returned BlockLocation will have different formats for replicated and erasure coded file. Please refer toFileContext.getFileBlockLocations(Path, long, long)for more details.- Specified by:
getFileBlockLocationsin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
getFileChecksum
public org.apache.hadoop.fs.FileChecksum getFileChecksum(org.apache.hadoop.fs.Path f) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
getFileChecksumin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
getFileStatus
public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path f) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException If the given path is a symlink, the path will be resolved to a target path and it will get the resolved path's FileStatus object. It will not be represented as a symlink and isDirectory API returns true if the resolved path is a directory, false otherwise.- Specified by:
getFileStatusin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
msync
Synchronize client metadata state with Active NameNode.In HA the client synchronizes its state with the Active NameNode in order to guarantee subsequent read consistency from Observer Nodes.
- Overrides:
msyncin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getFileLinkStatus
public org.apache.hadoop.fs.FileStatus getFileLinkStatus(org.apache.hadoop.fs.Path f) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Overrides:
getFileLinkStatusin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
getFsStatus
- Specified by:
getFsStatusin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getServerDefaults
Deprecated.- Specified by:
getServerDefaultsin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getServerDefaults
public org.apache.hadoop.fs.FsServerDefaults getServerDefaults(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
getServerDefaultsin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
listLocatedStatus
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listLocatedStatus(org.apache.hadoop.fs.Path p) throws FileNotFoundException, IOException The BlockLocation of returned LocatedFileStatus will have different formats for replicated and erasure coded file. Please refer toFileContext.getFileBlockLocations(Path, long, long)for more details.- Overrides:
listLocatedStatusin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
FileNotFoundExceptionIOException
-
listStatusIterator
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> listStatusIterator(org.apache.hadoop.fs.Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException - Overrides:
listStatusIteratorin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
org.apache.hadoop.security.AccessControlExceptionFileNotFoundExceptionorg.apache.hadoop.fs.UnresolvedLinkExceptionIOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException If any of the the immediate children of the given path f is a symlink, the returned FileStatus object of that children would be represented as a symlink. It will not be resolved to the target path and will not get the target path FileStatus object. The target path will be available via getSymlink on that children's FileStatus object. Since it represents as symlink, isDirectory on that children's FileStatus will return false. If you want to get the FileStatus of target path for that children, you may want to use GetFileStatus API with that children's symlink path. Please seegetFileStatus(Path f)- Specified by:
listStatusin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
listCorruptFileBlocks
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path> listCorruptFileBlocks(org.apache.hadoop.fs.Path path) throws IOException - Overrides:
listCorruptFileBlocksin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
mkdir
public void mkdir(org.apache.hadoop.fs.Path dir, org.apache.hadoop.fs.permission.FsPermission permission, boolean createParent) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
mkdirin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
open
public HdfsDataInputStream open(org.apache.hadoop.fs.Path f, int bufferSize) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
openin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
truncate
public boolean truncate(org.apache.hadoop.fs.Path f, long newLength) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Overrides:
truncatein classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
renameInternal
public void renameInternal(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
renameInternalin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
renameInternal
public void renameInternal(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, boolean overwrite) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Overrides:
renameInternalin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
setOwner
public void setOwner(org.apache.hadoop.fs.Path f, String username, String groupname) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
setOwnerin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
setPermission
public void setPermission(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
setPermissionin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
setReplication
public boolean setReplication(org.apache.hadoop.fs.Path f, short replication) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
setReplicationin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
setTimes
public void setTimes(org.apache.hadoop.fs.Path f, long mtime, long atime) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Specified by:
setTimesin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
setVerifyChecksum
- Specified by:
setVerifyChecksumin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
supportsSymlinks
public boolean supportsSymlinks()- Overrides:
supportsSymlinksin classorg.apache.hadoop.fs.AbstractFileSystem
-
createSymlink
public void createSymlink(org.apache.hadoop.fs.Path target, org.apache.hadoop.fs.Path link, boolean createParent) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException - Overrides:
createSymlinkin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOExceptionorg.apache.hadoop.fs.UnresolvedLinkException
-
getLinkTarget
- Overrides:
getLinkTargetin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getCanonicalServiceName
- Overrides:
getCanonicalServiceNamein classorg.apache.hadoop.fs.AbstractFileSystem
-
getDelegationTokens
public List<org.apache.hadoop.security.token.Token<?>> getDelegationTokens(String renewer) throws IOException - Overrides:
getDelegationTokensin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
modifyAclEntries
public void modifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
modifyAclEntriesin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
removeAclEntries
public void removeAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
removeAclEntriesin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
removeDefaultAcl
- Overrides:
removeDefaultAclin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
removeAcl
- Overrides:
removeAclin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
setAcl
public void setAcl(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
setAclin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getAclStatus
public org.apache.hadoop.fs.permission.AclStatus getAclStatus(org.apache.hadoop.fs.Path path) throws IOException - Overrides:
getAclStatusin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
setXAttr
public void setXAttr(org.apache.hadoop.fs.Path path, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag) throws IOException - Overrides:
setXAttrin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getXAttr
- Overrides:
getXAttrin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getXAttrs
- Overrides:
getXAttrsin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getXAttrs
public Map<String,byte[]> getXAttrs(org.apache.hadoop.fs.Path path, List<String> names) throws IOException - Overrides:
getXAttrsin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
listXAttrs
- Overrides:
listXAttrsin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
removeXAttr
- Overrides:
removeXAttrin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
access
public void access(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode) throws IOException - Overrides:
accessin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
satisfyStoragePolicy
- Overrides:
satisfyStoragePolicyin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
setStoragePolicy
- Overrides:
setStoragePolicyin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
unsetStoragePolicy
- Overrides:
unsetStoragePolicyin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getStoragePolicy
public org.apache.hadoop.fs.BlockStoragePolicySpi getStoragePolicy(org.apache.hadoop.fs.Path src) throws IOException - Overrides:
getStoragePolicyin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
getAllStoragePolicies
public Collection<? extends org.apache.hadoop.fs.BlockStoragePolicySpi> getAllStoragePolicies() throws IOException- Overrides:
getAllStoragePoliciesin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
renewDelegationToken
public long renewDelegationToken(org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier> token) throws org.apache.hadoop.security.token.SecretManager.InvalidToken, IOException Deprecated.Use Token.renew instead.Renew an existing delegation token.- Parameters:
token- delegation token obtained earlier- Returns:
- the new expiration time
- Throws:
org.apache.hadoop.security.token.SecretManager.InvalidTokenIOException
-
cancelDelegationToken
public void cancelDelegationToken(org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier> token) throws org.apache.hadoop.security.token.SecretManager.InvalidToken, IOException Deprecated.Use Token.cancel instead.Cancel an existing delegation token.- Parameters:
token- delegation token- Throws:
org.apache.hadoop.security.token.SecretManager.InvalidTokenIOException
-
createSnapshot
public org.apache.hadoop.fs.Path createSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) throws IOException - Overrides:
createSnapshotin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
renameSnapshot
public void renameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName) throws IOException - Overrides:
renameSnapshotin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-
deleteSnapshot
public void deleteSnapshot(org.apache.hadoop.fs.Path snapshotDir, String snapshotName) throws IOException - Overrides:
deleteSnapshotin classorg.apache.hadoop.fs.AbstractFileSystem- Throws:
IOException
-