All Classes and Interfaces
Class
Description
Abort data being written to a stream, so that close() does
not write the data.
Interface for the result of aborts; allows subclasses to extend
(IOStatistics etc) or for future enhancements if ever needed.
Class to encapsulate a token's renew date and password.
Look through tokens to find the first delegation token that matches the
service and return it.
This is a base class for DNS to Switch mappings.
This class provides an interface for implementors of a Hadoop file system
(analogous to the VFS of Unix).
Builder for filesystem/filecontext operations of various kinds,
with option support.
This the base class for Ganglia sink classes using metrics2.
define enum for various type of conf
ganglia slope values which equal the ordinal
Base implementation in case common methods/fields need to be added
in future.
Abstract class for implementing credential providers that are based on
Java Keystores as the underlying credential store.
Subclass of
AbstractService that provides basic implementations
of the LaunchableService methods.Abstract base class for MapWritable and SortedMapWritable
Unlike org.apache.nutch.crawl.MapWritable, this class allows creation of
MapWritable<Writable, MapWritable> so the CLASS_TO_ID and ID_TO_CLASS
maps travel with the class instead of being static.
The immutable metric
Standard base class for Multipart Uploaders.
Base class for pattern based filters
This is the base implementation class for services.
An exception class for access control related issues.
Class representing a configured access control list.
Defines a single entry in an ACL.
Builder for creating new AclEntry instances.
Specifies the scope or intended usage of an ACL entry.
Specifies the type of an ACL entry.
An AclStatus contains the ACL information of a specific file.
Builder for creating new Acl instances.
AclUtil contains utility methods for manipulating ACLs.
This class implements a simple library to perform leader election on top of
Apache Zookeeper.
Exception thrown when there is no active leader
Callback interface to interact with the ActiveStandbyElector object.
Interface to get the active thread span.
General servlet which is admin-authorized.
This interface intends to align the state between client and server
via RPC communication.
An exception class for when a closed compressor/decopressor is being used
Compressor
DecompressorConstructs SecurityInfo from Annotations provided in protocol interface.
A
URLClassLoader for application isolation.A dense file-based mapping from integers to values.
Provide access to an existing array file.
Write a new array file.
This is a wrapper class.
A Writable for arrays containing instances of a class.
Handle async calls.
Signals that an AsyncCallLimitExceededException has occurred.
This interface defines an asynchronous
AsyncGet.get(long, TimeUnit) method.Utility
Used to mark certain methods of an interface with at-most-once semantics.
Constants related to auditing.
A marker attribute simply to highlight which of the methods
in a FileSystem why are audit entry points.
Static methods to assist in working with Audit Spans.
This is a span created by an
AuditSpanSource.A source of audit spans.
Statistic Names for Auditing.
Initializes hadoop-auth AuthenticationFilter which provides support for
Kerberos HTTP SPNEGO authentication.
Utility for managing a thread-local authorization header for RPC calls.
An exception class for authorization-related issues.
This is a wrap class of a ReentrantLock.
Adapts an
FSDataInputStream to Avro's SeekableInput interface.Tag interface for Avro 'reflect' serializable classes.
Serialization for Avro Reflect classes.
Base class for providing serialization to Avro types.
Serialization for Avro Specific classes.
Holds reference to an object to be attached to a stream or store to avoid
the reference being lost to GC.
Indicates that the operator has specified an invalid configuration
for fencing methods.
Abstract expression for use in the
Find command.BasicDiskValidator is a wrapper around
DiskChecker.A RemoteIterator that fetches elements in batches.
Interface filesystems MAY implement to offer a batched list.
Byte array backed part handle.
Byte array backed upload handle.
Function of arity 2 which may raise an IOException.
Interface supported by
WritableComparable
types supporting ordering/permutation by a representative set of bytes.Utility methods to assist binding to Hadoop APIs through reflection.
Provides functionality necessary for caching blocks of data read from FileSystem.
A
CompressorStream which works
with 'block-based' based compression algorithms, as opposed to
'stream-based' compression algorithms.Holds information about blocks of data in a file.
A
DecompressorStream which works
with 'block-based' based compression algorithms, as opposed to
'stream-based' compression algorithms.As part of a codec, to handle how to form a block group for encoding
and provide instructions on how to recover erased blocks from a block group
This ExecutorService blocks the submission of new tasks when its queue is
already full by using a semaphore.
Represents the network location of a block, information about the hosts
that contain block replicas, and other block metadata (E.g. the file
offset associated with the block, length, whether it is corrupt, etc).
Provides read access to the underlying file one block at a time.
This class is used to provide parameters to
BlockManager.Block level operations performed on a file.
Operation kind.
A storage policy specifies the placement of block replicas on specific
storage types.
Implements a Bloom filter, as defined by Bloom in 1970.
This class extends
MapFile and provides very much the same
functionality.A WritableComparable for booleans.
A Comparator optimized for BooleanWritable.
CredentialProvider based on Bouncy Castle FIPS KeyStore file format.
The factory to create JksProviders, which is used by the ServiceLoader.
A byte array backed output stream with a limit.
BoundedRangeFIleInputStream abstracts a contiguous region of a Hadoop
FSDataInputStream as a regular input stream.
Manages a fixed pool of resources.
Holds the state of a ByteBuffer that is in use by
CachingBlockManager.A class that optimizes reading from FSInputStream by buffering.
An extension of
BufferedInputStream which implements
IOStatisticsSource and forwards requests for the
IOStatistics to the wrapped stream.An extension of
BufferedOutputStream which implements
IOStatisticsSource and forwards requests for the
IOStatistics to the wrapped stream.Manages a fixed pool of
ByteBuffer instances.A
Compressor based on the popular gzip compressed file format.A
Decompressor based on the popular gzip compressed file format.The current state of the gzip decoder, external to the Inflater context.
A wrapper around java.util.zip.Deflater to make it conform
to org.apache.hadoop.io.compress.Compressor interface.
A wrapper around java.util.zip.Inflater to make it conform
to org.apache.hadoop.io.compress.Decompressor interface.
API for bulk deletion of objects/files,
but not directories.
Interface for bulk deletion.
Utility class for bulk delete operations.
Adaptor class to wrap byte-array backed objects (including java byte array)
as RawComparable objects.
Provide an input stream from a byte buffer; supporting
ByteBufferInputStream.mark(int).Implementers of this interface provide a positioned read API that writes to a
ByteBuffer rather than a byte[].Implementers of this interface provide a read API that writes to a
ByteBuffer, not a byte[].
A byte sequence that is usable as a key or value.
A Comparator optimized for BytesWritable.
A WritableComparable for a single byte.
A Comparator optimized for ByteWritable.
This class provides output and input streams for bzip2 compression
and decompression.
A
Compressor based on the popular
bzip2 compression algorithm.Base class for both the compress and decompress classes.
A
Decompressor based on the popular
bzip2 compression algorithm.This is a dummy compressor for BZip2.
This is a dummy decompressor for BZip2.
A collection of factories to create the right
bzip2 compressor/decompressor instances.
CacheableIPList loads a list of subnets from a file.
A cached implementation of DNSToSwitchMapping that takes an
raw DNSToSwitchMapping and stores the resolved network location in
a cache.
Provides read access to the underlying file one block at a time.
Interface for class that can tell estimate much space
is used in a directory.
A
KeyProviderExtension implementation providing a short lived
cache for KeyVersions and Metadatato avoid burst
of requests to hit the underlying KeyProvider.This is a callable which only raises an IOException.
A class defining the caller context for auditing coarse granularity
operations.
The caller context builder.
Abstracts queue operations for different blocking queues.
FSDataInputStreams implement this interface to indicate that they can clear
their buffers on request.
An input stream that decompresses from the BZip2 format (without the file
header chars) to be read as any other stream.
A state machine to keep track of current state of the de-coder
An output stream that compresses into the BZip2 format (without the file
header chars) into another stream.
Thrown for checksum errors.
Abstract Checksumed FileSystem.
Abstract Checksumed Fs.
Parse a permission mode passed in from a chmod command and apply that
mode against an existing file.
Simplified List implementation which stores elements as a list
of chunks, each chunk having a maximum size.
Used between client and server to negotiate the
cipher suite, key and iv.
Defines properties of a CipherSuite.
Command-line utility for getting the full classpath needed to launch a Hadoop
client application.
sun.misc.Cleaner has moved in OpenJDK 9 and
sun.misc.Unsafe#invokeCleaner(ByteBuffer) is the replacement.
Pass in an implementation of this interface to cleanup ByteBuffers.
A client for an IPC service.
This class holds the address and the user ticket.
Manages the input and output streams for an IPC connection.
A class defining a set of static helper methods to provide conversion between
bytes and string for UUID-based client Id.
Deprecated.
use java.io.Closeable
A closeable object that maintains a reference count.
A task submitter which is closeable, and whose close() call
shuts down the pool.
Exception to denote if the underlying stream, cache or other closable resource
is closed.
Exception raised by HDFS indicating that storage capacity in the
cluster filesystem is exceeded.
Codec related constants.
A global compressor/decompressor pool used to save and reuse
(possibly native) compression/decompression codecs.
This class registers all coder implementations.
A codec & coder utility to help create coders conveniently.
Helpful utilities for implementing some raw erasure coders.
A file range that represents a set of underlying file ranges.
Util class to stores ips/hosts/subnets.
An abstract class for the execution of a file system command
class to search for and register commands
Parse the args of a command and check the format of args.
Used when a duplicated option is supplied to a command.
Used when the arguments exceed their bounds
Used when too few arguments are supplied to a command
Used when too many arguments are supplied to a command
Used when an unsupported option is supplied to a command
This program is a CLI utility base class utilizing hadoop Tool class.
The common audit context is a map of common context information
which can be used with any audit span.
A bridge from Callable to Supplier; catching exceptions
raised by the callable and wrapping them as appropriate.
This class contains constants for configuration keys used
in the common code.
This class contains constants for configuration keys used
in the common code.
Common path capabilities.
Generic implementation of version comparison.
Composite CRC.
An implementation of
GroupMappingServiceProvider which
composites other group mapping providers for determining group membership.Composition of services.
JVM Shutdown hook for CompositeService which will stop the give
CompositeService gracefully in case of JVM shutdown.
A base-class for Writables which store themselves compressed and lazily
inflate on field access.
Compression related stuff.
Compression algorithms.
This class encapsulates a streaming compression/decompression pair.
A factory that will find the correct codec for a given filename.
A compression input stream.
A compression output stream.
Specification of a stream-based 'compressor' which can be
plugged into a
CompressionOutputStream to compress data.Concat the given files.
Tool for redacting sensitive information when displaying config parameters.
Something that may be configured with a
Configuration.Provides access to configuration parameters.
A pending addition to the global set of deprecated keys.
A class that represents a set of positive integer ranges.
Configuration Helper class to provide advanced configuration parsing.
Logs access to
Configuration.Base class for things that may be configured with a
Configuration.Utilities for config variables of the viewFs See
ViewFsA servlet to print out the running configuration data.
This class validates configuration XML files in ${HADOOP_CONF_DIR} or
specified ones.
Thrown by
NetUtils.connect(java.net.Socket, java.net.SocketAddress, int)
if it times out while connecting to the remote host.Config variable prefixes for ViewFs -
see
ViewFs for examples.Version of java.util.function.Consumer which raises
exceptions.
Store the summary of a content (a directory or a file).
We don't use generics.
Additional helpers (besides guava Preconditions) for programming by contract
Abstract command to enable sub copy commands run with multi-thread.
Used by
DecayRpcScheduler to get the cost of users' operations.Count the number of directories, files, bytes, quota, and remaining quota.
Implements a counting Bloom filter, as defined by Fan et al. in a ToN
2000 paper.
Utility for sampling and computing CPU usage.
Encapsulates logic for composing multiple CRCs into one or more combined CRCs
corresponding to concatenated underlying data ranges.
This class provides utilities for working with CRCs.
CreateFlag specifies the file create semantic.
A provider of credentials or password for Hadoop applications.
The combination of both the alias and the actual credential value.
A factory to create a list of CredentialProvider based on the path given in a
Configuration.
Base class for all credentials.
A class that provides the facilities of reading and writing
secret keys and Tokens.
Credential used by RPCSEC_GSS
This program is the CLI utility for the CredentialProvider facilities in
Hadoop.
To facilitate testing since Console is a final class.
Credential used by AUTH_NONE
Credential used by AUTH_SYS
Crypto codec class, encapsulates encryptor/decryptor pair.
CryptoInputStream decrypts data.
CryptoOutputStream encrypts data.
Versions of the client/server protocol used for HDFS encryption.
Utility methods for the crypto related features.
For handling customized
Callback.A thread that has called
Thread.setDaemon(boolean) with true.Provide a factory for named daemon threads, for use in ExecutorServices
constructors
A class to provide disk, byteBuffer and byteArray option for Filesystem
OutputStreams.
Base class for block factories.
The output information for an upload.
This represents a block being uploaded.
This class provides interface and utilities for processing checksums for
DFS data transfers.
The checksum types
A reusable
DataInput implementation
that reads from an in-memory buffer.A reusable
DataOutput implementation that writes to an in-memory
buffer.OutputStream implementation that wraps a DataOutput.
The decay RPC scheduler tracks the cost of incoming requests in a map, then
decays the costs at a fixed time interval.
This TimerTask will call decayCurrentCosts until
the scheduler has been garbage collected.
MetricsProxy is a singleton because we may init multiple schedulers and we
want to clean up resources when a new scheduler replaces the old one.
This class is for maintaining queue (priority) level related
statistics when FairCallQueue is used and publishing them
through the metrics interface.
Provides metrics for Decay scheduler.
A utility class to validate decoding.
Specification of a stream-based 'de-compressor' which can be
plugged into a
CompressionInputStream to compress data.Default implementation of the
BulkDelete interface.Ignores process details and returns a constant value for each call.
An implementation of
FailoverProxyProvider which does nothing in the
event of failover, and always returns the same proxy object.Experimental interface to extend metrics dynamically
The default metrics system singleton.
No op default RPC scheduler.
DefaultStringifier is the default implementation of the
Stringifier
interface which stringifies the objects using base64 encoding of the
serialized version of the objects.Alias class for DefaultCodec to enable codec discovery by 'deflate' name.
Implementation of AbstractFileSystem based on the existing implementation of
FileSystem.A
SSLSocketFactory that can delegate to various SSL implementations.Default indicates Ordered, preferred OpenSSL, if failed to load then fall
back to Default_JSSE.
Key used for generating and verifying delegation tokens
The
DelegationTokenAuthenticatedURL is a
AuthenticatedURL sub-class with built-in Hadoop Delegation Token
functionality.Client side authentication token that handles Delegation Tokens.
The
DelegationTokenAuthenticationFilter filter is a
AuthenticationFilter with Hadoop Delegation Token support.An
AuthenticationHandler that implements Kerberos SPNEGO mechanism
for HTTP and supports Delegation Token functionality.Authenticator wrapper that enhances an Authenticator with
Delegation Token support.DelegationToken operations.
Concrete delegation token identifier used by
DelegationTokenManager,
KerberosDelegationTokenAuthenticationHandler and
DelegationTokenAuthenticationFilter.Class for issuing delegation tokens.
Cache for delegation tokens that can handle high volume of tokens.
Delegation Token Manager used by the
KerberosDelegationTokenAuthenticationHandler.A daemon thread that waits for the next file system to renew.
The renewable interface used by the renewer.
An action that will renew and replace the file system's delegation
tokens automatically.
Provides a facility for deserializing objects of type <T> from an
InputStream.
A
RawComparator that uses a Deserializer to deserialize
the objects to be compared so that the standard Comparator can
be used to compare them.Filesystem disk space usage statistics.
Fast but inaccurate class to tell how much space HDFS is using.
A simple class for pooling direct ByteBuffers.
This class encapsulates a codec which can decompress direct bytebuffers.
Specification of a direct ByteBuffer 'de-compressor'.
Thrown when the startAfter can't be found when listing a directory.
Class that provides utility functions for checking disk problem
A interface for disk validators.
The factory class to create instance of
DiskValidator.A class that provides direct and reverse lookup functionalities, allowing
the querying of specific network interfaces or nameservers.
DNSDomainNameResolver wraps up the default DNS service for forward/reverse
DNS lookup.
An interface that must be implemented to allow pluggable
DNS-name/IP-address to RackID resolvers.
An interface that must be implemented to allow pluggable
DNS-name/IP-address to RackID resolvers.
This interface provides methods for the failover proxy to get IP addresses
of the associated servers (NameNodes, RBF routers etc).
This class creates the DomainNameResolver instance based on the config.
The implementation of UNIX domain sockets in Java.
The DomainSocketWatcher watches a set of domain sockets to see when they
become readable, or closed.
This is a marker annotation that marks a compressor or decompressor
type as not to be pooled.
Writable for Double values.
A Comparator optimized for DoubleWritable.
DtFetcher is an interface which permits the abstraction and separation of
delegation token fetch implementaions across different packages and
compilation units.
DtFileOperations is a collection of delegation token file operations.
DtUtilShell is a set of command line token file management operations.
Filesystem disk space usage statistics.
Dummy erasure coder does not real coding computing.
Dummy erasure decoder does no real computation.
Dummy erasure encoder does no real computation.
A dummy raw decoder that does no real computation.
A dummy raw encoder that does no real computation.
A raw erasure coder factory for dummy raw coders.
A dump utility class for debugging data erasure coding/decoding issues.
A duration with logging of final state at info or debug
in the
close() call.Summary of duration tracking statistics
as extracted from an IOStatistics instance.
Interface to be implemented by objects which can track duration.
Interface for a source of duration tracking.
Implements a dynamic Bloom filter, as defined in the INFOCOM 2006 paper.
Builder of
DynamicIOStatistics.The wrapped IO methods in
org.apache.hadoop.io.wrappedio.WrappedIO,
dynamically loaded.The wrapped IOStatistics methods in
WrappedStatistics,
dynamically loaded.Dynamic constructors.
Dynamic method invocation.
Convenience wrapper class around
Method.A wrapper of block level data source/output that
ECChunks can be
extracted from.A group of blocks or
ECBlock incurred in an erasure coding task.A wrapper for ByteBuffer or bytes array for an erasure code chunk.
Erasure coding schema to housekeeper relevant information.
This is a simple ByteBufferPool which just creates ByteBuffers as needed.
Empty implementation of the prefetching statistics interface.
A Writable wrapper for EnumSet.
Abstract Erasure Codec is defines the interface of each actual erasure
codec classes.
Constants related to the erasure code feature.
Erasure codec options.
Erasure code native libraries (for now, Intel ISA-L) related utilities.
An erasure coder to perform encoding or decoding given a group.
Erasure coder configuration that maintains schema info and coder options.
Erasure coding step that's involved in encoding/decoding of a block group.
An abstract erasure decoder that's to be inherited by new decoders.
Erasure decoding step, a wrapper of all the necessary information to perform
a decoding step involved in the whole process of decoding a block group.
An abstract erasure encoder that's to be inherited by new encoders.
Erasure encoding step, a wrapper of all the necessary information to perform
an encoding step involved in the whole process of encoding a block group.
Enum representing POSIX errno values.
An etag as a checksum.
An optional interface for
FileStatus subclasses to implement
to provide access to etags.Helper functions for Executors.
A FuturePool implementation backed by a java.util.concurrent.ExecutorService.
Get the exit code of an exception.
Facilitates hooking process termination for tests, debugging
and embedding.
An exception raised when a call to
ExitUtil.terminate(int) was
called and system exits were blocked.An exception raised when a call to
ExitUtil.terminate(int) was
called and system halts were blocked.Interface describing an expression to be used in the
Find command.The FailOverController is responsible for electing an active service
on startup or when the current active is changing (eg due to failure),
monitoring the health of a service, and performing a fail-over when a
new active service is either manually selected by a user or elected.
Exception thrown to indicate service failover has failed.
An implementer of this interface is capable of providing proxy objects for
use in IPC communication, and potentially modifying these objects or creating
entirely new ones in the event of certain types of failures.
A queue with multiple levels for each priority.
Fast thread-safe version of NumberFormat
Class for dealing with caching SASL client factories.
Class for dealing with caching SASL server factories.
A fencing method is a method by which one node can forcibly prevent
another node from making continued progress.
Used when target file already exists for any operation and
is not configured to be overwritten.
FileBasedIPList loads a list of subnets in CIDR format and ip addresses from
a file.
KeyStoresFactory implementation that reads the certificates from
keystore files.An abstract class representing file checksums for files.
The FileContext class provides an interface for users of the Hadoop
file system.
FileEncryptionInfo encapsulates all the encryption-related information for
an encrypted file.
Implements basic logic to track when a file changes on disk and call the action
passed to the constructor when it does.
Provides functionality related to tracking the position within a file.
A byte range of a file.
A range of bytes from a file with an optional buffer to read those bytes
for zero copy.
A metrics sink that writes to a file
Interface that represents the client side information for a file.
Flags for entity attributes.
An abstract base class for a fairly generic filesystem.
Represents a batch of directory entries when iteratively listing a
directory.
Tracks statistics about how many reads, writes, and so forth have been
done in a FileSystem.
Statistics data.
FileSystem-specific class used to operate on and resolve symlinks in a path.
A MultipartUploader that uses the basic FileSystem commands.
Builder for
FileSystemMultipartUploader.Common statistic names for Filesystem-level statistics,
including internals.
A basic StorageStatistics instance which simply returns data from
FileSystem#Statistics.
A collection of file-processing util methods
Deprecated.
Use
HardLinkDefines the general behavior of a filter.
A container class for javax.servlet.Filter.
Provides an abstract composition filter for the
Expression interface.A
FilterFileSystem contains
some other file system, which it uses as
its basic file system, possibly transforming
the data along the way or providing additional
functionality.A
FilterFs contains some other file system, which it uses as its
basic file system, possibly transforming the data along the way or providing
additional functionality.Initialize a javax.servlet.Filter.
This entry point exists for diagnosing classloader problems:
is a class or resource present -and if so, where?
Options to be used by the
Find command and its Expressions.A set of flags, constructed from a configuration option or from a string,
with the semantics of
ConfigurationHelper.parseEnumSet(String, String, Class, boolean)
and implementing StreamCapabilities.A WritableComparable for floats.
A Comparator optimized for FloatWritable.
This may seem odd having an IOStatisticsStore which does nothing
but forward to a wrapped store, but it's designed to
assist in subclassing of selective methods, such
as those to increment counters, get durations etc.
File system actions, e.g. read, write, etc.
The base interface which various FileSystem FileContext Builder
interfaces can extend, and which underlying implementations
will then implement.
Class to help with use of FSBuilder.
Base class for all "hadoop fs" commands.
FileSystem related constants.
A class that stores both masked and unmasked create modes
and is a drop-in replacement for masked permission.
Utility that wraps a
FSInputStream in a DataInputStream
and buffers input through a BufferedInputStream.Utility that wraps a
OutputStream in a DataOutputStream.Builder for
FSDataOutputStream and its subclasses.Thrown for unexpected filesystem errors, presumed to reflect disk errors
in the native filesystem.
Standard strings to use in exception messages in filesystems
HDFS is used as the reference source of the strings
File system instance getter.
This is a generic input stream for verifying checksums for
data before it is read by a user.
FSInputStream is a generic old InputStream with a little bit
of RAF-style seek ability.
Class to allow Lambda expressions to be used in
FileContext
link resolution.The signature of the function to invoke.
Used primarily by
FileContext to operate on and resolve
symlinks in a path.This is a generic output stream for generating checksums for
data before it is written to the underlying stream
A class for file/directory permissions.
Provides server default configuration values to clients.
Provide command line access to a FileSystem.
This class is the home for file permissions related commands.
Used to change group of files
The pattern is almost as flexible as mode allowed by chmod shell command.
Used to change owner and/or group of files
This class is used to represent the capacity, free and used space on a
FileSystem.Holds the HTrace Tracer used for FileSystem operations.
Factory for URL stream handlers.
This class contains constants for configuration keys used
in the ftp file system.
A class to wrap a
Throwable into a Runtime Exception.
A
FileSystem backed by an FTP client provided by Apache Commons Net.The FtpFs implementation of AbstractFileSystem.
Function of arity 4 which may raise an IOException.
Functional utilities for IO operations.
Function of arity 1 which may raise an IOException.
Deprecated.
use
org.apache.hadoop.util.functionalDeprecated.
Deprecated.
Deprecated.
Builder for input streams and subclasses whose return value is
actually a completable future: this allows for better asynchronous
operation.
Builder for input streams and subclasses whose return value is
actually a completable future: this allows for better asynchronous
operation.
Future IO Helper methods.
Deprecated.
Implementation of Galois field arithmetic with 2^p elements.
This code supports Ganglia 3.0
This code supports Ganglia 3.1
This class monitors the percentage of time the JVM is paused in GC within
the specified observation window, say 1 minute.
Encapsulates data about GC pauses measured at the specific timestamp.
The user can provide an instance of a class implementing this interface
when initializing a GcTimeMonitor to receive alerts when GC time
percentage exceeds the specified threshold.
GenericOptionsParser is a utility to parse command line
arguments generic to the Hadoop framework.Protocol which is used to refresh arbitrary things at runtime.
Contains utility methods for dealing with Java Generics.
A wrapper for Writable instances.
Base class for the HDFS and MR implementations of tools which fetch and
display the groups that users belong to.
The builder class
Protocol implemented by the Name Node and Job Tracker which maps users to
groups.
A GaloisField utility class only caring of 256 fields for efficiency.
Stores global storage statistics objects.
A callback API for creating new StorageStatistics instances.
Implementation of
FileSystem.globStatus(Path, PathFilter).Builder for Globber instances.
A filter for POSIX glob pattern with brace expansions.
A glob pattern filter for metrics.
A class for POSIX glob pattern with brace expansions.
A metrics sink that writes to a Graphite server.
An interface for the implementation of a user-to-groups mapping service
used by
Groups.A user-to-groups mapping service.
A
GSet is set,
which supports the GSet.get(Object) operation.This class creates gzip compressors/decompressors.
A command-line tool for making calls in the HAServiceProtocol.
UsageInfo class holds args and help details.
Factory methods for ExecutorService, ScheduledExecutorService instances.
Indicates that a method has been passed illegal or invalid argument.
This class implements parsing and handling of Kerberos principal names.
An extension of ScheduledThreadPoolExecutor that provides additional
functionality.
An extension of ThreadPoolExecutor that provides additional functionality.
This class is intended to be installed by calling
Thread.setDefaultUncaughtExceptionHandler(UncaughtExceptionHandler)
in the main entry point.Class for creating hardlinks.
HardLink statistics counters and methods.
This is an implementation of the Hadoop Archive
Filesystem.
FSDataInputStreams implement this interface to provide enhanced
byte buffer access.
Protocol interface that provides High Availability related primitives to
monitor and fail-over the service.
An HA service may be in active or standby state.
Information describing the source for a request to change state.
This class is the client side translator to translate the requests made on
HAServiceProtocol interfaces to the RPC server implementing
HAServiceProtocolPB.Helper for making
HAServiceProtocol RPC calls.This class is used on the server side.
Represents a target of the client side HA administration commands.
Having a FileDescriptor
This class represents a common API for hashing functions.
Implements a hash object that returns a certain number of hashed values.
An implementation for Apache Hadoop compatible file system based mount-table
file loading.
Exception thrown to indicate that health check of a service failed.
This class is a daemon which runs in a loop, periodically heartbeating
with an HA service.
An implementation of the core algorithm of HeapSort.
Abstract class for Hitchhiker common facilities shared by
HHXORErasureEncodingStepand HHXORErasureDecodingStep.Some utilities for Hitchhiker coding.
A Hitchhiker-XOR erasure codec.
Hitchhiker is a new erasure coding algorithm developed as a research project
at UC Berkeley by Rashmi Vinayak.
Hitchhiker-XOR Erasure decoding step, a wrapper of all the necessary
information to perform a decoding step involved in the whole process of
decoding a block group.
Hitchhiker is a new erasure coding algorithm developed as a research project
at UC Berkeley by Rashmi Vinayak.
Hitchhiker-XOR Erasure encoding step, a wrapper of all the necessary
information to perform an encoding step involved in the whole process of
encoding a block group.
An atomic view of the included and excluded hosts
This class is responsible for quoting HTML characters.
Singleton to get access to Http related configuration.
HTTP utility class to help propagate server side exception to the client
over HTTP as a JSON payload.
A Filesystem that reads from HTTP endpoint.
Contains all the logic for generating an HTTP "Referer"
entry; includes escaping query params.
Builder.
RequestLog object for use with Http
Create a Jetty embedded server to answer http requests.
Class to construct instances of HTTP server with specific options.
A Servlet input filter that quotes all HTML active characters in the
parameter names and values.
A very simple servlet to serve up a text representation of the current
stack traces.
The X-FRAME-OPTIONS header in HTTP response to mitigate clickjacking
attack.
This class collects all the metrics of Jetty's StatisticsHandler
and expose them as Hadoop Metrics.
A Filesystem that reads from HTTPS endpoint.
Util class that returns the remote
UserGroupInformation in scope
for the HTTP request.Used to mark certain methods of an interface as being idempotent, and
therefore warrant being retried on failover.
The IdentityHashStore stores (key, value) mappings in an array.
The IdentityProvider creates identities for each schedulable
by extracting fields and returning an identity string.
Generic ID generator
used for generating various types of number sequences.
Some constants for IdMapping
An interface for the implementation of <userId,
userName> mapping and <groupId, groupName>
mapping.
Interface for collections capable of being sorted by
IndexedSorter
algorithms.Interface for sort algorithms accepting
IndexedSortable items.An implementation of SaslPropertiesResolver.
InnerNode represents a switch/router of a data center or rack.
InodeTree implements a mount-table as a tree of inodes.
An internal class to represent a mount link.
Resolve returns ResolveResult.
A reusable
InputStream implementation that reads from an in-memory
buffer.This is a debugging class that can be used by callers to track
whether a specific lock is being held for too long and periodically
log a warning and stack trace, if so.
Immutable class to capture a snapshot of suppressed log message stats.
This is a wrap class of a
ReadLock.This is a wrap class of a
ReentrantReadWriteLock.This is a wrap class of a
WriteLock.This method allows access to Package-scoped operations from classes
in org.apache.hadoop.fs.impl and other file system implementations
in the hadoop modules.
Helpers to create interned metrics info.
Handles interrupts by shutting down a service, escalating if the service
does not shut down in time, or when other interrupts are received.
Forced shutdown runnable.
Implements an intrusive doubly-linked list.
An element contained in this list.
A WritableComparable for ints.
A Comparator optimized for IntWritable.
Thrown when bytesPerChecksun field in the meta file is less than
or equal to 0 or type is invalid.
Thrown for invalid decoding.
Path string is invalid either because it has invalid characters or due to
other file system specific reasons.
Thrown when the constraints enoded in a
PathHandle do not hold.Thrown when the user makes a malformed request, for example missing required
parameters or parameters that are not valid.
This is a lambda-expression which may raises an IOException.
IO Statistics.
Interface exported by classes which support
aggregation of
IOStatistics.Support for implementing IOStatistics interfaces.
An interface defined to capture thread-level IOStatistics by using per
thread context.
Implementing the IOStatisticsContext.
A Utility class for IOStatisticsContext, which helps in creating and
getting the current active context.
Utility operations convert IO Statistics sources/instances
to strings, especially for robustly logging.
Setter for IOStatistics entries.
Snapshot of statistics from a different source.
A source of IO statistics.
Interface of an IOStatistics store intended for
use in classes which track statistics for reporting.
Builder of the
IOStatisticsStore implementation.Support for working with IOStatistics.
An utility class for I/O related functionality.
The /dev/null of OutputStreams.
IPC exception is thrown by IPC layer when the IPC
connection cannot be established.
Handler of interrupts that relays them to a registered
implementation of
IrqHandler.Interrupted.Interrupt data to pass on.
Callback issues on an interrupt.
Used by Load Balancers to detect the active NameNode/ResourceManager/Router.
KeyProvider based on Java's KeyStore file format.
CredentialProvider based on Java's KeyStore file format.
The factory to create JksProviders, which is used by the ServiceLoader.
The factory to create JksProviders, which is used by the ServiceLoader.
An adapter between a KeyStore Key and our Metadata.
An experimental
Serialization for Java Serializable classes.
A
RawComparator that uses a JavaSerialization
Deserializer to deserialize objects that are then compared via
their Comparable interfaces.Implement the AES-CTR crypto codec using JCE provider.
Implement the SM4-CTR crypto codec using JCE provider.
Produces 32-bit hash for hash table lookup.
Contains utility methods and constants relating to Jetty.
Provides Read only web access to JMX.
For example in case of MutableGauge we are using numbers,
but not implementing Number interface,
so we skip class check here because we can not be sure NaN values are wrapped
with classes which implements the Number interface
A JNI-based implementation of
GroupMappingServiceProvider
that invokes libC calls to get the group
memberships of a given user.A JNI-based implementation of
GroupMappingServiceProvider
that invokes libC calls to get the group
memberships of a given user.Support for marshalling objects to and from JSON.
JVM and logging related metrics.
JVM and logging related metrics info instances
Class which sets up a simple thread which runs in a loop sleeping
for a short interval of time.
Kerberos diagnostics
This operation expands some of the diagnostic output of the security code,
but not all.
Diagnostics failures return the exit code 41, "unauthorized".
Thrown when
UserGroupInformation failed with an unrecoverable error,
such as failure in kerberos login/logout, invalid subject etc.An
AuthenticationHandler that implements Kerberos SPNEGO mechanism
for HTTP and supports Delegation Token functionality.The
KerberosDelegationTokenAuthenticator provides support for
Kerberos SPNEGO authentication mechanism and support for Hadoop Delegation
Token operations.Indicates Kerberos related information to be used
The general behavior of a key that must be stored in a filter.
A provider of secret key material for Hadoop applications.
The combination of both the key version name and the key material.
Key metadata that is associated with the key.
Options when creating key objects.
A KeyProvider with Cryptographic Extensions specifically for generating
and decrypting encrypted encryption keys.
CryptoExtension is a type of Extension that exposes methods to generate
EncryptedKeys and to decrypt the same.
An encrypted encryption key (EEK) and related information.
A KeyProvider extension with the ability to add a renewer's Delegation
Tokens to the provided Credentials.
DelegationTokenExtension is a type of Extension that exposes methods
needed to work with Delegation Tokens.
This is a utility class used to extend the functionality of KeyProvider, that
takes a KeyProvider and an Extension.
A marker interface for the KeyProviderExtension subclass implement.
A factory to create a list of KeyProvider based on the path given in a
Configuration.
File systems that support Encryption Zones have to implement this interface.
This program is the CLI utility for the KeyProvider facilities in Hadoop.
CredentialProvider based on Java Key Store API.
Interface that gives access to
KeyManager and TrustManager
implementations.KMS client
KeyProvider implementation.The factory to create KMSClientProvider, which is used by the
ServiceLoader.
The KMS implementation of
TokenRenewer.Holder class for KMS delegation tokens.
DelegationTokenIdentifier used for the KMS.
KMS REST and JSON constants and utility methods for the KMSServer.
Utils for KMS.
Lambda-expression utilities be they generic or specific to
Hadoop datatypes.
An interface which services can implement to have their
execution managed by the ServiceLauncher.
Standard launcher arguments.
Common Exit codes.
A lazily constructed reference, whose reference
constructor is a
CallableRaisingIOE so
may raise IOExceptions.A subclass of
LazyAtomicReference which
holds an AutoCloseable reference and calls close()
when it itself is closed.An implementation of
GroupMappingServiceProvider which
connects directly to an LDAP server for determining group membership.An private internal socket factory used to create SSL sockets with custom
configuration.
A class to report leaks of streams.
Whether the given Path of the FileSystem has the capability to perform lease recovery.
A serializable lifecycle event: the time a state
transition occurred, and what state was entered.
A low memory footprint Cache which extends
LightWeightGSet.Entries of
LightWeightCache.A low memory footprint
GSet implementation,
which uses an array for storing the elements
and linked lists for collision resolution.Elements of
LightWeightGSet.A low memory footprint
GSet implementation,
which uses an array for storing the elements
and linked lists for collision resolution.Copied from guava source code v15 (LimitedInputStream)
Guava deprecated LimitInputStream in v14 and removed it in v15.
A class that provides a line reader from an input stream.
Static utility methods pertaining to
List instances.A simple LoadBalancing KMSClientProvider that round-robins requests
across a provided array of KMSClientProviders.
CredentialProvider based on bouncy castle FIPS KeyStore file format.
The factory to create KeyStore Providers, which is used by the
ServiceLoader.
This class contains constants for configuration keys used
in the local file system, raw local fs and checksum fs.
An implementation of a round-robin scheme for disk allocation for creating
files.
Implement the FileSystem API for the checksumed local filesystem.
This class contains constants for configuration keys used
in the local file system, raw local fs and checksum fs.
Opaque handle to an entity in a FileSystem.
The LocalFs implementation of ChecksumFs.
CredentialProvider based on Java's KeyStore file format.
The factory to create JksProviders, which is used by the ServiceLoader.
CredentialProvider based on Java's KeyStore file format.
This class defines a FileStatus that includes a file's block locations.
Log exactly once, even across threads.
This is a state change listener that logs events at INFO level
Change log level in runtime.
A servlet implementation
This is a class to help easily throttle log statements, so that they will
not be emitted more frequently than a certain rate.
An indication of what action the caller should take.
A WritableComparable for longs.
A Comparator optimized for LongWritable.
A decreasing Comparator optimized for LongWritable.
A dummy invocation handler extending RetryInvocationHandler.
This class creates lz4 compressors/decompressors.
A
Compressor based on the lz4 compression algorithm.A
Decompressor based on the lz4 compression algorithm.Container class which holds a list of ip/host addresses and
answers membership queries.
InetAddressFactory is used to obtain InetAddress from host.
A file-based map from keys to values.
Class to merge multiple MapFiles of same Key and Value types to one MapFile
Provide access to an existing map.
Writes a new map.
A Writable Map.
This util class provides a method to register an MBean using
our standard naming convention as described in the doc
for {link
MBeans.register(String, String, Object).A Writable for MD5 hash values.
A WritableComparator optimized for MD5Hash keys.
For CRC32 with the Castagnoli polynomial
MD5 of MD5 of CRC32.
For CRC32 with the Gzip polynomial
A mean statistic represented as the sum and the sample count;
the mean is calculated on demand.
An implementation of the core algorithm of MergeSort.
Exception - Meta Block with the same name already exists.
Exception - No such Meta Block with the given name.
Annotation interface for a single metric used to annotate a field or a method
in the class.
Annotation interface for a group of metrics
Utility class to simplify creation of hadoop metrics2 source/sink.
A pair of a name and its corresponding value.
A fixed-size priority queue, used to retrieve top-n of offered entries.
Metrics annotation helpers.
A metrics cache for sinks that don't support sparse updates.
Cached record
The metrics collector interface
A general metrics exception wrapper
The metrics filter interface.
Interface to provide immutable metainfo for metrics.
Build a JSON dump of the metrics.
The plugin interface for the metrics framework
An immutable snapshot of metrics with a timestamp
The metrics record builder interface
An optional metrics registry class for creating and maintaining a
collection of MetricsMutables, making writing metrics source easier.
The metrics sink interface.
The source of metrics information.
Helper class to build
MetricsSource object from annotations.The metrics system interface.
Convenient abstract class for implementing callback interface
The metrics system callback interface (needed for proxies.)
A base class for metrics system singletons
The JMX interface to the metrics system
Immutable tag for metrics (for grouping on host/queue/username etc.)
Build a string dump of the metrics.
A visitor interface for metrics
An interface for loading mount-table configuration.
Metrics system related metrics info instances
Holder class that clients can use to return multiple exceptions.
MultipartUploader is an interface for copying files multipart and across
multiple nodes.
Builder interface for Multipart readers.
Builder for
MultipartUploader implementations.Encapsulate a list of
IOException into an IOExceptionBuild an
IOException using MultipleIOException
if there are more than one.A
CompositeAuthenticationHandler that supports multiple HTTP
authentication schemes along with Delegation Token functionality. e.g.This is a very fast, non-cryptographic hash suitable for general hash-based
lookup.
The mutable counter (monotonically increasing) metric interface
A mutable int counter for implementing metrics sources
A mutable long counter
The mutable gauge metric interface
A mutable float gauge.
A mutable int gauge
A mutable long gauge
Watches a stream of long values, maintaining online estimates of specific
quantiles with provably low error bounds.
The mutable metric interface
Watches a stream of long values, maintaining online estimates of specific
quantiles with provably low error bounds.
A convenient mutable metric for throughput measurement
Helper class to manage a group of mutable rate metrics
This class synchronizes all accesses to the metrics it
contains, so it should not be used in situations where
there is high contention on the metrics.
Helper class to manage a group of mutable rate metrics.
This class maintains a group of rolling average metrics.
A mutable metric with stats.
A helper to load the native hadoop code i.e. libhadoop.so.
JNI wrappers for various native IO-related calls not available in Java.
Used to manipulate the operating system cache.
A CacheManipulator used for testing which does not actually call mlock.
JNI wrapper of persist memory operations.
Denote memory region for a file mapped.
Result type of the fstat call
Keeps the support state of PMDK.
Supported list of Windows access right flags
An exception generated by a call to the native IO code.
A Reed-Solomon raw decoder using Intel ISA-L library.
A Reed-Solomon raw encoder using Intel ISA-L library.
A raw coder factory for raw Reed-Solomon coder in native using Intel ISA-L.
A XOR raw decoder using Intel ISA-L library.
A XOR raw encoder using Intel ISA-L library.
A raw coder factory for xor coder in native using Intel ISA-L library.
Class that caches the netgroups and inverts group-to-user map
to user-to-group map, primarily intended for use with
netgroups (as returned by getent netgrgoup) which only returns
group to user mapping.
The class represents a cluster of computer with a tree hierarchical
network topology.
The class extends NetworkTopology to represents a cluster of computer with
a 4-layers hierarchical network topology.
The interface defines a node in a network topology.
A base class that implements interface Node
This class parses the configured list of fencing methods, and
is responsible for trying each one in turn while logging informative
output.
NotInMountpointException extends the UnsupportedOperationException.
This class provides groups mapping for
UserGroupInformation when the
user group information will not be used.Singleton Writable with no data.
A Comparator "optimized" for NullWritable.
A polymorphic Writable that writes an instance with it's class name.
Thrown by a remote ObserverNode indicating the operation has failed and the
client should retry active namenode directly (instead of retry other
ObserverNodes).
All the parameters from the openFile builder for the
openFileWithOptions commands.Implement the AES-CTR crypto codec using JNI into OpenSSL.
OpenSSL cipher using JNI.
OpenSSL secure random using JNI.
Implement the SM4-CTR crypto codec using JNI into OpenSSL.
Little duration counter.
This class contains options related to file system operations.
This class allows generic access to variable length type-safe parameter
lists.
Enum for indicating what mode to use when combining chunk and block
checksums to define an aggregate FileChecksum.
This is used in FileSystem and FileContext to specify checksum options.
The standard
createFile() options.Class to support the varargs for create() options.
This is not needed if ChecksumParam is specified.
Options for creating
PathHandle references.Option storing standard constraints on data.
Option storing standard constraints on location.
The standard
openFile() options.Enum to support the varargs for rename() options
A Random implementation that uses random bytes sourced from the
operating system.
A reusable
OutputStream implementation that writes to an in-memory
buffer.Indicates that the parent of specified Path is not a directory
as expected.
Opaque, serializable reference to a part id for multipart uploads.
A partial listing of the children of a parent directory.
This is a special codec which does not transform the output.
The decompressor.
The decompressor is a no-op.
Names a file or directory in a
FileSystem.EACCES
The Path counterpoint to
StreamCapabilities; a query to see if,
a FileSystem/FileContext instance has a specific capability under the given
path.Encapsulates a Path (path), its FileStatus (stat), and its FileSystem (fs).
Exception corresponding to File Exists - EEXISTS
Opaque, serializable reference to an entity in the FileSystem.
Exceptions based on standard posix/linux style exceptions for path related
errors.
EISDIR
ENOTDIR
Generated by rm commands
Exception corresponding to path not found: ENOENT/ENOFILE
ENOTSUP
Exception corresponding to Operation Not Permitted - EPERM
Utility methods aiding conversion of fs data structures.
Store permission related information.
Represents a mapping entry for in the Portmap service for binding RPC
protocols.
Helper utility for building portmap request
Helper utility for sending portmap response.
Stream that permits positional reading.
Fencer method that uses PowerShell to remotely connect to a machine and kill
the required process.
This class replaces
guava.Preconditions which provides helpers
to validate the following conditions:
An invalid null obj causes a NullPointerException.
An invalid argument causes an IllegalArgumentException.
An invalid state causes an IllegalStateException.
An invalid index causes an IndexOutOfBoundsException.
Constants used by prefetch implementations.
A micro-application that prints the main class name out of a jar file.
A PriorityQueue maintains a partial ordering of its elements such that the
least element can always be found in constant time.
Stores the times that a call takes to be processed through each step and
its response status.
The different stages to track the time of.
Process related utilities.
Servlet to serve files generated by
ProfileServlet.Servlet for disabled async-profiler.
Servlet that runs async-profiler as web-endpoint.
A driver that is used to run programs added to it
Utility to assist with generation of progress reports.
A facility for reporting progress.
Metrics sink for prometheus exporter.
Servlet to publish hadoop metrics in prometheus format.
Deprecated.
hadoop code MUST use
ShadedProtobufHelper.Deprecated.
Server implementation is always ProtobufRpcEngine2 based implementation,
supports backward compatibility for protobuf 2.5 based implementations,
which uses non-shaded protobuf classes.
RPC Engine for for protobuf based RPCs.
Deprecated.
A RpcWritable wrapper for unshaded protobuf messages.
The protocol name that is used when a client and server connect.
Protocol to get versions and signatures for supported protocols from the
server.
This class serves the requests for protocol versions and signatures by
looking them up in the server registry.
This interface is implemented by the client side translators and can be used
to obtain information about underlying protocol e.g. to check if a method is
supported on the server side.
a class wraps around a server's proxy,
containing a list of its supported methods.
An interface implemented by client-side protocol translators to get the
underlying proxy object the translator is operating on.
Utility methods for both key and credential provider APIs.
A utility class used to combine two protocol proxies.
AuthenticationFilter which adds support to perform operations
using end user instead of proxy user.
Filter initializer to initialize
ProxyUserAuthenticationFilter which adds support
to perform operations using end user instead of proxy user.An
AuthenticationHandler that implements Kerberos SPNEGO mechanism
for HTTP and supports Delegation Token functionality.The
PseudoDelegationTokenAuthenticator provides support for
Hadoop's pseudo authentication mechanism that accepts
the user name specified as a query string parameter and support for Hadoop
Delegation Token operations.A pure-java implementation of the CRC32 checksum that uses
the same polynomial as the built-in native CRC32.
A pure-java implementation of the CRC32 checksum that uses
the CRC32-C polynomial, the same polynomial used by iSCSI
and implemented on many Intel chipsets supporting SSE4.2.
Specifies a quantile (with error bounds) to be watched by a
SampleQuantiles object.An implementation of the core algorithm of QuickSort.
Store the quota usage of a directory.
Builder class for QuotaUsage.
Minimal subset of google rate limiter class.
Factory for Rate Limiting.
Interface for objects that can be compared through
RawComparator.
A
Comparator that operates directly on byte representations of
objects.Raw erasure coder factory that can be used to create raw encoder and decoder.
An abstract raw erasure decoder that's to be inherited by new decoders.
An abstract raw erasure encoder that's to be inherited by new encoders.
Implement the FileSystem API for the raw local filesystem.
The RawLocalFs implementation of AbstractFileSystem.
Generic format of FileStatus objects.
Manages a pool of threads which can issue readahead requests on file descriptors.
An outstanding readahead request that has been submitted to
the pool.
Options that can be used when reading from a FileSystem.
ReadWriteDiskValidator is the class to check a directory by to create a file,
write some bytes into it, read back, and verify if they are identical.
The metrics for a directory generated by
ReadWriteDiskValidator.Something whose
Configuration can be changed at run time.Utility base class for implementing the Reconfigurable interface.
Exception indicating that configuration property cannot be changed
at run time.
A servlet for changing a node's configuration.
General reflection utils
Protocol which is used to refresh the authorization policy in use currently.
Protocol which is used to refresh the call queue in use currently.
Used to registry custom methods to refresh at runtime.
Used to registry custom methods to refresh at runtime.
Return a response in the handler method for the user to see.
Protocol use
A regex pattern filter for metrics
RegexMountPointInterceptorType.
A simple client that registers an RPC program with portmap.
An implementation of
X509KeyManager that exposes a method,
ReloadingX509KeystoreManager.loadFrom(Path) to reload its configuration.A
TrustManager implementation that exposes a method, ReloadingX509TrustManager.loadFrom(Path)
to reload its configuration for example when the truststore file on disk changes.An iterator over a collection whose elements need to be fetched remotely
A set of remote iterators supporting transformation and filtering,
with IOStatisticsSource passthrough, and of conversions of
the iterators to lists/arrays and of performing actions
on the values.
Wrapper of another remote iterator; IOStatistics
and Closeable methods are passed down if implemented.
Defines the different remove scheme for retouched Bloom filters.
Manages a fixed pool of resources.
This filter provides protection against cross site request forgery (CSRF)
attacks for REST APIs.
Defines the minimal API requirements for the filter to execute its
filtering logic.
Implements a retouched Bloom filter, as defined in the CoNEXT 2006 paper.
Exception thrown by a server typically to indicate that server is in a state
where request cannot be processed temporarily (such as still starting up).
Maintains a cache of non-idempotent requests that have been successfully
processed by the RPC server implementation, to handle the retries.
CacheEntry is tracked using unique client ID and callId of the RPC request.
CacheEntry with payload that tracks the previous response or parts of
previous response to be used for generating response for retried requests.
This class is for maintaining the various RetryCache-related statistics
and publishing them through the metrics interfaces.
Provides retry related functionality.
A
RpcInvocationHandler which supports client side retry .
A collection of useful implementations of
RetryPolicy.Given pairs of number of retries and sleep time (n0, t0), (n1, t1), ...,
the first n0 retries sleep t0 milliseconds on average,
the following n1 retries sleep t1 milliseconds on average, and so on.
Pairs of numRetries and sleepSeconds
Specifies a policy for retrying method failures.
A factory for creating retry proxies.
This class is a metrics sink that uses
FileSystem to write the metrics logs.A simple RPC mechanism.
Class to construct instances of RPC server with specific options.
An RPC Server.
A version mismatch for the RPC protocol.
Represents RPC message MSG_ACCEPTED reply body.
Authentication Info.
Different types of authentication as defined in RFC 1831
Represents an RPC message of type RPC call as defined in RFC 1831
This class is used for handling the duplicate non-idempotenty Rpc
calls.
Call that is used to track a client in the
RpcCallCacheIndicates an exception in the RPC client
This class maintains a cache of protocol versions and corresponding protocol
signatures, keyed by server address, protocol and rpc kind.
Represents RPC message MSG_DENIED reply body.
This class is for maintaining RPC method related statistics
and publishing them through the metrics interfaces.
An RPC implementation.
Indicates an exception during the execution of remote procedure call.
RpcInfo records all contextual information of an RPC message.
This interface must be implemented by all InvocationHandler
implementations.
Represent an RPC message as defined in RFC 1831.
Message type
This class is for maintaining the various RPC statistics
and publishing them through the metrics interfaces.
Implement this interface to make a pluggable multiplexer in the
FairCallQueue.
No such Method for an Rpc Call
No such protocol (i.e. interface) for and Rpc Call
Class for writing RPC server programs based on RFC 1050.
Represents an RPC message of type RPC reply as defined in RFC 1831
RPC reply_stat as defined in RFC 1831
RpcResponse encapsulates a response to a RPC request.
Implement this interface to be used for RPC scheduling and backoff.
Indicates an exception on the RPC server
adapter to allow decoding of writables and protobufs from a byte buffer.
A Reed-Solomon erasure codec.
Reed-Solomon erasure decoder that decodes a block group.
Reed-Solomon erasure encoder that encodes a block group.
A raw erasure decoder in RS code scheme in pure Java in case native one
isn't available in some environment.
A raw erasure encoder in RS code scheme in pure Java in case native one
isn't available in some environment.
A raw coder factory for the legacy raw Reed-Solomon coder in Java.
A raw erasure decoder in RS code scheme in pure Java in case native one
isn't available in some environment.
A raw erasure encoder in RS code scheme in pure Java in case native one
isn't available in some environment.
A raw coder factory for the new raw Reed-Solomon coder in Java.
Utilities for implementing Reed-Solomon code, used by RS coder.
This class uses
LdapGroupsMapping for group lookup and applies the
rule configured on the group names.Run a Hadoop job jar.
Runnable interface whose
RunnableRaisingIOE.apply() method may raise
an IOE.Whether the given filesystem is in any status of safe mode.
An identical copy from org.apache.hadoop.hdfs.protocol.HdfsConstants.SafeModeAction, that helps
the other file system implementation to define
SafeMode.Implementation of the Cormode, Korn, Muthukrishnan, and Srivastava algorithm
for streaming calculation of targeted high-percentile epsilon-approximate
quantiles.
Helper to compute running sample stats
Helper to keep running min/max
A SaslInputStream is composed of an InputStream and a SaslServer (or
SaslClient) so that read() methods return data that are read in from the
underlying InputStream but have been additionally processed by the SaslServer
(or SaslClient) object.
SASL related constants.
A SaslOutputStream is composed of an OutputStream and a SaslServer (or
SaslClient) so that write() methods first process the data before writing
them out to the underlying OutputStream.
Provides SaslProperties to be used for a connection.
A utility class that encapsulates SASL logic for RPC client
A utility class for dealing with SASL on RPC server
Authentication method
CallbackHandler for SASL mechanism.
CallbackHandler for SASL GSSAPI Kerberos mechanism
Interface which allows extracting information necessary to
create schedulable identity strings.
Groups a list of ACL entries into separate lists for access entries vs.
This class implements the
DNSToSwitchMapping interface using a
script configured via the
CommonConfigurationKeysPublic.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY option.This is the uncached script mapping that is fed into the cache managed
by the superclass
CachedDNSToSwitchMappingThis class extends ScriptBasedMapping class and implements
the
DNSToSwitchMappingWithDependency interface using
a script configured via the
CommonConfigurationKeysPublic.NET_DEPENDENCY_SCRIPT_FILE_NAME_KEY option.The server-side secret manager for each token type.
The token was invalid and the message explains why.
This class provides secure APIs for opening and creating files on the local
disk.
Signals that an attempt to create a file at a given pathname has failed
because another file already existed at that path.
Security Utils.
This an alternate resolver with important properties that the standard
java resolver lacks:
1) The hostname is fully qualified.
Helper class to contain the Truststore/Keystore paths for the ZK client connection over
SSL/TLS.
Stream that permits seeking.
This ExecutorService blocks the submission of new tasks when its queue is
already full by using a semaphore.
SequenceFiles are flat files consisting of binary key/value
pairs.The compression type used to compress key/value pairs in the
SequenceFile.The class encapsulating with the metadata of a file.
Reads key/value pairs from a sequence-format file.
A tag interface for all of the Reader options
Sorts key/value pairs in a sequence-format file.
The interface to iterate over raw keys/values of SequenceFiles.
The interface to 'raw' values of SequenceFiles.
Write key/value pairs to a sequence-format file.
Sequential number generator.
Encapsulates a
Serializer/Deserializer pair.
A factory for
Serializations.
Provides a facility for serializing objects of type <T> to an
OutputStream.An abstract IPC service.
A generic call queued for handling.
Helpers to handle server addresses
An abstract definition of service as related to
Service Level Authorization for Hadoop.
Service LifeCycle.
Service states
An authorization manager which handles service-level authorization
for incoming service requests.
Exception thrown to indicate that an operation performed
to modify the state of a service or application failed.
A class to launch any YARN service by name.
A generic options parser which does not parse any of the traditional
Hadoop options.
A service launch exception that includes an exit code.
This class contains a set of methods to work with services, especially
to walk them through their lifecycle.
Class to manage a list of
ServiceStateChangeListener instances,
including a notification loop that is robust against changes to the list
during the notification process.Service plug-in interface.
JVM Shutdown hook for Service which will stop the
Service gracefully in case of JVM shutdown.
Interface to notify state changes of a service.
Exception that can be raised on state change operations, whose
exit code can be explicitly set, determined from that of any nested
cause, or a default value of
LauncherExitCodes.EXIT_SERVICE_LIFECYCLE_EXCEPTION.Implements the service state model.
A file-based set of keys.
Provide access to an existing set file.
Write a new set file.
Static utility methods pertaining to
Set instances.SFTP FileSystem.
Helper methods for protobuf related RPC implementation using the
hadoop
org.apache.hadoop.thirdparty.protobuf shaded version.A factory for creating shared file descriptors inside a given directory.
A base class for running a Shell command.
This is an IOException with exit code added.
A simple shell command executor.
A simple shell-based implementation of
IdMappingServiceProvider
Map id to user name or group name.A simple shell-based implementation of
GroupMappingServiceProvider
that exec's the groups shell command to fetch the group
memberships of a given user.A simple shell-based implementation of
GroupMappingServiceProvider
that exec's the groups shell command to fetch the group
memberships of a given user.Fencing method that runs a shell command.
A WritableComparable for shorts.
A Comparator optimized for ShortWritable.
The
ShutdownHookManager enables running shutdownHook
in a deterministic order, higher priority first.Helper class to shutdown
Threads and ExecutorServices.This class logs a message whenever we're about to exit on a UNIX signal.
A simplified BufferedOutputStream with borrowed buffer, and allow users to
see how much data have been buffered.
A simple TCP based RPC client which just sends a request to a server.
A simple TCP based RPC client handler used by
SimpleTcpServer.Simple UDP server implemented using netty.
A simple UDP based RPC client which just sends one request to a server.
Simple UDP server implemented based on netty.
Provides functionality necessary for caching blocks of data read from FileSystem.
Sizes of binary values and other some common sizes.
This class creates snappy compressors/decompressors.
A
Compressor based on the snappy compression algorithm.A
Decompressor based on the snappy compression algorithm.This implements an input stream that can have a timeout while reading.
A wrapper stream around a socket which allows setting of its timeout.
This implements an output stream that can have a timeout while writing.
Specialized SocketFactory to create sockets with a SOCKS proxy
A Writable SortedMap.
Wrap a statistics instance with an
IOStatisticsSource
instance which will then serve up the statistics when asked.An InputStream covering a range of compressed data.
This interface is meant to be implemented by those compression codecs
which are capable to compress / de-compress a stream starting at any
arbitrary position.
During decompression, data can be read off from the decompressor in two
modes, namely continuous and blocked.
An implementation of
AbstractDelegationTokenSecretManager that
persists TokenIdentifiers and DelegationKeys in an existing SQL database.This fencing implementation sshes to the target node and uses
fuser to kill the process listening on the service's
TCP port.Factory that creates SSLEngine and SSLSocketFactory instances using
Hadoop configuration information.
Copied from the not-yet-commons-ssl project at
http://juliusdavies.ca/commons-ssl/
This project is not yet in Apache, but it is Apache 2.0 licensed.
Specialized SocketFactory to create sockets with a SOCKS proxy
Thrown by a remote server when it is up, but is not the active server in a
set of servers in which only a subset may be active.
Wrapper for the Unix stat(1) command.
Provides a servlet filter that pretends to authenticate a fake user (Dr.Who)
so that the web UI is usable for a secure cluster without authentication.
Track the duration of an object.
A metrics sink that writes metrics to a StatsD daemon.
Class that sends UDP packets to StatsD daemon.
A simplified StopWatch implementation which can measure times in nanoseconds.
A class that contains the numeric value and the unit of measure.
StorageStatistics contains statistics data for a FileSystem or FileContext
instance.
These are common statistic names.
A 64-bit storage statistic.
Returns all the counters of an IOStatistics instance as StorageStatistics.
Defines the types of supported storage media.
Class that maintains different forms of Storage Units.
Utility classes to help implementing filesystems and streams.
Common statistic names for object store operations..
Interface to query streams for supported capabilities.
Deprecated.
Static methods to implement policies for
StreamCapabilities.These are common statistic names.
Stringifier interface offers two methods to convert an object
to a string representation and restore the object given its
string representation.
Provides string interning utility methods.
General string utils
The traditional binary prefixes, kilo, mega, ..., exa,
which can be represented by a 64-bit integer.
A simple stub duration tracker which can be issued in interfaces
and other places where full duration tracking is not implemented.
This is a stub factory which always returns no-op duration
trackers.
Helper class to restore Subject propagation behavior of threads after the
JEP411/JEP486 changes.
This is the interface for flush/sync operations.
Plugin to calculate resource information on the system.
Plugin to calculate resource information on Linux systems.
Plugin to calculate resource information on Windows systems.
This class implements a "table listing" with column headers.
Simple
DNSToSwitchMapping implementation that reads a 2 column text
file.Utility class for parallel execution, takes closures for the various
actions.
Builder for task execution.
Callback invoked on a failure.
Interface to whatever lets us submit tasks.
Callback invoked to process an item.
This class stores text using standard UTF8 encoding.
A WritableComparator optimized for Text keys.
A TFile is a container of key-value pairs.
TFile Reader.
The TFile Scanner.
TFile Writer.
Utility methods for getting the time and computing intervals.
Utility methods for getting the time and computing intervals.
The client-side form of the token.
A trivial renewer for token kinds that aren't managed.
An identifier that identifies a token, may contain public information
about a token, including its kind (or type).
Indicates Token related information to be used
This is the interface for plugins that handle tokens.
Select token of type T from tokens for use with named service
A tool interface that supports handling of generic command-line options.
A utility to help run
Tools.Unix touch like commands
A UNIX like touch command.
(Re)create zero-length file at the specified path.
A wrapper
ByteBufferPool implementation that tracks whether all allocated buffers
are released.Strack trace of allocation as saved in the tracking map.
Exception raised in
TrackingByteBufferPool.close() if there
was an unreleased buffer.Exception raised in
TrackingByteBufferPool.putBuffer(ByteBuffer) if the
buffer to release was not in the hash map.Provides a trash facility which supports pluggable Trash policies.
This interface is used for implementing different Trash policies.
Provides a trash feature.
Truncates a file to a new size
Tuple support.
A Writable for 2D arrays containing a matrix of instances of a class.
Indicates that the RPC server encountered an undeclared exception from the
service
A StorageStatistics instance which combines the outputs of several other
StorageStatistics instances.
Generates predictable and user-friendly unique names
Thrown when a symbolic link is encountered in a path.
Thrown to indicate that the specific codec is not supported.
File system for a given file system name/scheme is not supported
MultipartUploader for a given file system name/scheme is not supported.
Opaque, serializable reference to an uploadId for multipart uploads.
User and group information for Hadoop.
existing types of authentications' methods
A login module that looks at the Kerberos, Unix, or Windows principal and
adds the corresponding UserName.
The UserIdentityProvider creates uses the username as the
identity.
A KeyProvider factory for UGIs.
A CredentialProvider for UGIs.
Deprecated.
replaced by Text
A WritableComparator optimized for UTF8 keys.
Supporting Utility classes used by TFile, and shared by users of TFile.
A generic Version class.
A superset of Validate class in Apache commons lang3.
A Utility class that maintains a Queue of entries for a given key.
QueueRefiller interface a client must implement to use this class
Policy to decide how many values to return to client when client asks for
"n" values and Queue is empty.
Utility class which implements helper methods used
in vectored IO implementation.
A ByteBufferPool implementation that uses a pair of functions to allocate
and release ByteBuffers; intended for use implementing the VectorIO API
as it makes the pair of functions easier to pass around and use in
existing code.
Base class for verifier.
Verifier mapped to RPCSEC_GSS.
Verifier used by AUTH_NONE.
Superclass of all protocols that use Hadoop RPC.
A base class for Writables that provides version checking.
This class returns build information about Hadoop components.
Thrown by
VersionedWritable.readFields(DataInput) when the
version of an object being read does not match the current implementation
version as returned by VersionedWritable.getVersion().A wrapper class to maven's ComparableVersion class, to comply
with maven's version name string convention
ViewFileSystem (extends the FileSystem interface) implements a client-side
mount table.
MountPoint representation built from the configuration.
This class is extended from the ViewFileSystem for the overloaded
scheme file system.
A class to maintain the target file system and a path to pass to the target
file system.
Utility APIs for ViewFileSystem.
ViewFs (extends the AbstractFileSystem interface) implements a client-side
mount table.
A WritableComparable for integer values stored in variable-length format.
A WritableComparable for longs in a variable-length format.
Represents an object that you can wait for.
Buffer pool implementation which uses weak references to store
buffers in the pool, such that they are garbage collected when
there are no references to the buffer during a gc run.
A map of keys type K to objects of type V which uses weak references,
so does lot leak memory through long-lived references
at the expense of losing references when GC takes place..
A WeakReferenceMap for threads.
A weak referenced metrics source which avoids hanging on to large objects
if somehow they don't get fully closed/cleaned up.
Hadoop DefaultServlet for serving static web content.
Determines which queue to start reading from, occasionally drawing from
low-priority queues in order to prevent starvation.
A
CostProvider that calculates the cost for an operation
as a weighted sum of its processing time values (see
ProcessingDetails).An implementation of the SaslPropertiesResolver.
Class to tell the size of a path on windows.
Filesystems that support EC can implement this interface.
Reflection-friendly access to APIs which are not available in
some of the older Hadoop versions which libraries still
compile against.
Deprecated.
use the
UncheckedIOException directly.]Wrap IOStatistics source with another (dynamic) wrapper.
Reflection-friendly access to IOStatistics APIs.
A serializable object which implements a simple, efficient, serialization
protocol, based on
DataInput and DataOutput.A
Writable which is also Comparable.A Comparator for
WritableComparables.Factories for non-public writables.
A factory for a class of Writable.
Utility to permit renaming of Writable implementation classes without
invalidiating files that contain their class name.
Deprecated.
Deprecated.
A
Serialization for Writables that delegates to
Writable.write(java.io.DataOutput) and
Writable.readFields(java.io.DataInput).The value of
XAttr is byte[], this class is to
covert byte[] to some kind of string representation or convert back.Utility class for building XDR messages based on RFC 4506.
This filter protects webapps from clickjacking attacks that
are possible through use of Frames to embed the resources in another
application and intercept clicks to accomplish nefarious things.
General xml utilities.
A XOR erasure codec.
Xor erasure decoder that decodes a block group.
Xor erasure encoder that encodes a block group.
A raw decoder in XOR code scheme in pure Java, adapted from HDFS-RAID.
A raw encoder in XOR code scheme in pure Java, adapted from HDFS-RAID.
A raw coder factory for raw XOR coder.
Helper class that provides utility methods specific to ZK operations.
An implementation of
AbstractDelegationTokenSecretManager that
persists TokenIdentifiers and DelegationKeys in Zookeeper.Protocol exposed by the ZKFailoverController, allowing for graceful
failover.
Utilities for working with ZooKeeper.
An authentication token passed to ZooKeeper.addAuthInfo
A
Compressor based on the popular
zlib compression algorithm.The type of header for compressed data.
The compression level for zlib library.
The compression level for zlib library.
A
Decompressor based on the popular
zlib compression algorithm.The headers to detect from compressed data.
A collection of factories to create the right
zlib/gzip compressor/decompressor instances.
This class creates zstd compressors/decompressors.
A
Compressor based on the zStandard compression algorithm.A
Decompressor based on the zStandard compression algorithm.A
DirectDecompressor for ZStandard
https://github.com/facebook/zstd.