Package org.apache.hadoop.fs

Class BufferedFSInputStream

java.lang.Object
java.io.InputStream
java.io.FilterInputStream
java.io.BufferedInputStream
org.apache.hadoop.fs.BufferedFSInputStream
All Implemented Interfaces:
Closeable, AutoCloseable, HasFileDescriptor, PositionedReadable, Seekable, IOStatisticsSource, StreamCapabilities
@Private @Unstable public class BufferedFSInputStream extends BufferedInputStream implements Seekable, PositionedReadable, HasFileDescriptor, IOStatisticsSource, StreamCapabilities
A class that optimizes reading from FSInputStream by buffering.

  • Constructor Details

    • BufferedFSInputStream

      public BufferedFSInputStream(FSInputStream in, int size)
      Creates a BufferedFSInputStream with the specified buffer size, and saves its argument, the input stream in, for later use. An internal buffer array of length size is created and stored in buf.
      Parameters:
      in - the underlying input stream.
      size - the buffer size.
      Throws:
      IllegalArgumentException - if size <= 0.

  • Method Details

    • getPos

      public long getPos() throws IOException
      Description copied from interface: Seekable
      Return the current offset from the start of the file
      Specified by:
      getPos in interface Seekable
      Returns:
      offset from the start of the file.
      Throws:
      IOException - raised on errors performing I/O.

    • skip

      public long skip(long n) throws IOException
      Overrides:
      skip in class BufferedInputStream
      Throws:
      IOException

    • seek

      public void seek(long pos) throws IOException
      Description copied from interface: Seekable
      Seek to the given offset from the start of the file. The next read() will be from that location. Can't seek past the end of the file.
      Specified by:
      seek in interface Seekable
      Parameters:
      pos - offset from the start of the file.
      Throws:
      IOException - raised on errors performing I/O.

    • seekToNewSource

      public boolean seekToNewSource(long targetPos) throws IOException
      Description copied from interface: Seekable
      Seeks a different copy of the data. Returns true if found a new source, false otherwise.
      Specified by:
      seekToNewSource in interface Seekable
      Parameters:
      targetPos - target position.
      Returns:
      true if found a new source, false otherwise.
      Throws:
      IOException - raised on errors performing I/O.

    • read

      public int read(long position, byte[] buffer, int offset, int length) throws IOException
      Description copied from interface: PositionedReadable
      Read up to the specified number of bytes, from a given position within a file, and return the number of bytes read. This does not change the current offset of a file, and is thread-safe. Warning: Not all filesystems satisfy the thread-safety requirement.
      Specified by:
      read in interface PositionedReadable
      Parameters:
      position - position within file
      buffer - destination buffer
      offset - offset in the buffer
      length - number of bytes to read
      Returns:
      actual number of bytes read; -1 means "none"
      Throws:
      IOException - IO problems.

    • readFully

      public void readFully(long position, byte[] buffer, int offset, int length) throws IOException
      Description copied from interface: PositionedReadable
      Read the specified number of bytes, from a given position within a file. This does not change the current offset of a file, and is thread-safe. Warning: Not all filesystems satisfy the thread-safety requirement.
      Specified by:
      readFully in interface PositionedReadable
      Parameters:
      position - position within file
      buffer - destination buffer
      offset - offset in the buffer
      length - number of bytes to read
      Throws:
      IOException - IO problems.
      EOFException - the end of the data was reached before the read operation completed

    • readFully

      public void readFully(long position, byte[] buffer) throws IOException
      Description copied from interface: PositionedReadable
      Read number of bytes equal to the length of the buffer, from a given position within a file. This does not change the current offset of a file, and is thread-safe. Warning: Not all filesystems satisfy the thread-safety requirement.
      Specified by:
      readFully in interface PositionedReadable
      Parameters:
      position - position within file
      buffer - destination buffer
      Throws:
      IOException - IO problems.
      EOFException - the end of the data was reached before the read operation completed

    • getFileDescriptor

      public FileDescriptor getFileDescriptor() throws IOException
      Specified by:
      getFileDescriptor in interface HasFileDescriptor
      Returns:
      the FileDescriptor
      Throws:
      IOException - raised on errors performing I/O.

    • hasCapability

      public boolean hasCapability(String capability)
      If the inner stream supports StreamCapabilities, forward the probe to it. Otherwise: return false.
      Specified by:
      hasCapability in interface StreamCapabilities
      Parameters:
      capability - string to query the stream support for.
      Returns:
      true if a capability is known to be supported.

    • getIOStatistics

      public IOStatistics getIOStatistics()
      Description copied from interface: IOStatisticsSource
      Return a statistics instance.

      It is not a requirement that the same instance is returned every time. IOStatisticsSource.

      If the object implementing this is Closeable, this method may return null if invoked on a closed object, even if it returns a valid instance when called earlier.

      Specified by:
      getIOStatistics in interface IOStatisticsSource
      Returns:
      an IOStatistics instance or null

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • minSeekForVectorReads

      public int minSeekForVectorReads()
      Description copied from interface: PositionedReadable
      What is the smallest reasonable seek?
      Specified by:
      minSeekForVectorReads in interface PositionedReadable
      Returns:
      the minimum number of bytes

    • maxReadSizeForVectorReads

      public int maxReadSizeForVectorReads()
      Description copied from interface: PositionedReadable
      What is the largest size that we should group ranges together as?
      Specified by:
      maxReadSizeForVectorReads in interface PositionedReadable
      Returns:
      the number of bytes to read at once

    • readVectored

      public void readVectored(List<? extends FileRange> ranges, IntFunction<ByteBuffer> allocate) throws IOException
      Description copied from interface: PositionedReadable
      Read fully a list of file ranges asynchronously from this file. The default iterates through the ranges to read each synchronously, but the intent is that FSDataInputStream subclasses can make more efficient readers. As a result of the call, each range will have FileRange.setData(CompletableFuture) called with a future that when complete will have a ByteBuffer with the data from the file's range.

      The position returned by getPos() after readVectored() is undefined.

      If a file is changed while the readVectored() operation is in progress, the output is undefined. Some ranges may have old data, some may have new and some may have both.

      While a readVectored() operation is in progress, normal read api calls may block.

      Specified by:
      readVectored in interface PositionedReadable
      Parameters:
      ranges - the byte ranges to read
      allocate - the function to allocate ByteBuffer
      Throws:
      IOException - any IOE.

    • readVectored

      public void readVectored(List<? extends FileRange> ranges, IntFunction<ByteBuffer> allocate, Consumer<ByteBuffer> release) throws IOException
      Description copied from interface: PositionedReadable
      Extension of PositionedReadable.readVectored(List, IntFunction) where a release(buffer) operation may be invoked if problems surface during reads.

      The release operation is invoked after an IOException to return the actively buffer to a pool before reporting a failure in the future.

      The default implementation calls PositionedReadable.readVectored(List, IntFunction).p

      Implementations SHOULD override this method if they can release buffers as part of their error handling.

      Specified by:
      readVectored in interface PositionedReadable
      Parameters:
      ranges - the byte ranges to read
      allocate - function to allocate ByteBuffer
      release - callable to release a ByteBuffer.
      Throws:
      IOException - any IOE.