Package org.apache.hadoop.hdfs.web

Class WebHdfsFileSystem.ReadRunner

java.lang.Object
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.ReadRunner
Enclosing class:
WebHdfsFileSystem
@VisibleForTesting protected class WebHdfsFileSystem.ReadRunner extends Object
This class will allow retries to occur for both open and read operations. The first WebHdfsFileSystem#open creates a new WebHdfsInputStream object, which creates a new ReadRunner object that will be used to open a connection and read or seek into the input stream. ReadRunner is a subclass of the AbstractRunner class, which will run the ReadRunner#getUrl(), ReadRunner#connect(URL), and ReadRunner#getResponse methods within a retry loop, based on the configured retry policy. ReadRunner#connect will create a connection if one has not already been created. Otherwise, it will return the previously created connection object. This is necessary because a new connection should not be created for every read. Likewise, ReadRunner#getUrl will construct a new URL object only if the connection has not previously been established. Otherwise, it will return the previously created URL object. ReadRunner#getResponse will initialize the input stream if it has not already been initialized and read the requested data from the specified input stream.

  • Field Details

  • Method Details

    • close

      public void close() throws IOException
      Throws:
      IOException

    • getUrl

      protected URL getUrl() throws IOException
      Throws:
      IOException

    • connect

      protected HttpURLConnection connect(URL url) throws IOException
      Two-step requests redirected to a DN Create/Append: Step 1) Submit a Http request with neither auto-redirect nor data. Step 2) Submit another Http request with the URL from the Location header with data. The reason of having two-step create/append is for preventing clients to send out the data before the redirect. This issue is addressed by the "Expect: 100-continue" header in HTTP/1.1; see RFC 2616, Section 8.2.3. Unfortunately, there are software library bugs (e.g. Jetty 6 http server and Java 6 http client), which do not correctly implement "Expect: 100-continue". The two-step create/append is a temporary workaround for the software library bugs. Open/Checksum Also implements two-step connects for other operations redirected to a DN such as open and checksum
      Throws:
      IOException

    • getInputStream

      @VisibleForTesting protected InputStream getInputStream()

    • setInputStream

      @VisibleForTesting protected void setInputStream(InputStream inStream)

    • getFileEncryptionInfo

      protected org.apache.hadoop.fs.FileEncryptionInfo getFileEncryptionInfo()

    • updateURLParameters

      protected void updateURLParameters(Param<?,?>... p)