Package org.apache.hadoop.hdfs.server.common

Class Util

java.lang.Object

org.apache.hadoop.hdfs.server.common.Util

@Private
public final class Util
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
static final org.apache.hadoop.hdfs.web.URLConnectionFactory	connectionFactory
static final String	CONTENT_LENGTH
static final String	CONTENT_TRANSFER_ENCODING
static final String	CONTENT_TYPE
static final String	FILE_LENGTH
static final int	IO_FILE_BUFFER_SIZE
static final String	MD5_HEADER

Constructor Summary

Constructors

Constructor	Description
Util()

Method Summary

Modifier and Type	Method	Description
static org.apache.hadoop.io.MD5Hash	doGetUrl(URL url, List<File> localPaths, Storage dstStorage, boolean getChecksum, int timeout, DataTransferThrottler throttler)	Downloads the files at the specified url location into destination storage.
static URI	fileAsURI(File f)	Converts the passed File to a URI.
static List<InetSocketAddress>	getAddressesList(URI uri, org.apache.hadoop.conf.Configuration conf)
static float	getBlockPoolUsedPercentStdDev(org.apache.hadoop.hdfs.server.protocol.StorageReport[] storageReports)	Return the standard deviation of storage block pool usage.
static List<InetSocketAddress>	getLoggerAddresses(URI uri, Set<InetSocketAddress> addrsToExclude, org.apache.hadoop.conf.Configuration conf)
static boolean	isDiskStatsEnabled(int fileIOSamplingPercentage)
static org.apache.hadoop.io.MD5Hash	receiveFile(String url, List<File> localPaths, Storage dstStorage, boolean getChecksum, long advertisedSize, org.apache.hadoop.io.MD5Hash advertisedDigest, String fsImageName, InputStream stream, DataTransferThrottler throttler)	Receives file at the url location from the input stream and puts them in the specified destination storage location.
static void	setTimeout(HttpURLConnection connection, int timeout)	Sets a timeout value in millisecods for the Http connection.
static List<URI>	stringCollectionAsURIs(Collection<String> names)	Converts a collection of strings into a collection of URIs.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

FILE_LENGTH

public static final String FILE_LENGTH

See Also:

• Constant Field Values

CONTENT_LENGTH

public static final String CONTENT_LENGTH

See Also:

• Constant Field Values

MD5_HEADER

public static final String MD5_HEADER

See Also:

• Constant Field Values

CONTENT_TYPE

public static final String CONTENT_TYPE

See Also:

• Constant Field Values

CONTENT_TRANSFER_ENCODING

public static final String CONTENT_TRANSFER_ENCODING

See Also:

• Constant Field Values

IO_FILE_BUFFER_SIZE

public static final int IO_FILE_BUFFER_SIZE

connectionFactory

public static final org.apache.hadoop.hdfs.web.URLConnectionFactory connectionFactory

Constructor Details

Util

public Util()

Method Details

fileAsURI

public static URI fileAsURI(File f)
                     throws IOException

Converts the passed File to a URI. This method trims the trailing slash if one is appended because the underlying file is in fact a directory that exists.

Parameters:

f - the file to convert

Returns:

the resulting URI

Throws:

IOException

stringCollectionAsURIs

public static List<URI> stringCollectionAsURIs(Collection<String> names)

Converts a collection of strings into a collection of URIs.

Parameters:

names - collection of strings to convert to URIs

Returns:

collection of URIs

doGetUrl

public static org.apache.hadoop.io.MD5Hash doGetUrl(URL url,
 List<File> localPaths,
 Storage dstStorage,
 boolean getChecksum,
 int timeout,
 DataTransferThrottler throttler)
                                             throws IOException

Downloads the files at the specified url location into destination storage.

Throws:

IOException

receiveFile

public static org.apache.hadoop.io.MD5Hash receiveFile(String url,
 List<File> localPaths,
 Storage dstStorage,
 boolean getChecksum,
 long advertisedSize,
 org.apache.hadoop.io.MD5Hash advertisedDigest,
 String fsImageName,
 InputStream stream,
 DataTransferThrottler throttler)
                                                throws IOException

Receives file at the url location from the input stream and puts them in the specified destination storage location.

Throws:

IOException

setTimeout

public static void setTimeout(HttpURLConnection connection,
 int timeout)

Sets a timeout value in millisecods for the Http connection.

Parameters:

connection - the Http connection for which timeout needs to be set

timeout - value to be set as timeout in milliseconds

getAddressesList

public static List<InetSocketAddress> getAddressesList(URI uri,
 org.apache.hadoop.conf.Configuration conf)
                                                throws IOException

Throws:

IOException

getLoggerAddresses

public static List<InetSocketAddress> getLoggerAddresses(URI uri,
 Set<InetSocketAddress> addrsToExclude,
 org.apache.hadoop.conf.Configuration conf)
                                                  throws IOException

Throws:

IOException

isDiskStatsEnabled

public static boolean isDiskStatsEnabled(int fileIOSamplingPercentage)

getBlockPoolUsedPercentStdDev

public static float getBlockPoolUsedPercentStdDev(org.apache.hadoop.hdfs.server.protocol.StorageReport[] storageReports)

Return the standard deviation of storage block pool usage.