Package org.apache.hadoop.hdfs.server.datanode

Class DirectoryScanner

java.lang.Object

org.apache.hadoop.hdfs.server.datanode.DirectoryScanner

All Implemented Interfaces:

Runnable

@Private
public class DirectoryScanner
extends Object
implements Runnable

Periodically scans the data directories for block and block metadata files. Reconciles the differences with block information maintained in the dataset.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DirectoryScanner.BlockPoolReport	Helper class for compiling block info reports per block pool.
class	DirectoryScanner.ReportCompiler	The ReportCompiler class encapsulates the process of searching a datanode's disks for block information.
static class	DirectoryScanner.ScanInfoVolumeReport	Helper class for compiling block info reports from report compiler threads.

Constructor Summary

Constructors

Constructor	Description
DirectoryScanner(FsDatasetSpi<?> dataset, org.apache.hadoop.conf.Configuration conf)	Create a new directory scanner, but don't cycle it running yet.

Method Summary

Modifier and Type	Method	Description
Collection<DirectoryScanner.ScanInfoVolumeReport>	getVolumeReports()	Get the lists of blocks on the disks in the data set.
void	reconcile()	Reconcile differences between disk and in-memory blocks
void	run()	Main program loop for DirectoryScanner.
void	setRetainDiffs(boolean b)	Allow retaining diffs for unit test and analysis.
void	start()	Start the scanner.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DirectoryScanner

public DirectoryScanner(FsDatasetSpi<?> dataset,
 org.apache.hadoop.conf.Configuration conf)

Create a new directory scanner, but don't cycle it running yet.

Parameters:

dataset - the dataset to scan

conf - the Configuration object

Method Details

setRetainDiffs

@VisibleForTesting
public void setRetainDiffs(boolean b)

Allow retaining diffs for unit test and analysis. Defaults to false (off).

Parameters:

b - whether to retain diffs

start

@VisibleForTesting
public void start()

Start the scanner. The scanner will run every DFSConfigKeys.DFS_DATANODE_DIRECTORYSCAN_INTERVAL_KEY seconds.

run

public void run()

Main program loop for DirectoryScanner. Runs reconcile() and handles any exceptions.

Specified by:

run in interface Runnable

reconcile

@VisibleForTesting
public void reconcile()
               throws IOException

Reconcile differences between disk and in-memory blocks

Throws:

IOException

getVolumeReports

@VisibleForTesting
public Collection<DirectoryScanner.ScanInfoVolumeReport> getVolumeReports()

Get the lists of blocks on the disks in the data set.