Package org.apache.hadoop.hdfs.tools

Class DiskBalancerCLI

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.hdfs.tools.DiskBalancerCLI
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
public class DiskBalancerCLI extends org.apache.hadoop.conf.Configured implements org.apache.hadoop.util.Tool
DiskBalancer is a tool that can be used to ensure that data is spread evenly across volumes of same storage type.

For example, if you have 3 disks, with 100 GB , 600 GB and 200 GB on each disk, this tool will ensure that each disk will have 300 GB.

This tool can be run while data nodes are fully functional.

At very high level diskbalancer computes a set of moves that will make disk utilization equal and then those moves are executed by the datanode.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    BANDWIDTH
    Specifies the maximum disk bandwidth to use per second.
    static final String
    BEFORE_TEMPLATE
    Template for the Before File.
    static final String
    CANCEL
    Cancels a running plan.
    static final int
    DEFAULT_TOP
    specify default top number of nodes to be processed.
    static final String
    EXECUTE
    Executes a given plan file on the target datanode.
    static final String
    HELP
    Help for the program.
    static final String
    MAXERROR
    Specifies the maximum errors to tolerate.
    static final String
    NODE
    Name or address of the node to execute against.
    static final String
    OUTFILE
    Output file name, for commands like report, plan etc.
    static final String
    PLAN
    Computes a plan for a given set of nodes.
    static final String
    PLAN_TEMPLATE
    Template for the plan file. it is node.plan.json.
    static final int
    PLAN_VERSION
     
    static final String
    QUERY
    Reports the status of disk balancer operation.
    static final String
    REPORT
    The report command prints out a disk fragmentation report about the data cluster.
    static final String
    SKIPDATECHECK
    Skips date check(now by default the plan is valid for 24 hours), and force execute the plan.
    static final String
    THRESHOLD
    Percentage of data unevenness that we are willing to live with.
    static final String
    TOP
    specify top number of nodes to be processed.
    static final String
    VERBOSE
    Runs the command in verbose mode.

  • Constructor Summary

    Constructors
    Constructor
    Description
    DiskBalancerCLI(org.apache.hadoop.conf.Configuration conf)
    Construct a DiskBalancer.
    DiskBalancerCLI(org.apache.hadoop.conf.Configuration conf, PrintStream printStream)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static org.apache.commons.cli.Options
    getCancelOptions()
    Returns Cancel Options.
    Command
    getCurrentCommand()
    Gets current command associated with this instance of DiskBalancer.
    static org.apache.commons.cli.Options
    getExecuteOptions()
    Retuns execute options.
    static org.apache.commons.cli.Options
    getHelpOptions()
    Returns help options.
    static org.apache.commons.cli.Options
    getPlanOptions()
    Returns Plan options.
    static org.apache.commons.cli.Options
    getQueryOptions()
    Returns Query Options.
    static org.apache.commons.cli.Options
    getReportOptions()
    Returns Report Options.
    static void
    main(String[] argv)
    Main for the DiskBalancer Command handling.
    int
    run(String[] args)
    Execute the command with the given arguments.

    Methods inherited from class org.apache.hadoop.conf.Configured

    getConf, setConf

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.conf.Configurable

    getConf, setConf

  • Field Details

    • PLAN

      public static final String PLAN
      Computes a plan for a given set of nodes.
      See Also:

    • OUTFILE

      public static final String OUTFILE
      Output file name, for commands like report, plan etc. This is an optional argument, by default diskbalancer will write all its output to /system/reports/diskbalancer of the current cluster it is operating against.
      See Also:

    • HELP

      public static final String HELP
      Help for the program.
      See Also:

    • THRESHOLD

      public static final String THRESHOLD
      Percentage of data unevenness that we are willing to live with. For example - a value like 10 indicates that we are okay with 10 % +/- from idealStorage Target.
      See Also:

    • BANDWIDTH

      public static final String BANDWIDTH
      Specifies the maximum disk bandwidth to use per second.
      See Also:

    • MAXERROR

      public static final String MAXERROR
      Specifies the maximum errors to tolerate.
      See Also:

    • EXECUTE

      public static final String EXECUTE
      Executes a given plan file on the target datanode.
      See Also:

    • SKIPDATECHECK

      public static final String SKIPDATECHECK
      Skips date check(now by default the plan is valid for 24 hours), and force execute the plan.
      See Also:

    • REPORT

      public static final String REPORT
      The report command prints out a disk fragmentation report about the data cluster. By default it prints the DEFAULT_TOP machines names with high nodeDataDensity {DiskBalancerDataNode#getNodeDataDensity} values. This means that these are the nodes that deviates from the ideal data distribution.
      See Also:

    • TOP

      public static final String TOP
      specify top number of nodes to be processed.
      See Also:

    • DEFAULT_TOP

      public static final int DEFAULT_TOP
      specify default top number of nodes to be processed.
      See Also:

    • NODE

      public static final String NODE
      Name or address of the node to execute against.
      See Also:

    • VERBOSE

      public static final String VERBOSE
      Runs the command in verbose mode.
      See Also:

    • PLAN_VERSION

      public static final int PLAN_VERSION
      See Also:

    • QUERY

      public static final String QUERY
      Reports the status of disk balancer operation.
      See Also:

    • CANCEL

      public static final String CANCEL
      Cancels a running plan.
      See Also:

    • BEFORE_TEMPLATE

      public static final String BEFORE_TEMPLATE
      Template for the Before File. It is node.before.json.
      See Also:

    • PLAN_TEMPLATE

      public static final String PLAN_TEMPLATE
      Template for the plan file. it is node.plan.json.
      See Also:

  • Constructor Details

    • DiskBalancerCLI

      public DiskBalancerCLI(org.apache.hadoop.conf.Configuration conf)
      Construct a DiskBalancer.
      Parameters:
      conf -

    • DiskBalancerCLI

      public DiskBalancerCLI(org.apache.hadoop.conf.Configuration conf, PrintStream printStream)

  • Method Details

    • main

      public static void main(String[] argv) throws Exception
      Main for the DiskBalancer Command handling.
      Parameters:
      argv - - System Args Strings[]
      Throws:
      Exception

    • run

      public int run(String[] args) throws Exception
      Execute the command with the given arguments.
      Specified by:
      run in interface org.apache.hadoop.util.Tool
      Parameters:
      args - command specific arguments.
      Returns:
      exit code.
      Throws:
      Exception

    • getPlanOptions

      public static org.apache.commons.cli.Options getPlanOptions()
      Returns Plan options.
      Returns:
      Options.

    • getHelpOptions

      public static org.apache.commons.cli.Options getHelpOptions()
      Returns help options.
      Returns:
      - help options.

    • getExecuteOptions

      public static org.apache.commons.cli.Options getExecuteOptions()
      Retuns execute options.
      Returns:
      - execute options.

    • getQueryOptions

      public static org.apache.commons.cli.Options getQueryOptions()
      Returns Query Options.
      Returns:
      query Options

    • getCancelOptions

      public static org.apache.commons.cli.Options getCancelOptions()
      Returns Cancel Options.
      Returns:
      Options

    • getReportOptions

      public static org.apache.commons.cli.Options getReportOptions()
      Returns Report Options.
      Returns:
      Options

    • getCurrentCommand

      public Command getCurrentCommand()
      Gets current command associated with this instance of DiskBalancer.