Package org.apache.hadoop.fs.shell

Class Command

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.fs.shell.Command
All Implemented Interfaces:
Configurable
Direct Known Subclasses:
FsCommand
@Private @Evolving public abstract class Command extends Configured
An abstract class for the execution of a file system command

  • Field Details

    • COMMAND_NAME_FIELD

      public static final String COMMAND_NAME_FIELD
      field name indicating the default name of the command
      See Also:

    • COMMAND_USAGE_FIELD

      public static final String COMMAND_USAGE_FIELD
      field name indicating the command's usage switches and arguments format
      See Also:

    • COMMAND_DESCRIPTION_FIELD

      public static final String COMMAND_DESCRIPTION_FIELD
      field name indicating the command's long description
      See Also:

    • args

      protected String[] args

    • name

      protected String name

    • exitCode

      protected int exitCode

    • numErrors

      protected int numErrors

    • recursive

      protected boolean recursive

    • exceptions

      protected ArrayList<Exception> exceptions

    • out

      public PrintStream out
      allows stdout to be captured if necessary

    • err

      public PrintStream err
      allows stderr to be captured if necessary

  • Constructor Details

    • Command

      protected Command()
      Constructor

    • Command

      protected Command(Configuration conf)
      Constructor.
      Parameters:
      conf - configuration.

  • Method Details

    • getCommandName

      public abstract String getCommandName()
      Returns:
      the command's name excluding the leading character -

    • setRecursive

      protected void setRecursive(boolean flag)

    • isRecursive

      protected boolean isRecursive()

    • getDepth

      protected int getDepth()

    • run

      protected abstract void run(Path path) throws IOException
      Execute the command on the input path
      Parameters:
      path - the input path
      Throws:
      IOException - if any error occurs

    • run

      protected void run(PathData pathData) throws IOException
      Execute the command on the input path data. Commands can override to make use of the resolved filesystem.
      Parameters:
      pathData - The input path with resolved filesystem
      Throws:
      IOException - raised on errors performing I/O.

    • runAll

      public int runAll()
      For each source path, execute the command
      Returns:
      0 if it runs successfully; -1 if it fails

    • setCommandFactory

      public void setCommandFactory(CommandFactory factory)
      sets the command factory for later use.
      Parameters:
      factory - factory.

    • getCommandFactory

      protected CommandFactory getCommandFactory()
      retrieves the command factory.
      Returns:
      command factory.

    • run

      public int run(String... argv)
      Invokes the command handler. The default behavior is to process options, expand arguments, and then process each argument. 
       run
 |-> processOptions(LinkedList)
 \-> processRawArguments(LinkedList)
      |-> expandArguments(LinkedList)
      |   \-> expandArgument(String)*
      \-> processArguments(LinkedList)
          |-> processArgument(PathData)*
          |   |-> processPathArgument(PathData)
          |   \-> processPaths(PathData, PathData...)
          |        \-> processPath(PathData)*
          \-> processNonexistentPath(PathData)
      Most commands will chose to implement just processOptions(LinkedList) and processPath(PathData)
      Parameters:
      argv - the list of command line arguments
      Returns:
      the exit code for the command
      Throws:
      IllegalArgumentException - if called with invalid arguments

    • exitCodeForError

      protected int exitCodeForError()
      The exit code to be returned if any errors occur during execution. This method is needed to account for the inconsistency in the exit codes returned by various commands.
      Returns:
      a non-zero exit code

    • processOptions

      protected void processOptions(LinkedList<String> args) throws IOException
      Must be implemented by commands to process the command line flags and check the bounds of the remaining arguments. If an IllegalArgumentException is thrown, the FsShell object will print the short usage of the command.
      Parameters:
      args - the command line arguments
      Throws:
      IOException - raised on errors performing I/O.

    • processRawArguments

      protected void processRawArguments(LinkedList<String> args) throws IOException
      Allows commands that don't use paths to handle the raw arguments. Default behavior is to expand the arguments via expandArguments(LinkedList) and pass the resulting list to processArguments(LinkedList)
      Parameters:
      args - the list of argument strings
      Throws:
      IOException - raised on errors performing I/O.

    • expandArguments

      protected LinkedList<PathData> expandArguments(LinkedList<String> args) throws IOException
      Expands a list of arguments into PathData objects. The default behavior is to call expandArgument(String) on each element which by default globs the argument. The loop catches IOExceptions, increments the error count, and displays the exception.
      Parameters:
      args - strings to expand into PathData objects
      Returns:
      list of all PathData objects the arguments
      Throws:
      IOException - if anything goes wrong...

    • expandArgument

      protected List<PathData> expandArgument(String arg) throws IOException
      Expand the given argument into a list of PathData objects. The default behavior is to expand globs. Commands may override to perform other expansions on an argument.
      Parameters:
      arg - string pattern to expand
      Returns:
      list of PathData objects
      Throws:
      IOException - if anything goes wrong...

    • processArguments

      protected void processArguments(LinkedList<PathData> args) throws IOException
      Processes the command's list of expanded arguments. processArgument(PathData) will be invoked with each item in the list. The loop catches IOExceptions, increments the error count, and displays the exception.
      Parameters:
      args - a list of PathData to process
      Throws:
      IOException - if anything goes wrong...

    • processArgument

      protected void processArgument(PathData item) throws IOException
      Processes a PathData item, calling processPathArgument(PathData) or processNonexistentPath(PathData) on each item.
      Parameters:
      item - PathData item to process
      Throws:
      IOException - if anything goes wrong...

    • processPathArgument

      protected void processPathArgument(PathData item) throws IOException
      This is the last chance to modify an argument before going into the (possibly) recursive processPaths(PathData, PathData...) -> processPath(PathData) loop. Ex. ls and du use this to expand out directories.
      Parameters:
      item - a PathData representing a path which exists
      Throws:
      IOException - if anything goes wrong...

    • processNonexistentPath

      protected void processNonexistentPath(PathData item) throws IOException
      Provides a hook for handling paths that don't exist. By default it will throw an exception. Primarily overriden by commands that create paths such as mkdir or touch.
      Parameters:
      item - the PathData that doesn't exist
      Throws:
      FileNotFoundException - if arg is a path and it doesn't exist
      IOException - if anything else goes wrong...

    • processPaths

      protected void processPaths(PathData parent, PathData... items) throws IOException
      Iterates over the given expanded paths and invokes processPath(PathData) on each element. If "recursive" is true, will do a post-visit DFS on directories.
      Parameters:
      parent - if called via a recurse, will be the parent dir, else null
      items - a list of PathData objects to process
      Throws:
      IOException - if anything goes wrong...

    • processPaths

      protected void processPaths(PathData parent, RemoteIterator<PathData> itemsIterator) throws IOException
      Iterates over the given expanded paths and invokes processPath(PathData) on each element. If "recursive" is true, will do a post-visit DFS on directories.
      Parameters:
      parent - if called via a recurse, will be the parent dir, else null
      itemsIterator - a iterator of PathData objects to process
      Throws:
      IOException - if anything goes wrong...

    • isSorted

      protected boolean isSorted()
      Whether the directory listing for a path should be sorted.?
      Returns:
      true/false.

    • getListingGroupSize

      protected int getListingGroupSize()
      While using iterator method for listing for a path, whether to group items and process as array? If so what is the size of array?
      Returns:
      size of the grouping array.

    • isPathRecursable

      protected boolean isPathRecursable(PathData item) throws IOException
      Determines whether a PathData item is recursable. Default implementation is to recurse directories but can be overridden to recurse through symbolic links.
      Parameters:
      item - a PathData object
      Returns:
      true if the item is recursable, false otherwise
      Throws:
      IOException - if anything goes wrong in the user-implementation

    • processPath

      protected void processPath(PathData item) throws IOException
      Hook for commands to implement an operation to be applied on each path for the command. Note implementation of this method is optional if earlier methods in the chain handle the operation.
      Parameters:
      item - a PathData object
      Throws:
      RuntimeException - if invoked but not implemented
      IOException - if anything else goes wrong in the user-implementation

    • postProcessPath

      protected void postProcessPath(PathData item) throws IOException
      Hook for commands to implement an operation to be applied on each path for the command after being processed successfully
      Parameters:
      item - a PathData object
      Throws:
      IOException - if anything goes wrong...

    • recursePath

      protected void recursePath(PathData item) throws IOException
      Gets the directory listing for a path and invokes processPaths(PathData, PathData...)
      Parameters:
      item - PathData for directory to recurse into
      Throws:
      IOException - if anything goes wrong...

    • displayError

      public void displayError(Exception e)
      Display an exception prefaced with the command name. Also increments the error count for the command which will result in a non-zero exit code.
      Parameters:
      e - exception to display

    • displayError

      public void displayError(String message)
      Display an error string prefaced with the command name. Also increments the error count for the command which will result in a non-zero exit code.
      Parameters:
      message - error message to display

    • displayWarning

      public void displayWarning(String message)
      Display an warning string prefaced with the command name.
      Parameters:
      message - warning message to display

    • getName

      public String getName()
      The name of the command. Will first try to use the assigned name else fallback to the command's preferred name
      Returns:
      name of the command

    • setName

      public void setName(String name)
      Define the name of the command.
      Parameters:
      name - as invoked

    • getUsage

      public String getUsage()
      The short usage suitable for the synopsis
      Returns:
      "name options"

    • getDescription

      public String getDescription()
      The long usage suitable for help output
      Returns:
      text of the usage

    • isDeprecated

      public final boolean isDeprecated()
      Is the command deprecated?
      Returns:
      boolean

    • getReplacementCommand

      public String getReplacementCommand()
      The replacement for a deprecated command
      Returns:
      null if not deprecated, else alternative command