Package org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime

Class DockerLinuxContainerRuntime

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.OCIContainerRuntime

org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime

All Implemented Interfaces:

LinuxContainerRuntime, ContainerRuntime

@Private
@Unstable
public class DockerLinuxContainerRuntime
extends OCIContainerRuntime

This class is an extension of OCIContainerRuntime that uses the native container-executor binary via a PrivilegedOperationExecutor instance to launch processes inside Docker containers.

The following environment variables are used to configure the Docker engine:

• YARN_CONTAINER_RUNTIME_TYPE ultimately determines whether a Docker container will be used. If the value is docker, a Docker container will be used. Otherwise a regular process tree container will be used. This environment variable is checked by the isDockerContainerRequested(org.apache.hadoop.conf.Configuration, java.util.Map<java.lang.String, java.lang.String>) method, which is called by the DelegatingLinuxContainerRuntime.
• YARN_CONTAINER_RUNTIME_DOCKER_IMAGE names which image will be used to launch the Docker container.
• YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE controls whether the Docker container's default command is overridden. When set to true, the Docker container's command will be bash <path_to_launch_script>. When unset or set to false the Docker container's default command is used.
• YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK sets the network type to be used by the Docker container. It must be a valid value as determined by the yarn.nodemanager.runtime.linux.docker.allowed-container-networks property.
• YARN_CONTAINER_RUNTIME_DOCKER_PORTS_MAPPING allows users to specify ports mapping for the bridge network Docker container. The value of the environment variable should be a comma-separated list of ports mapping. It's the same to "-p" option for the Docker run command. If the value is empty, "-P" will be added.
• YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_PID_NAMESPACE controls which PID namespace will be used by the Docker container. By default, each Docker container has its own PID namespace. To share the namespace of the host, the yarn.nodemanager.runtime.linux.docker.host-pid-namespace.allowed property must be set to true. If the host PID namespace is allowed and this environment variable is set to host, the Docker container will share the host's PID namespace. No other value is allowed.
• YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_HOSTNAME sets the hostname to be used by the Docker container. If not specified, a hostname will be derived from the container ID and set as default hostname for networks other than 'host'.
• YARN_CONTAINER_RUNTIME_DOCKER_RUN_PRIVILEGED_CONTAINER controls whether the Docker container is a privileged container. In order to use privileged containers, the yarn.nodemanager.runtime.linux.docker.privileged-containers.allowed property must be set to true, and the application owner must appear in the value of the yarn.nodemanager.runtime.linux.docker.privileged-containers.acl property. If this environment variable is set to true, a privileged Docker container will be used if allowed. No other value is allowed, so the environment variable should be left unset rather than setting it to false.
• YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS allows users to specify additional volume mounts for the Docker container. The value of the environment variable should be a comma-separated list of mounts. All such mounts must be given as source:dest[:mode] and the mode must be "ro" (read-only) or "rw" (read-write) to specify the type of access being requested. If neither is specified, read-write will be assumed. The mode may include a bind propagation option. In that case, the mode should either be of the form [option], rw+[option], or ro+[option]. Valid bind propagation options are shared, rshared, slave, rslave, private, and rprivate. The requested mounts will be validated by container-executor based on the values set in container-executor.cfg for docker.allowed.ro-mounts and docker.allowed.rw-mounts.
• YARN_CONTAINER_RUNTIME_DOCKER_TMPFS_MOUNTS allows users to specify additional tmpfs mounts for the Docker container. The value of the environment variable should be a comma-separated list of mounts.
• YARN_CONTAINER_RUNTIME_DOCKER_DELAYED_REMOVAL allows a user to request delayed deletion of the Docker containers on a per container basis. If true, Docker containers will not be removed until the duration defined by yarn.nodemanager.delete.debug-delay-sec has elapsed. Administrators can disable this feature through the yarn-site property yarn.nodemanager.runtime.linux.docker.delayed-removal.allowed. This feature is disabled by default. When this feature is disabled or set to false, the container will be removed as soon as it exits.
• YARN_CONTAINER_RUNTIME_YARN_SYSFS_ENABLE allows export yarn service json to docker container. This feature is disabled by default. When this feature is set, app.json will be available in /hadoop/yarn/sysfs/app.json.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	DOCKER_IMAGE_PATTERN
static final String	ENV_DOCKER_CONTAINER_CLIENT_CONFIG
static final String	ENV_DOCKER_CONTAINER_DELAYED_REMOVAL
static final String	ENV_DOCKER_CONTAINER_DOCKER_RUNTIME
static final String	ENV_DOCKER_CONTAINER_DOCKER_SERVICE_MODE
static final String	ENV_DOCKER_CONTAINER_HOSTNAME
static final String	ENV_DOCKER_CONTAINER_IMAGE
static final String	ENV_DOCKER_CONTAINER_MOUNTS
static final String	ENV_DOCKER_CONTAINER_NETWORK
static final String	ENV_DOCKER_CONTAINER_PORTS_MAPPING
static final String	ENV_DOCKER_CONTAINER_TMPFS_MOUNTS
static final String	ENV_DOCKER_CONTAINER_YARN_SYSFS
static final String	ENV_OCI_CONTAINER_PID_NAMESPACE
static final String	ENV_OCI_CONTAINER_RUN_PRIVILEGED_CONTAINER

Fields inherited from class org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.OCIContainerRuntime

CONTAINER_PID_NAMESPACE_SUFFIX, RUN_PRIVILEGED_CONTAINER_SUFFIX, RUNTIME_PREFIX

Constructor Summary

Constructors

Constructor	Description
DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor)	Create an instance using the given PrivilegedOperationExecutor instance for performing operations.
DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor, CGroupsHandler cGroupsHandler)	Create an instance using the given PrivilegedOperationExecutor instance for performing operations and the given CGroupsHandler instance.

Method Summary

Modifier and Type	Method	Description
protected void	addCGroupParentIfRequired(String resourcesOptions, String containerIdStr, DockerRunCommand runCommand)	If CGROUPS in enabled and not set to none, then set the CGROUP parent for the command instance.
org.apache.hadoop.hdfs.protocol.datatransfer.IOStreamPair	execContainer(ContainerExecContext ctx)	Perform docker exec command into running container.
Set<String>	getCapabilities()
String	getExposedPorts(Container container)	Return the exposed ports of the container.
String[]	getIpAndHost(Container container)	Return the host and ip of the container.
void	initialize(org.apache.hadoop.conf.Configuration conf, Context nmContext)	Initialize the runtime.
static boolean	isDockerContainerRequested(org.apache.hadoop.conf.Configuration daemonConf, Map<String,String> env)	Return whether the given environment variables indicate that the operation is requesting a Docker container.
boolean	isRuntimeRequested(Map<String,String> env)	Return whether the given environment variables indicate that the operation is requesting this runtime.
void	launchContainer(ContainerRuntimeContext ctx)	Launch a container.
void	pullImageFromRemote(String containerIdStr, String imageName)
void	reapContainer(ContainerRuntimeContext ctx)	Reap the docker container.
void	relaunchContainer(ContainerRuntimeContext ctx)	Relaunch a container.
void	signalContainer(ContainerRuntimeContext ctx)	Signal the docker container.
static void	validateImageName(String imageName)

Methods inherited from class org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.OCIContainerRuntime

allowHostPidNamespace, allowPrivilegedContainerExecution, formatOciEnvKey, getCsiClients, getGroupIdInfo, getUserIdInfo, initiateCsiClients, isContainerRequestedAsPrivileged, isOCICompliantContainerRequested, mountReadOnlyPath, prepareContainer, validateContainerNetworkType, validateContainerRuntimeType, validateHostname

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.LinuxContainerRuntime

getLocalResources, start, stop

Field Details

DOCKER_IMAGE_PATTERN

public static final String DOCKER_IMAGE_PATTERN

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_IMAGE

@Private
public static final String ENV_DOCKER_CONTAINER_IMAGE

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_CLIENT_CONFIG

@Private
public static final String ENV_DOCKER_CONTAINER_CLIENT_CONFIG

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_NETWORK

@Private
public static final String ENV_DOCKER_CONTAINER_NETWORK

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_HOSTNAME

@Private
public static final String ENV_DOCKER_CONTAINER_HOSTNAME

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_MOUNTS

@Private
public static final String ENV_DOCKER_CONTAINER_MOUNTS

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_TMPFS_MOUNTS

@Private
public static final String ENV_DOCKER_CONTAINER_TMPFS_MOUNTS

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_DELAYED_REMOVAL

@Private
public static final String ENV_DOCKER_CONTAINER_DELAYED_REMOVAL

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_PORTS_MAPPING

@Private
public static final String ENV_DOCKER_CONTAINER_PORTS_MAPPING

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_YARN_SYSFS

@Private
public static final String ENV_DOCKER_CONTAINER_YARN_SYSFS

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_DOCKER_RUNTIME

@Private
public static final String ENV_DOCKER_CONTAINER_DOCKER_RUNTIME

See Also:

• Constant Field Values

ENV_DOCKER_CONTAINER_DOCKER_SERVICE_MODE

@Private
public static final String ENV_DOCKER_CONTAINER_DOCKER_SERVICE_MODE

See Also:

• Constant Field Values

ENV_OCI_CONTAINER_PID_NAMESPACE

@Private
public static final String ENV_OCI_CONTAINER_PID_NAMESPACE

ENV_OCI_CONTAINER_RUN_PRIVILEGED_CONTAINER

@Private
public static final String ENV_OCI_CONTAINER_RUN_PRIVILEGED_CONTAINER

Constructor Details

DockerLinuxContainerRuntime

public DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor)

Create an instance using the given PrivilegedOperationExecutor instance for performing operations.

Parameters:

privilegedOperationExecutor - the PrivilegedOperationExecutor instance

DockerLinuxContainerRuntime

@VisibleForTesting
public DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor,
 CGroupsHandler cGroupsHandler)

Create an instance using the given PrivilegedOperationExecutor instance for performing operations and the given CGroupsHandler instance. This constructor is intended for use in testing.

Parameters:

privilegedOperationExecutor - the PrivilegedOperationExecutor instance

cGroupsHandler - the CGroupsHandler instance

Method Details

isDockerContainerRequested

public static boolean isDockerContainerRequested(org.apache.hadoop.conf.Configuration daemonConf,
 Map<String,String> env)

Return whether the given environment variables indicate that the operation is requesting a Docker container. If the environment contains a key called YARN_CONTAINER_RUNTIME_TYPE whose value is docker, this method will return true. Otherwise it will return false.

Parameters:

daemonConf - the NodeManager daemon configuration

env - the environment variable settings for the operation

Returns:

whether a Docker container is requested

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf,
 Context nmContext)
                throws ContainerExecutionException

Description copied from interface: LinuxContainerRuntime

Initialize the runtime.

Specified by:

initialize in interface LinuxContainerRuntime

Overrides:

initialize in class OCIContainerRuntime

Parameters:

conf - the Configuration to use

nmContext - NMContext

Throws:

ContainerExecutionException - if an error occurs while initializing the runtime

isRuntimeRequested

public boolean isRuntimeRequested(Map<String,String> env)

Description copied from interface: LinuxContainerRuntime

Return whether the given environment variables indicate that the operation is requesting this runtime.

Parameters:

env - the environment variable settings for the operation

Returns:

whether this runtime is requested

getCapabilities

public Set<String> getCapabilities()

addCGroupParentIfRequired

@VisibleForTesting
protected void addCGroupParentIfRequired(String resourcesOptions,
 String containerIdStr,
 DockerRunCommand runCommand)

If CGROUPS in enabled and not set to none, then set the CGROUP parent for the command instance.

Parameters:

resourcesOptions - the resource options to check for "cgroups=none"

containerIdStr - the container ID

runCommand - the command to set with the CGROUP parent

launchContainer

public void launchContainer(ContainerRuntimeContext ctx)
                     throws ContainerExecutionException

Description copied from interface: ContainerRuntime

Launch a container.

Parameters:

ctx - the ContainerRuntimeContext

Throws:

ContainerExecutionException - if an error occurs while launching the container

relaunchContainer

public void relaunchContainer(ContainerRuntimeContext ctx)
                       throws ContainerExecutionException

Description copied from interface: ContainerRuntime

Relaunch a container.

Parameters:

ctx - the ContainerRuntimeContext

Throws:

ContainerExecutionException - if an error occurs while relaunching the container

signalContainer

public void signalContainer(ContainerRuntimeContext ctx)
                     throws ContainerExecutionException

Signal the docker container. Signals are used to check the liveliness of the container as well as to stop/kill the container. The following outlines the docker container signal handling.

1. If the null signal is sent, run kill -0 on the pid. This is used to check if the container is still alive, which is necessary for reacquiring containers on NM restart.
2. If SIGTERM, SIGKILL is sent, attempt to stop and remove the docker container.
3. If the docker container exists and is running, execute docker stop.
4. If any other signal is sent, signal the container using docker kill.

Parameters:

ctx - the ContainerRuntimeContext.

Throws:

ContainerExecutionException - if the signaling fails.

reapContainer

public void reapContainer(ContainerRuntimeContext ctx)
                   throws ContainerExecutionException

Reap the docker container.

Parameters:

ctx - the ContainerRuntimeContext.

Throws:

ContainerExecutionException - if the removal fails.

execContainer

public org.apache.hadoop.hdfs.protocol.datatransfer.IOStreamPair execContainer(ContainerExecContext ctx)
                                                                        throws ContainerExecutionException

Perform docker exec command into running container.

Parameters:

ctx - container exec context

Returns:

IOStreams of docker exec

Throws:

ContainerExecutionException

getIpAndHost

public String[] getIpAndHost(Container container)

Description copied from interface: ContainerRuntime

Return the host and ip of the container.

Parameters:

container - the Container

getExposedPorts

public String getExposedPorts(Container container)

Description copied from interface: ContainerRuntime

Return the exposed ports of the container.

Parameters:

container - the Container

Returns:

List of exposed ports

validateImageName

public static void validateImageName(String imageName)
                              throws ContainerExecutionException

Throws:

ContainerExecutionException

pullImageFromRemote

public void pullImageFromRemote(String containerIdStr,
 String imageName)
                         throws ContainerExecutionException

Throws:

ContainerExecutionException