Package org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation

Class SampleContainerLogAggregationPolicy

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.SampleContainerLogAggregationPolicy

All Implemented Interfaces:

org.apache.hadoop.yarn.server.api.ContainerLogAggregationPolicy

@Private
public class SampleContainerLogAggregationPolicy
extends Object
implements org.apache.hadoop.yarn.server.api.ContainerLogAggregationPolicy

The sample policy samples logs of successful worker containers to aggregate. It always aggregates AM container and failed/killed worker containers' logs. To make sure small applications have enough logs, it only applies sampling beyond minimal number of containers. The parameters can be configured by SAMPLE_RATE and MIN_THRESHOLD. For example if SAMPLE_RATE is 0.2 and MIN_THRESHOLD is 20, for an application with 100 successful worker containers, 20 + (100-20) * 0.2 = 36 containers's logs will be aggregated.

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DEFAULT_SAMPLE_MIN_THRESHOLD
static final float	DEFAULT_SAMPLE_RATE

Constructor Summary

Constructors

Constructor	Description
SampleContainerLogAggregationPolicy()

Method Summary

Modifier and Type	Method	Description
static String	buildParameters(float sampleRate, int minThreshold)
void	parseParameters(String parameters)
boolean	shouldDoLogAggregation(org.apache.hadoop.yarn.server.api.ContainerLogContext logContext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_SAMPLE_RATE

public static final float DEFAULT_SAMPLE_RATE

See Also:

• Constant Field Values

DEFAULT_SAMPLE_MIN_THRESHOLD

public static final int DEFAULT_SAMPLE_MIN_THRESHOLD

See Also:

• Constant Field Values

Constructor Details

SampleContainerLogAggregationPolicy

public SampleContainerLogAggregationPolicy()

Method Details

buildParameters

public static String buildParameters(float sampleRate,
 int minThreshold)

parseParameters

public void parseParameters(String parameters)

Specified by:

parseParameters in interface org.apache.hadoop.yarn.server.api.ContainerLogAggregationPolicy

shouldDoLogAggregation

public boolean shouldDoLogAggregation(org.apache.hadoop.yarn.server.api.ContainerLogContext logContext)

Specified by:

shouldDoLogAggregation in interface org.apache.hadoop.yarn.server.api.ContainerLogAggregationPolicy