Package org.apache.hadoop.io.erasurecode.rawcoder

Class RSLegacyRawDecoder

java.lang.Object

org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder

org.apache.hadoop.io.erasurecode.rawcoder.RSLegacyRawDecoder

@Private
public class RSLegacyRawDecoder
extends RawErasureDecoder

A raw erasure decoder in RS code scheme in pure Java in case native one isn't available in some environment. Please always use native implementations when possible. Currently this implementation will compute and decode not to read units unnecessarily due to the underlying implementation limit in GF. This will be addressed in HADOOP-11871.

Constructor Summary

Constructors

Constructor	Description
RSLegacyRawDecoder(ErasureCoderOptions coderOptions)

Method Summary

Modifier and Type	Method	Description
void	decode(byte[][] inputs, int[] erasedIndexes, byte[][] outputs)	Decode with inputs and erasedIndexes, generates outputs.
void	decode(ByteBuffer[] inputs, int[] erasedIndexes, ByteBuffer[] outputs)	Decode with inputs and erasedIndexes, generates outputs.
protected void	doDecode(org.apache.hadoop.io.erasurecode.rawcoder.ByteArrayDecodingState decodingState)	Perform the real decoding using bytes array, supporting offsets and lengths.
protected void	doDecode(org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState decodingState)	Perform the real decoding using Direct ByteBuffer.

Methods inherited from class org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder

allowChangeInputs, allowVerboseDump, decode, getNumAllUnits, getNumDataUnits, getNumParityUnits, preferDirectBuffer, release

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

RSLegacyRawDecoder

public RSLegacyRawDecoder(ErasureCoderOptions coderOptions)

Method Details

decode

public void decode(ByteBuffer[] inputs,
 int[] erasedIndexes,
 ByteBuffer[] outputs)
            throws IOException

Description copied from class: RawErasureDecoder

Decode with inputs and erasedIndexes, generates outputs. How to prepare for inputs: 1. Create an array containing data units + parity units. Please note the data units should be first or before the parity units. 2. Set null in the array locations specified via erasedIndexes to indicate they're erased and no data are to read from; 3. Set null in the array locations for extra redundant items, as they're not necessary to read when decoding. For example in RS-6-3, if only 1 unit is really erased, then we have 2 extra items as redundant. They can be set as null to indicate no data will be used from them. For an example using RS (6, 3), assuming sources (d0, d1, d2, d3, d4, d5) and parities (p0, p1, p2), d2 being erased. We can and may want to use only 6 units like (d1, d3, d4, d5, p0, p2) to recover d2. We will have: inputs = [null(d0), d1, null(d2), d3, d4, d5, p0, null(p1), p2] erasedIndexes = [2] // index of d2 into inputs array outputs = [a-writable-buffer] Note, for both inputs and outputs, no mixing of on-heap buffers and direct buffers are allowed. If the coder option ALLOW_CHANGE_INPUTS is set true (false by default), the content of input buffers may change after the call, subject to concrete implementation.

Overrides:

decode in class RawErasureDecoder

Parameters:

inputs - input buffers to read data from. The buffers' remaining will be 0 after decoding

erasedIndexes - indexes of erased units in the inputs array

outputs - output buffers to put decoded data into according to erasedIndexes, ready for read after the call

Throws:

IOException - raised on errors performing I/O.

decode

public void decode(byte[][] inputs,
 int[] erasedIndexes,
 byte[][] outputs)
            throws IOException

Description copied from class: RawErasureDecoder

Decode with inputs and erasedIndexes, generates outputs. More see above.

Overrides:

decode in class RawErasureDecoder

Parameters:

inputs - input buffers to read data from

erasedIndexes - indexes of erased units in the inputs array

outputs - output buffers to put decoded data into according to erasedIndexes, ready for read after the call

Throws:

IOException - if the decoder is closed.

doDecode

protected void doDecode(org.apache.hadoop.io.erasurecode.rawcoder.ByteArrayDecodingState decodingState)

Description copied from class: RawErasureDecoder

Perform the real decoding using bytes array, supporting offsets and lengths.

Specified by:

doDecode in class RawErasureDecoder

Parameters:

decodingState - the decoding state

doDecode

protected void doDecode(org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState decodingState)

Description copied from class: RawErasureDecoder

Perform the real decoding using Direct ByteBuffer.

Specified by:

doDecode in class RawErasureDecoder

Parameters:

decodingState - the decoding state