Package org.apache.hadoop.hdfs

Class DFSStripedOutputStream

java.lang.Object

java.io.OutputStream

org.apache.hadoop.fs.FSOutputSummer

org.apache.hadoop.hdfs.DFSOutputStream

org.apache.hadoop.hdfs.DFSStripedOutputStream

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable, org.apache.hadoop.fs.CanSetDropBehind, org.apache.hadoop.fs.StreamCapabilities, org.apache.hadoop.fs.Syncable

@Private
public class DFSStripedOutputStream
extends DFSOutputStream
implements org.apache.hadoop.fs.StreamCapabilities

This class supports writing files in striped layout and erasure coded format. Each stripe contains a sequence of cells.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.fs.StreamCapabilities

org.apache.hadoop.fs.StreamCapabilities.StreamCapability

Field Summary

Fields inherited from class org.apache.hadoop.hdfs.DFSOutputStream

blockSize, byteArrayManager, bytesPerChecksum, cachingStrategy, chunksPerPacket, closed, currentPacket, dfsClient, fileId, initialFileSize, lastFlushOffset, packetSize, shouldSyncBlock, src, streamer

Fields inherited from interface org.apache.hadoop.fs.StreamCapabilities

ABORTABLE_STREAM, DROPBEHIND, HFLUSH, HSYNC, IOSTATISTICS, IOSTATISTICS_CONTEXT, PREADBYTEBUFFER, READAHEAD, READBYTEBUFFER, UNBUFFER, VECTOREDIO, VECTOREDIO_BUFFERS_SLICED

Method Summary

Modifier and Type	Method	Description
protected void	closeImpl()
protected void	closeThreads(boolean force)
boolean	hasCapability(String capability)
void	hflush()	Flushes out to all replicas of the block.
void	hsync()
void	hsync(EnumSet<HdfsDataOutputStream.SyncFlag> syncFlags)	The expected semantics is all data have flushed out to all replicas and all replicas have done posix fsync equivalent - ie the OS has flushed it to the disk device (but the disk may have it in its cache).
protected void	start()
protected void	writeChunk(byte[] bytes, int offset, int len, byte[] checksum, int ckoff, int cklen)

Methods inherited from class org.apache.hadoop.hdfs.DFSOutputStream

adjustChunkBoundary, checkClosed, close, completeFile, computePacketChunkSize, createPacket, createWriteTraceScope, flushInternal, flushInternalWithoutWaitingAck, getAddBlockFlags, getCurrentBlockReplication, getFileEncryptionInfo, getFileId, getInitialLen, getNamespace, getNumCurrentReplicas, getPipeline, getStreamer, getUniqKey, recoverLease, setArtificialSlowdown, setChunksPerPacket, setDropBehind, toString, writeChunk

Methods inherited from class org.apache.hadoop.fs.FSOutputSummer

convertToByteStream, flush, flushBuffer, flushBuffer, getBufferedDataSize, getChecksumSize, getDataChecksum, resetChecksumBufSize, setChecksumBufSize, write, write

Methods inherited from class java.io.OutputStream

nullOutputStream, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

writeChunk

protected void writeChunk(byte[] bytes,
 int offset,
 int len,
 byte[] checksum,
 int ckoff,
 int cklen)
                   throws IOException

Overrides:

writeChunk in class DFSOutputStream

Throws:

IOException

hasCapability

public boolean hasCapability(String capability)

Specified by:

hasCapability in interface org.apache.hadoop.fs.StreamCapabilities

Overrides:

hasCapability in class DFSOutputStream

hflush

public void hflush()

Description copied from class: DFSOutputStream

Flushes out to all replicas of the block. The data is in the buffers of the DNs but not necessarily in the DN's OS buffers. It is a synchronous operation. When it returns, it guarantees that flushed data become visible to new readers. It is not guaranteed that data has been flushed to persistent store on the datanode. Block allocations are persisted on namenode.

Specified by:

hflush in interface org.apache.hadoop.fs.Syncable

Overrides:

hflush in class DFSOutputStream

hsync

public void hsync()

Specified by:

hsync in interface org.apache.hadoop.fs.Syncable

Overrides:

hsync in class DFSOutputStream

hsync

public void hsync(EnumSet<HdfsDataOutputStream.SyncFlag> syncFlags)

Description copied from class: DFSOutputStream

The expected semantics is all data have flushed out to all replicas and all replicas have done posix fsync equivalent - ie the OS has flushed it to the disk device (but the disk may have it in its cache). Note that only the current block is flushed to the disk device. To guarantee durable sync across block boundaries the stream should be created with CreateFlag.SYNC_BLOCK.

Overrides:

hsync in class DFSOutputStream

Parameters:

syncFlags - Indicate the semantic of the sync. Currently used to specify whether or not to update the block length in NameNode.

start

protected void start()

Overrides:

start in class DFSOutputStream

closeThreads

protected void closeThreads(boolean force)
                     throws IOException

Overrides:

closeThreads in class DFSOutputStream

Throws:

IOException

closeImpl

protected void closeImpl()
                  throws IOException

Overrides:

closeImpl in class DFSOutputStream

Throws:

IOException