Package org.apache.hadoop.io

Class SequenceFile.Writer

java.lang.Object

org.apache.hadoop.io.SequenceFile.Writer

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable, StreamCapabilities, Syncable

Enclosing class:

SequenceFile

public static class SequenceFile.Writer
extends Object
implements Closeable, Syncable, Flushable, StreamCapabilities

Write key/value pairs to a sequence-format file.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	SequenceFile.Writer.Option

Nested classes/interfaces inherited from interface org.apache.hadoop.fs.StreamCapabilities

StreamCapabilities.StreamCapability

Field Summary

Fields

Modifier and Type	Field	Description
protected Serializer	compressedValSerializer
protected Serializer	keySerializer
protected Serializer	uncompressedValSerializer

Fields inherited from interface org.apache.hadoop.fs.StreamCapabilities

ABORTABLE_STREAM, DROPBEHIND, HFLUSH, HSYNC, IOSTATISTICS, IOSTATISTICS_CONTEXT, PREADBYTEBUFFER, READAHEAD, READBYTEBUFFER, UNBUFFER, VECTOREDIO, VECTOREDIO_BUFFERS_SLICED

Constructor Summary

Constructors

Constructor	Description
Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass)	Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.
Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, int bufferSize, short replication, long blockSize, Progressable progress, SequenceFile.Metadata metadata)	Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.
Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, Progressable progress, SequenceFile.Metadata metadata)	Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Method Summary

Modifier and Type	Method	Description
void	append(Object key, Object val)	Append a key/value pair.
void	append(Writable key, Writable val)	Append a key/value pair.
static SequenceFile.Writer.Option	appendIfExists(boolean value)
void	appendRaw(byte[] keyData, int keyOffset, int keyLength, SequenceFile.ValueBytes val)
static SequenceFile.Writer.Option	blockSize(long value)
static SequenceFile.Writer.Option	bufferSize(int value)
void	close()	Close the file.
static SequenceFile.Writer.Option	compression(SequenceFile.CompressionType value)
static SequenceFile.Writer.Option	compression(SequenceFile.CompressionType value, CompressionCodec codec)
static SequenceFile.Writer.Option	file(Path value)
void	flush()
CompressionCodec	getCompressionCodec()
Class	getKeyClass()
long	getLength()
Class	getValueClass()
boolean	hasCapability(String capability)	Query the stream for a specific capability.
void	hflush()	Flush out the data in client's user buffer.
void	hsync()	Similar to posix fsync, flush out the data in client's user buffer all the way to the disk device (but the disk may have it in its cache).
static SequenceFile.Writer.Option	keyClass(Class<?> value)
static SequenceFile.Writer.Option	metadata(SequenceFile.Metadata value)
static SequenceFile.Writer.Option	progressable(Progressable value)
static SequenceFile.Writer.Option	replication(short value)
static SequenceFile.Writer.Option	stream(FSDataOutputStream value)
void	sync()	create a sync point.
void	syncFs()	Deprecated. Use hsync() or hflush() instead
static SequenceFile.Writer.Option	syncInterval(int value)
static SequenceFile.Writer.Option	valueClass(Class<?> value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

keySerializer

protected Serializer keySerializer

uncompressedValSerializer

protected Serializer uncompressedValSerializer

compressedValSerializer

protected Serializer compressedValSerializer

Constructor Details

Writer

@Deprecated
public Writer(FileSystem fs,
 Configuration conf,
 Path name,
 Class keyClass,
 Class valClass)
       throws IOException

Deprecated.

Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Create the named file.

Parameters:

fs - input filesystem.

conf - input configuration.

name - input name.

keyClass - input keyClass.

valClass - input valClass.

Throws:

IOException - raised on errors performing I/O.

Writer

@Deprecated
public Writer(FileSystem fs,
 Configuration conf,
 Path name,
 Class keyClass,
 Class valClass,
 Progressable progress,
 SequenceFile.Metadata metadata)
       throws IOException

Deprecated.

Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Create the named file with write-progress reporter.

Parameters:

fs - input filesystem.

conf - input configuration.

name - input name.

keyClass - input keyClass.

valClass - input valClass.

progress - input progress.

metadata - input metadata.

Throws:

IOException - raised on errors performing I/O.

Writer

@Deprecated
public Writer(FileSystem fs,
 Configuration conf,
 Path name,
 Class keyClass,
 Class valClass,
 int bufferSize,
 short replication,
 long blockSize,
 Progressable progress,
 SequenceFile.Metadata metadata)
       throws IOException

Deprecated.

Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Create the named file with write-progress reporter.

Parameters:

fs - input filesystem.

conf - input configuration.

name - input name.

keyClass - input keyClass.

valClass - input valClass.

bufferSize - input bufferSize.

replication - input replication.

blockSize - input blockSize.

progress - input progress.

metadata - input metadata.

Throws:

IOException - raised on errors performing I/O.

Method Details

file

public static SequenceFile.Writer.Option file(Path value)

bufferSize

public static SequenceFile.Writer.Option bufferSize(int value)

stream

public static SequenceFile.Writer.Option stream(FSDataOutputStream value)

replication

public static SequenceFile.Writer.Option replication(short value)

appendIfExists

public static SequenceFile.Writer.Option appendIfExists(boolean value)

blockSize

public static SequenceFile.Writer.Option blockSize(long value)

progressable

public static SequenceFile.Writer.Option progressable(Progressable value)

keyClass

public static SequenceFile.Writer.Option keyClass(Class<?> value)

valueClass

public static SequenceFile.Writer.Option valueClass(Class<?> value)

metadata

public static SequenceFile.Writer.Option metadata(SequenceFile.Metadata value)

compression

public static SequenceFile.Writer.Option compression(SequenceFile.CompressionType value)

compression

public static SequenceFile.Writer.Option compression(SequenceFile.CompressionType value,
 CompressionCodec codec)

syncInterval

public static SequenceFile.Writer.Option syncInterval(int value)

getKeyClass

public Class getKeyClass()

Returns:

Returns the class of keys in this file.

getValueClass

public Class getValueClass()

Returns:

Returns the class of values in this file.

getCompressionCodec

public CompressionCodec getCompressionCodec()

Returns:

Returns the compression codec of data in this file.

sync

public void sync()
          throws IOException

create a sync point.

Throws:

IOException - raised on errors performing I/O.

syncFs

@Deprecated
public void syncFs()
            throws IOException

Deprecated.

Use hsync() or hflush() instead

flush all currently written data to the file system.

Throws:

IOException - raised on errors performing I/O.

hsync

public void hsync()
           throws IOException

Description copied from interface: Syncable

Similar to posix fsync, flush out the data in client's user buffer all the way to the disk device (but the disk may have it in its cache).

Specified by:

hsync in interface Syncable

Throws:

IOException - if error occurs

hflush

public void hflush()
            throws IOException

Description copied from interface: Syncable

Flush out the data in client's user buffer. After the return of this call, new readers will see the data.

Specified by:

hflush in interface Syncable

Throws:

IOException - if any error occurs

flush

public void flush()
           throws IOException

Specified by:

flush in interface Flushable

Throws:

IOException

hasCapability

public boolean hasCapability(String capability)

Description copied from interface: StreamCapabilities

Query the stream for a specific capability.

Specified by:

hasCapability in interface StreamCapabilities

Parameters:

capability - string to query the stream support for.

Returns:

True if the stream supports capability.

close

public void close()
           throws IOException

Close the file.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

append

public void append(Writable key,
 Writable val)
            throws IOException

Append a key/value pair.

Parameters:

key - input Writable key.

val - input Writable val.

Throws:

IOException - raised on errors performing I/O.

append

public void append(Object key,
 Object val)
            throws IOException

Append a key/value pair.

Parameters:

key - input Object key.

val - input Object val.

Throws:

IOException - raised on errors performing I/O.

appendRaw

public void appendRaw(byte[] keyData,
 int keyOffset,
 int keyLength,
 SequenceFile.ValueBytes val)
               throws IOException

Throws:

IOException

getLength

public long getLength()
               throws IOException

Returns:

Returns the current length of the output file.

This always returns a synchronized position. In other words, immediately after calling SequenceFile.Reader.seek(long) with a position returned by this method, SequenceFile.Reader.next(Writable) may be called. However the key may be earlier in the file than key last written when this method was called (e.g., with block-compression, it may be the first key in the block that was being written when this method was called).

Throws:

IOException - raised on errors performing I/O.