org.apache.hive.streaming

Class StrictDelimitedInputWriter

java.lang.Object

org.apache.hive.streaming.AbstractRecordWriter

org.apache.hive.streaming.StrictDelimitedInputWriter

All Implemented Interfaces:

RecordWriter

public class StrictDelimitedInputWriter
extends AbstractRecordWriter

Streaming Writer handles delimited input (eg. CSV). Delimited input is parsed to extract partition values, bucketing info and is forwarded to record updater. Uses Lazy Simple SerDe to process delimited input. NOTE: This record writer is NOT thread-safe. Use one record writer per streaming connection.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StrictDelimitedInputWriter.Builder

Nested classes/interfaces inherited from class org.apache.hive.streaming.AbstractRecordWriter

AbstractRecordWriter.OrcMemoryPressureMonitor

Field Summary

Fields inherited from class org.apache.hive.streaming.AbstractRecordWriter

acidOutputFormat, addedPartitions, autoFlush, bucketFieldData, bucketIds, bucketObjInspectors, bucketStructFields, conf, conn, curBatchMaxWriteId, curBatchMinWriteId, defaultPartitionName, fs, fullyQualifiedTableName, heapMemoryMonitor, ingestSizeBytes, ingestSizeThreshold, inputColumns, inputRowObjectInspector, inputTypes, isBucketed, lineDelimiter, lowMemoryCanary, memoryUsageThreshold, outputRowObjectInspector, partitionColumns, partitionFieldData, partitionObjInspectors, partitionPaths, partitionStructFields, table, totalBuckets, updaters

Method Summary

Modifier and Type	Method and Description
LazySimpleSerDe	createSerde() Create SerDe for the record writer.
Object	encode(byte[] record) Encode a record as an Object that Hive can read with the ObjectInspector associated with the serde returned by AbstractRecordWriter.createSerde().
static StrictDelimitedInputWriter.Builder	newBuilder()

Methods inherited from class org.apache.hive.streaming.AbstractRecordWriter

checkAutoFlush, close, createRecordUpdater, flush, getBucket, getBucketColIDs, getBucketFields, getObjectInspectorsForBucketedCols, getPartitionFields, getPartitions, getPartitionValues, getRecordUpdater, getWatermark, init, initializeBuckets, logStats, prepareBucketingFields, preparePartitioningFields, setupMemoryMonitoring, write, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

newBuilder

public static StrictDelimitedInputWriter.Builder newBuilder()

encode

public Object encode(byte[] record)
              throws SerializationError

Description copied from class: AbstractRecordWriter

Encode a record as an Object that Hive can read with the ObjectInspector associated with the serde returned by AbstractRecordWriter.createSerde(). This is public so that test frameworks can use it.

Specified by:

encode in class AbstractRecordWriter

Parameters:

record - record to be deserialized

Returns:

deserialized record as an Object

Throws:

SerializationError - - any error during serialization or deserialization of record

createSerde

public LazySimpleSerDe createSerde()
                            throws SerializationError

Description copied from class: AbstractRecordWriter

Create SerDe for the record writer.

Specified by:

createSerde in class AbstractRecordWriter

Returns:

- serde

Throws:

SerializationError - - if serde cannot be created.