org.apache.hive.hcatalog.streaming

Class DelimitedInputWriter

java.lang.Object

org.apache.hive.hcatalog.streaming.AbstractRecordWriter

org.apache.hive.hcatalog.streaming.DelimitedInputWriter

All Implemented Interfaces:

RecordWriter

Deprecated.

as of Hive 3.0.0, replaced by StrictDelimitedInputWriter

@Deprecated
public class DelimitedInputWriter
extends AbstractRecordWriter

Streaming Writer handles delimited input (eg. CSV). Delimited input is parsed & reordered to match column order in table Uses Lazy Simple Serde to process delimited input

Constructor Summary

Constructors

Constructor and Description
DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint) Deprecated. As of release 1.3/2.1. Replaced by DelimitedInputWriter(String[], String, HiveEndPoint, StreamingConnection)
DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf) Deprecated. As of release 1.3/2.1. Replaced by DelimitedInputWriter(String[], String, HiveEndPoint, HiveConf, StreamingConnection)
DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf, char serdeSeparator) Deprecated. As of release 1.3/2.1. Replaced by DelimitedInputWriter(String[], String, HiveEndPoint, HiveConf, char, StreamingConnection)
DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf, char serdeSeparator, StreamingConnection conn) Deprecated. Constructor.
DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf, StreamingConnection conn) Deprecated. Constructor.
DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, StreamingConnection conn) Deprecated. Constructor.

Method Summary

Modifier and Type	Method and Description
protected static LazySimpleSerDe	createSerde(org.apache.hadoop.hive.metastore.api.Table tbl, HiveConf conf, char serdeSeparator) Deprecated. Creates LazySimpleSerde
Object	encode(byte[] record) Deprecated. Encode a record as an Object that Hive can read with the ObjectInspector associated with the serde returned by AbstractRecordWriter.getSerde().
protected ObjectInspector[]	getBucketObjectInspectors() Deprecated.
protected StructField[]	getBucketStructFields() Deprecated.
protected LazySimpleStructObjectInspector	getRecordObjectInspector() Deprecated.
AbstractSerDe	getSerde() Deprecated. Get the SerDe for the Objects created by AbstractRecordWriter.encode(byte[]).
char	getSerdeSeparator() Deprecated.
protected ArrayList<String>	getTableColumns() Deprecated.
protected byte[]	reorderFields(byte[] record) Deprecated.
void	write(long writeId, byte[] record) Deprecated. Writes using a hive RecordUpdater

Methods inherited from class org.apache.hive.hcatalog.streaming.AbstractRecordWriter

clear, closeBatch, flush, getBucket, getObjectInspectorsForBucketedCols, newBatch

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DelimitedInputWriter

public DelimitedInputWriter(String[] colNamesForFields,
                            String delimiter,
                            HiveEndPoint endPoint,
                            StreamingConnection conn)
                     throws ClassNotFoundException,
                            ConnectionError,
                            SerializationError,
                            InvalidColumn,
                            StreamingException

Deprecated.

Constructor. Uses default separator of the LazySimpleSerde

Parameters:

colNamesForFields - Column name assignment for input fields. nulls or empty strings in the array indicates the fields to be skipped

delimiter - input field delimiter

endPoint - Hive endpoint

Throws:

ConnectionError - Problem talking to Hive

ClassNotFoundException - Serde class not found

SerializationError - Serde initialization/interaction failed

StreamingException - Problem acquiring file system path for partition

InvalidColumn - any element in colNamesForFields refers to a non existing column

DelimitedInputWriter

public DelimitedInputWriter(String[] colNamesForFields,
                            String delimiter,
                            HiveEndPoint endPoint,
                            HiveConf conf,
                            StreamingConnection conn)
                     throws ClassNotFoundException,
                            ConnectionError,
                            SerializationError,
                            InvalidColumn,
                            StreamingException

Deprecated.

Constructor. Uses default separator of the LazySimpleSerde

Parameters:

colNamesForFields - Column name assignment for input fields. nulls or empty strings in the array indicates the fields to be skipped

delimiter - input field delimiter

endPoint - Hive endpoint

conf - a Hive conf object. Can be null if not using advanced hive settings.

Throws:

ConnectionError - Problem talking to Hive

ClassNotFoundException - Serde class not found

SerializationError - Serde initialization/interaction failed

StreamingException - Problem acquiring file system path for partition

InvalidColumn - any element in colNamesForFields refers to a non existing column

DelimitedInputWriter

public DelimitedInputWriter(String[] colNamesForFields,
                            String delimiter,
                            HiveEndPoint endPoint,
                            HiveConf conf,
                            char serdeSeparator,
                            StreamingConnection conn)
                     throws ClassNotFoundException,
                            ConnectionError,
                            SerializationError,
                            InvalidColumn,
                            StreamingException

Deprecated.

Constructor. Allows overriding separator of the LazySimpleSerde

Parameters:

colNamesForFields - Column name assignment for input fields

delimiter - input field delimiter

endPoint - Hive endpoint

conf - a Hive conf object. Set to null if not using advanced hive settings.

serdeSeparator - separator used when encoding data that is fed into the LazySimpleSerde. Ensure this separator does not occur in the field data

conn - connection this Writer is to be used with

Throws:

ConnectionError - Problem talking to Hive

ClassNotFoundException - Serde class not found

SerializationError - Serde initialization/interaction failed

StreamingException - Problem acquiring file system path for partition

InvalidColumn - any element in colNamesForFields refers to a non existing column

DelimitedInputWriter

public DelimitedInputWriter(String[] colNamesForFields,
                            String delimiter,
                            HiveEndPoint endPoint)
                     throws ClassNotFoundException,
                            ConnectionError,
                            SerializationError,
                            InvalidColumn,
                            StreamingException

Deprecated. As of release 1.3/2.1. Replaced by DelimitedInputWriter(String[], String, HiveEndPoint, StreamingConnection)

Throws:

ClassNotFoundException

ConnectionError

SerializationError

InvalidColumn

StreamingException

DelimitedInputWriter

public DelimitedInputWriter(String[] colNamesForFields,
                            String delimiter,
                            HiveEndPoint endPoint,
                            HiveConf conf)
                     throws ClassNotFoundException,
                            ConnectionError,
                            SerializationError,
                            InvalidColumn,
                            StreamingException

Deprecated. As of release 1.3/2.1. Replaced by DelimitedInputWriter(String[], String, HiveEndPoint, HiveConf, StreamingConnection)

Throws:

ClassNotFoundException

ConnectionError

SerializationError

InvalidColumn

StreamingException

DelimitedInputWriter

public DelimitedInputWriter(String[] colNamesForFields,
                            String delimiter,
                            HiveEndPoint endPoint,
                            HiveConf conf,
                            char serdeSeparator)
                     throws ClassNotFoundException,
                            StreamingException

Deprecated. As of release 1.3/2.1. Replaced by DelimitedInputWriter(String[], String, HiveEndPoint, HiveConf, char, StreamingConnection)

Throws:

ClassNotFoundException

StreamingException

Method Detail

reorderFields

protected byte[] reorderFields(byte[] record)
                        throws UnsupportedEncodingException

Deprecated.

Throws:

UnsupportedEncodingException

getTableColumns

protected ArrayList<String> getTableColumns()

Deprecated.

write

public void write(long writeId,
                  byte[] record)
           throws SerializationError,
                  StreamingIOFailure

Deprecated.

Description copied from interface: RecordWriter

Writes using a hive RecordUpdater

Parameters:

writeId - the write ID of the table mapping to Txn in which the write occurs

record - the record to be written

Throws:

SerializationError

StreamingIOFailure

getSerde

public AbstractSerDe getSerde()

Deprecated.

Description copied from class: AbstractRecordWriter

Get the SerDe for the Objects created by AbstractRecordWriter.encode(byte[]). This is public so that test frameworks can use it.

Specified by:

getSerde in class AbstractRecordWriter

Returns:

serde

getRecordObjectInspector

protected LazySimpleStructObjectInspector getRecordObjectInspector()

Deprecated.

Specified by:

getRecordObjectInspector in class AbstractRecordWriter

getBucketStructFields

protected StructField[] getBucketStructFields()

Deprecated.

Specified by:

getBucketStructFields in class AbstractRecordWriter

getBucketObjectInspectors

protected ObjectInspector[] getBucketObjectInspectors()

Deprecated.

Specified by:

getBucketObjectInspectors in class AbstractRecordWriter

encode

public Object encode(byte[] record)
              throws SerializationError

Deprecated.

Description copied from class: AbstractRecordWriter

Encode a record as an Object that Hive can read with the ObjectInspector associated with the serde returned by AbstractRecordWriter.getSerde(). This is public so that test frameworks can use it.

Specified by:

encode in class AbstractRecordWriter

Parameters:

record - record to be deserialized

Returns:

deserialized record as an Object

Throws:

SerializationError

createSerde

protected static LazySimpleSerDe createSerde(org.apache.hadoop.hive.metastore.api.Table tbl,
                                             HiveConf conf,
                                             char serdeSeparator)
                                      throws SerializationError

Deprecated.

Creates LazySimpleSerde

Parameters:

tbl -

Returns:

Throws:

SerializationError - if serde could not be initialized

getSerdeSeparator

public char getSerdeSeparator()

Deprecated.