org.apache.hadoop.hive.serde2.lazybinary

Class LazyBinarySerDe

java.lang.Object

org.apache.hadoop.hive.serde2.AbstractSerDe

org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe

All Implemented Interfaces:

Deserializer, Serializer

Direct Known Subclasses:

LazyBinarySerDe2

public class LazyBinarySerDe
extends AbstractSerDe

The LazyBinarySerDe class combines the lazy property of LazySimpleSerDe class and the binary property of BinarySortable class. Lazy means a field is not deserialized until required. Binary means a field is serialized in binary compact format.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LazyBinarySerDe.BooleanRef
static class	LazyBinarySerDe.StringWrapper

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hive.serde2.AbstractSerDe

configErrors

Constructor Summary

Constructors

Constructor and Description
LazyBinarySerDe()

Method Summary

Modifier and Type	Method and Description
Object	deserialize(org.apache.hadoop.io.Writable field) Deserialize a table record to a lazybinary struct.
ObjectInspector	getObjectInspector() Returns the ObjectInspector for the row.
SerDeStats	getSerDeStats() Returns the statistics after (de)serialization)
Class<? extends org.apache.hadoop.io.Writable>	getSerializedClass() Returns the Writable Class after serialization.
void	initialize(org.apache.hadoop.conf.Configuration conf, Properties tbl) Initialize the SerDe with configuration and table information.
static void	serialize(ByteStream.RandomAccessOutput byteStream, Object obj, ObjectInspector objInspector, boolean skipLengthPrefix, LazyBinarySerDe.BooleanRef warnedOnceNullMapKey) A recursive function that serialize an object to a byte buffer based on its object inspector.
org.apache.hadoop.io.Writable	serialize(Object obj, ObjectInspector objInspector) Serialize an object to a byte buffer in a binary compact way.
static void	serializeStruct(ByteStream.RandomAccessOutput byteStream, Object[] fieldData, List<ObjectInspector> fieldOis)
protected static void	serializeText(ByteStream.RandomAccessOutput byteStream, org.apache.hadoop.io.Text t, boolean skipLengthPrefix)
static void	setFromBigIntegerBytesAndScale(byte[] bytes, int offset, int length, org.apache.hadoop.hive.serde2.io.HiveDecimalWritable dec)
static void	writeDateToByteStream(ByteStream.RandomAccessOutput byteStream, DateWritableV2 date)
protected static void	writeSizeAtOffset(ByteStream.RandomAccessOutput byteStream, int byteSizeStart, int size)
static void	writeToByteStream(ByteStream.RandomAccessOutput byteStream, org.apache.hadoop.hive.common.type.HiveDecimal dec, long[] scratchLongs, byte[] scratchBytes) Allocate scratchLongs with HiveDecimal.SCRATCH_LONGS_LEN longs.
static void	writeToByteStream(ByteStream.RandomAccessOutput byteStream, org.apache.hadoop.hive.serde2.io.HiveDecimalWritable decWritable)
static void	writeToByteStream(ByteStream.RandomAccessOutput byteStream, org.apache.hadoop.hive.serde2.io.HiveDecimalWritable decWritable, long[] scratchLongs, byte[] scratchBytes) Allocate scratchLongs with HiveDecimal.SCRATCH_LONGS_LEN longs.

Methods inherited from class org.apache.hadoop.hive.serde2.AbstractSerDe

getConfigurationErrors, initialize, shouldStoreFieldsInMetastore

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

LazyBinarySerDe

public LazyBinarySerDe()
                throws SerDeException

Throws:

SerDeException

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf,
                       Properties tbl)
                throws SerDeException

Initialize the SerDe with configuration and table information.

Specified by:

initialize in interface Deserializer

Specified by:

initialize in interface Serializer

Specified by:

initialize in class AbstractSerDe

Parameters:

conf - System properties. Can be null in compile time

tbl - table properties

Throws:

SerDeException

getObjectInspector

public ObjectInspector getObjectInspector()
                                   throws SerDeException

Returns the ObjectInspector for the row.

Specified by:

getObjectInspector in interface Deserializer

Specified by:

getObjectInspector in class AbstractSerDe

Throws:

SerDeException

getSerializedClass

public Class<? extends org.apache.hadoop.io.Writable> getSerializedClass()

Returns the Writable Class after serialization.

Specified by:

getSerializedClass in interface Serializer

Specified by:

getSerializedClass in class AbstractSerDe

deserialize

public Object deserialize(org.apache.hadoop.io.Writable field)
                   throws SerDeException

Deserialize a table record to a lazybinary struct.

Specified by:

deserialize in interface Deserializer

Specified by:

deserialize in class AbstractSerDe

Parameters:

field - The Writable object containing a serialized object

Returns:

A Java object representing the contents in the blob.

Throws:

SerDeException

serialize

public org.apache.hadoop.io.Writable serialize(Object obj,
                                               ObjectInspector objInspector)
                                        throws SerDeException

Serialize an object to a byte buffer in a binary compact way.

Specified by:

serialize in interface Serializer

Specified by:

serialize in class AbstractSerDe

Throws:

SerDeException

serializeStruct

public static void serializeStruct(ByteStream.RandomAccessOutput byteStream,
                                   Object[] fieldData,
                                   List<ObjectInspector> fieldOis)
                            throws SerDeException

Throws:

SerDeException

serializeText

protected static void serializeText(ByteStream.RandomAccessOutput byteStream,
                                    org.apache.hadoop.io.Text t,
                                    boolean skipLengthPrefix)

writeDateToByteStream

public static void writeDateToByteStream(ByteStream.RandomAccessOutput byteStream,
                                         DateWritableV2 date)

setFromBigIntegerBytesAndScale

public static void setFromBigIntegerBytesAndScale(byte[] bytes,
                                                  int offset,
                                                  int length,
                                                  org.apache.hadoop.hive.serde2.io.HiveDecimalWritable dec)

writeToByteStream

public static void writeToByteStream(ByteStream.RandomAccessOutput byteStream,
                                     org.apache.hadoop.hive.serde2.io.HiveDecimalWritable decWritable)

writeToByteStream

public static void writeToByteStream(ByteStream.RandomAccessOutput byteStream,
                                     org.apache.hadoop.hive.common.type.HiveDecimal dec,
                                     long[] scratchLongs,
                                     byte[] scratchBytes)

Allocate scratchLongs with HiveDecimal.SCRATCH_LONGS_LEN longs. And, allocate scratch buffer with HiveDecimal.SCRATCH_BUFFER_LEN_BIG_INTEGER_BYTES bytes.

Parameters:

byteStream -

dec -

scratchLongs -

scratchBytes -

writeToByteStream

public static void writeToByteStream(ByteStream.RandomAccessOutput byteStream,
                                     org.apache.hadoop.hive.serde2.io.HiveDecimalWritable decWritable,
                                     long[] scratchLongs,
                                     byte[] scratchBytes)

Allocate scratchLongs with HiveDecimal.SCRATCH_LONGS_LEN longs. And, allocate scratch buffer with HiveDecimal.SCRATCH_BUFFER_LEN_BIG_INTEGER_BYTES bytes.

Parameters:

byteStream -

decWritable -

scratchLongs -

scratchBytes -

serialize

public static void serialize(ByteStream.RandomAccessOutput byteStream,
                             Object obj,
                             ObjectInspector objInspector,
                             boolean skipLengthPrefix,
                             LazyBinarySerDe.BooleanRef warnedOnceNullMapKey)
                      throws SerDeException

A recursive function that serialize an object to a byte buffer based on its object inspector.

Parameters:

byteStream - the byte stream storing the serialization data

obj - the object to serialize

objInspector - the object inspector

skipLengthPrefix - a boolean indicating whether length prefix is needed for list/map/struct

warnedOnceNullMapKey - a boolean indicating whether a warning has been issued once already when encountering null map keys

Throws:

SerDeException

writeSizeAtOffset

protected static void writeSizeAtOffset(ByteStream.RandomAccessOutput byteStream,
                                        int byteSizeStart,
                                        int size)

getSerDeStats

public SerDeStats getSerDeStats()

Returns the statistics after (de)serialization)

Specified by:

getSerDeStats in interface Deserializer

Specified by:

getSerDeStats in interface Serializer

Specified by:

getSerDeStats in class AbstractSerDe