org.apache.hadoop.hive.ql.exec.vector

Class VectorizedRowBatchCtx

java.lang.Object

org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatchCtx

public class VectorizedRowBatchCtx
extends Object

Context for Vectorized row batch. this class does eager deserialization of row data using serde in the RecordReader layer. It has supports partitions in this layer so that the vectorized batch is populated correctly with the partition column.

Constructor Summary

Constructors

Constructor and Description
VectorizedRowBatchCtx() Constructor for VectorizedRowBatchCtx
VectorizedRowBatchCtx(String[] rowColumnNames, TypeInfo[] rowColumnTypeInfos, org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] rowDataTypePhysicalVariations, int[] dataColumnNums, int partitionColumnCount, int virtualColumnCount, VirtualColumn[] neededVirtualColumns, String[] scratchColumnTypeNames, org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] scratchDataTypePhysicalVariations)

Method Summary

Modifier and Type	Method and Description
void	addPartitionColsToBatch(org.apache.hadoop.hive.ql.exec.vector.ColumnVector[] cols, Object[] partitionValues)
void	addPartitionColsToBatch(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch batch, Object[] partitionValues) Add the partition values to the batch
org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch	createVectorizedRowBatch() Creates a Vectorized row batch and the column vectors.
int	findVirtualColumnNum(VirtualColumn virtualColumn)
int	getDataColumnCount()
int[]	getDataColumnNums()
VirtualColumn[]	getNeededVirtualColumns()
int	getPartitionColumnCount()
static void	getPartitionValues(VectorizedRowBatchCtx vrbCtx, org.apache.hadoop.conf.Configuration hiveConf, org.apache.hadoop.mapred.FileSplit split, Object[] partitionValues)
static void	getPartitionValues(VectorizedRowBatchCtx vrbCtx, MapWork mapWork, org.apache.hadoop.mapred.FileSplit split, Object[] partitionValues)
static void	getPartitionValues(VectorizedRowBatchCtx vrbCtx, PartitionDesc partDesc, Object[] partitionValues)
String[]	getRowColumnNames()
TypeInfo[]	getRowColumnTypeInfos()
org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[]	getRowdataTypePhysicalVariations()
String[]	getScratchColumnTypeNames()
org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[]	getScratchDataTypePhysicalVariations()
int	getVirtualColumnCount()
void	init(StructObjectInspector structObjectInspector, String[] scratchColumnTypeNames) Initializes the VectorizedRowBatch context based on an scratch column type names and object inspector.
void	init(StructObjectInspector structObjectInspector, String[] scratchColumnTypeNames, org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] scratchDataTypePhysicalVariations) Initializes the VectorizedRowBatch context based on an scratch column type names and object inspector.
boolean	isPartitionCol(int colNum) Determine whether a given column is a partition column
boolean	isVirtualColumnNeeded(String virtualColumnName)
void	setRowDataTypePhysicalVariations(org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] rowDataTypePhysicalVariations)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VectorizedRowBatchCtx

public VectorizedRowBatchCtx()

Constructor for VectorizedRowBatchCtx

VectorizedRowBatchCtx

public VectorizedRowBatchCtx(String[] rowColumnNames,
                             TypeInfo[] rowColumnTypeInfos,
                             org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] rowDataTypePhysicalVariations,
                             int[] dataColumnNums,
                             int partitionColumnCount,
                             int virtualColumnCount,
                             VirtualColumn[] neededVirtualColumns,
                             String[] scratchColumnTypeNames,
                             org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] scratchDataTypePhysicalVariations)

Method Detail

getRowColumnNames

public String[] getRowColumnNames()

getRowColumnTypeInfos

public TypeInfo[] getRowColumnTypeInfos()

getRowdataTypePhysicalVariations

public org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] getRowdataTypePhysicalVariations()

setRowDataTypePhysicalVariations

public void setRowDataTypePhysicalVariations(org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] rowDataTypePhysicalVariations)

getDataColumnNums

public int[] getDataColumnNums()

getDataColumnCount

public int getDataColumnCount()

getPartitionColumnCount

public int getPartitionColumnCount()

getVirtualColumnCount

public int getVirtualColumnCount()

getNeededVirtualColumns

public VirtualColumn[] getNeededVirtualColumns()

isVirtualColumnNeeded

public boolean isVirtualColumnNeeded(String virtualColumnName)

findVirtualColumnNum

public int findVirtualColumnNum(VirtualColumn virtualColumn)

getScratchColumnTypeNames

public String[] getScratchColumnTypeNames()

getScratchDataTypePhysicalVariations

public org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] getScratchDataTypePhysicalVariations()

init

public void init(StructObjectInspector structObjectInspector,
                 String[] scratchColumnTypeNames)
          throws HiveException

Initializes the VectorizedRowBatch context based on an scratch column type names and object inspector.

Parameters:

structObjectInspector -

scratchColumnTypeNames - Object inspector that shapes the column types

Throws:

HiveException

init

public void init(StructObjectInspector structObjectInspector,
                 String[] scratchColumnTypeNames,
                 org.apache.hadoop.hive.common.type.DataTypePhysicalVariation[] scratchDataTypePhysicalVariations)
          throws HiveException

Initializes the VectorizedRowBatch context based on an scratch column type names and object inspector.

Parameters:

structObjectInspector -

scratchColumnTypeNames - Object inspector that shapes the column types

Throws:

HiveException

getPartitionValues

public static void getPartitionValues(VectorizedRowBatchCtx vrbCtx,
                                      org.apache.hadoop.conf.Configuration hiveConf,
                                      org.apache.hadoop.mapred.FileSplit split,
                                      Object[] partitionValues)
                               throws IOException

Throws:

IOException

getPartitionValues

public static void getPartitionValues(VectorizedRowBatchCtx vrbCtx,
                                      MapWork mapWork,
                                      org.apache.hadoop.mapred.FileSplit split,
                                      Object[] partitionValues)
                               throws IOException

Throws:

IOException

getPartitionValues

public static void getPartitionValues(VectorizedRowBatchCtx vrbCtx,
                                      PartitionDesc partDesc,
                                      Object[] partitionValues)

createVectorizedRowBatch

public org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch createVectorizedRowBatch()

Creates a Vectorized row batch and the column vectors.

Returns:

VectorizedRowBatch

Throws:

HiveException

addPartitionColsToBatch

public void addPartitionColsToBatch(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch batch,
                                    Object[] partitionValues)

Add the partition values to the batch

Parameters:

batch -

partitionValues -

Throws:

HiveException

addPartitionColsToBatch

public void addPartitionColsToBatch(org.apache.hadoop.hive.ql.exec.vector.ColumnVector[] cols,
                                    Object[] partitionValues)

isPartitionCol

public final boolean isPartitionCol(int colNum)

Determine whether a given column is a partition column

Parameters:

colNum - column number in VectorizedRowBatchs created by this context.

Returns:

true if it is a partition column, false otherwise