org.apache.hadoop.hive.ql.io.parquet.vector

Class VectorizedPrimitiveColumnReader

java.lang.Object

org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedPrimitiveColumnReader

All Implemented Interfaces:

VectorizedColumnReader

public class VectorizedPrimitiveColumnReader
extends BaseVectorizedColumnReader

It's column level Parquet reader which is used to read a batch of records for a column, part of the code is referred from Apache Spark and Apache Parquet.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

BaseVectorizedColumnReader.NullIntIterator, BaseVectorizedColumnReader.RLEIntIterator, BaseVectorizedColumnReader.ValuesReaderIntIterator

Field Summary

Fields inherited from class org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

dataColumn, definitionLevel, definitionLevelColumn, descriptor, dictionary, endOfPageValueCount, hiveType, isCurrentPageDictionaryEncoded, maxDefLevel, pageReader, pageValueCount, repetitionLevel, repetitionLevelColumn, skipTimestampConversion, type, valuesRead

Constructor Summary

Constructors

Constructor and Description
VectorizedPrimitiveColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.column.page.PageReader pageReader, boolean skipTimestampConversion, org.apache.parquet.schema.Type type, TypeInfo hiveType)

Method Summary

Modifier and Type	Method and Description
void	readBatch(int total, org.apache.hadoop.hive.ql.exec.vector.ColumnVector column, TypeInfo columnType) read records with specified size and type into the columnVector

Methods inherited from class org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

decimalTypeCheck, readPage, readRepetitionAndDefinitionLevels

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VectorizedPrimitiveColumnReader

public VectorizedPrimitiveColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor,
                                       org.apache.parquet.column.page.PageReader pageReader,
                                       boolean skipTimestampConversion,
                                       org.apache.parquet.schema.Type type,
                                       TypeInfo hiveType)
                                throws IOException

Throws:

IOException

Method Detail

readBatch

public void readBatch(int total,
                      org.apache.hadoop.hive.ql.exec.vector.ColumnVector column,
                      TypeInfo columnType)
               throws IOException

Description copied from interface: VectorizedColumnReader

read records with specified size and type into the columnVector

Parameters:

total - number of records to read into the column vector

column - column vector where the reader will read data into

columnType - the type of column vector

Throws:

IOException