Package org.apache.hadoop.hive.ql.io.orc

The Optimized Row Columnar (ORC) File Format.

See: Description

Interface Summary

Interface	Description
ExternalCache.ExternalFooterCachesByConf	An abstraction for testing ExternalCache in OrcInputFormat.
ExternalCache.ExternalFooterCachesByConf.Cache
OrcInputFormat.FooterCache	Represents footer cache.
Reader	The interface for reading ORC files.
RecordReader	A row-by-row iterator for ORC files.
VectorizedOrcAcidRowBatchReader.DeleteEventRegistry	An interface that can determine which rows have been deleted from a given vectorized row batch.
Writer	The HIVE interface for writing ORC files.

Class Summary

Class	Description
ExternalCache	Metastore-based footer cache storing serialized footers.
FixAcidKeyIndex	Utility to check and fix the ACID key index of an ORC file if it has been written incorrectly due to HIVE-18817.
MetastoreExternalCachesByConf	An implementation of external cache and factory based on metastore.
MetastoreExternalCachesByConf.HBaseCache
OrcFile	Contains factory methods to read or write ORC files.
OrcFile.ReaderOptions
OrcFile.WriterOptions	Options for creating ORC file writers.
OrcFileFormatProxy	File format proxy for ORC.
OrcFileKeyWrapper	Key for OrcFileMergeMapper task.
OrcFileStripeMergeInputFormat
OrcFileStripeMergeRecordReader
OrcFileValueWrapper	Value for OrcFileMergeMapper.
OrcInputFormat	A MapReduce/Hive input format for ORC files.
OrcInputFormat.ContextFactory
OrcInputFormat.FooterCacheKey
OrcInputFormat.NullKeyRecordReader	Return a RecordReader that is compatible with the Hive 0.12 reader with NullWritable for the key instead of RecordIdentifier.
OrcNewInputFormat	An InputFormat for ORC files.
OrcNewOutputFormat	An OutputFormat that writes ORC files.
OrcNewSplit	OrcFileSplit.
OrcOiBatchToRowReader	BatchToRowReader that returns the rows readable by ORC IOs.
OrcOutputFormat	A Hive OutputFormat for ORC files.
OrcRawRecordMerger	Merges a base and a list of delta files together into a single stream of events.
OrcRawRecordMerger.ReaderKey	A RecordIdentifier extended with the current write id.
OrcRecordUpdater	A RecordUpdater where the files are stored as ORC.
OrcSerde	A serde class for ORC.
OrcSplit	OrcFileSplit.
OrcStruct
ReaderImpl
RecordReaderImpl
VectorizedOrcAcidRowBatchReader	A fast vectorized batch reader class for ACID.
VectorizedOrcInputFormat	A MapReduce/Hive input format for ORC files.
VectorizedOrcSerde	A serde class for ORC.
WriterImpl	An ORC file writer.

Enum Summary

Enum	Description
CompressionKind	An enumeration that lists the generic compression algorithms that can be applied to ORC files.

Package org.apache.hadoop.hive.ql.io.orc Description

The Optimized Row Columnar (ORC) File Format. This format:

• Decomposes complex column types into primitives
• Uses type-specific encoders for each column
  • Dictionary encodings for low cardinality columns
  • Run length encoding of data
  • variable length encoding of integers
• Divides file into large stripes
• Each stripe includes light-weight indexes that enable the reader to skip large sets of rows that don't satisfy the filter condition
• A file footer that contains meta-information about file
  • Precise byte range for each stripe
  • Type information for the file
  • Any user meta-information
• Seek to row number is implemented to support secondary indexes
• Support for additional generic compression: LZO, SNAPPY, ZLIB.

Format:

 
 HEADER (3 bytes) "ORC"
 STRIPE (0 or more stripes)
 FILE-FOOTER
 POST SCRIPT
 PS LENGTH (1 byte)
 
 

Stripe:

 
 INDEX-STREAM (0 or more)
 DATA-STREAM (0 or more)
 STRIPE-FOOTER