org.apache.hadoop.hive.ql.io

Class AcidUtils.MetaDataFile

java.lang.Object

org.apache.hadoop.hive.ql.io.AcidUtils.MetaDataFile

Enclosing class:

AcidUtils

public static class AcidUtils.MetaDataFile
extends Object

General facility to place a metadta file into a dir created by acid/compactor write. Load Data commands against Acid tables write AcidUtils.AcidBaseFileType.ORIGINAL_BASE type files into delta_x_x/ (or base_x in case there is Overwrite clause). AcidUtils.MetaDataFile is a small JSON file in this directory that indicates that these files don't have Acid metadata columns and so the values for these columns need to be assigned at read time/compaction.

Constructor Summary

Constructors

Constructor and Description
MetaDataFile()

Method Summary

Modifier and Type	Method and Description
static void	createCompactorMarker(org.apache.hadoop.fs.Path baseOrDeltaDir, org.apache.hadoop.fs.FileSystem fs)
static boolean	isRawFormat(org.apache.hadoop.fs.Path baseOrDeltaDir, org.apache.hadoop.fs.FileSystem fs) Checks if the files in base/delta dir are a result of Load Data statement and thus do not have ROW_IDs embedded in the data.
static boolean	isRawFormatFile(org.apache.hadoop.fs.Path dataFile, org.apache.hadoop.fs.FileSystem fs)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MetaDataFile

public MetaDataFile()

Method Detail

createCompactorMarker

public static void createCompactorMarker(org.apache.hadoop.fs.Path baseOrDeltaDir,
                                         org.apache.hadoop.fs.FileSystem fs)
                                  throws IOException

Parameters:

baseOrDeltaDir - detla or base dir, must exist

Throws:

IOException

isRawFormat

public static boolean isRawFormat(org.apache.hadoop.fs.Path baseOrDeltaDir,
                                  org.apache.hadoop.fs.FileSystem fs)
                           throws IOException

Checks if the files in base/delta dir are a result of Load Data statement and thus do not have ROW_IDs embedded in the data.

Parameters:

baseOrDeltaDir - base or delta file.

Throws:

IOException

isRawFormatFile

public static boolean isRawFormatFile(org.apache.hadoop.fs.Path dataFile,
                                      org.apache.hadoop.fs.FileSystem fs)
                               throws IOException

Throws:

IOException