org.apache.hadoop.hive.ql.exec.vector

Class VectorMapJoinBaseOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<T>

org.apache.hadoop.hive.ql.exec.CommonJoinOperator<T>

org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator<MapJoinDesc>

org.apache.hadoop.hive.ql.exec.MapJoinOperator

org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinBaseOperator

All Implemented Interfaces:

Serializable, Cloneable, VectorizationContextRegion, VectorizationOperator, Node

Direct Known Subclasses:

VectorMapJoinOperator, VectorMapJoinOuterFilteredOperator

public class VectorMapJoinBaseOperator
extends MapJoinOperator
implements VectorizationOperator, VectorizationContextRegion

The *NON-NATIVE* base vector map join operator class used by VectorMapJoinOperator and VectorMapJoinOuterFilteredOperator. It has common variables and code for the output batch, Hybrid Grace spill batch, and more.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.Counter, Operator.OperatorFunc, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch	outputBatch
protected Map<ObjectInspector,VectorAssignRow>	outputVectorAssignRowMap
protected org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch	scratchBatch
protected int	tag
protected VectorizationContext	vContext
protected VectorMapJoinDesc	vectorDesc
protected VectorizationContext	vOutContext
protected VectorizedRowBatchCtx	vrbCtx

Fields inherited from class org.apache.hadoop.hive.ql.exec.MapJoinOperator

firstSmallTable, hashMapRowGetters, isTestingNoHashTableLoad, loader, mapJoinTables, spilledMapJoinTables

Fields inherited from class org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator

emptyList, joinKeys, joinKeysObjectInspectors, posBigTable

Fields inherited from class org.apache.hadoop.hive.ql.exec.CommonJoinOperator

alias, aliasFilterTags, condn, countAfterReport, dummyObj, dummyObjVectors, filterMaps, filterTags, forwardCache, heartbeatInterval, intermediate, joinFilterObjectInspectors, joinFilters, joinValues, joinValuesObjectInspectors, joinValuesStandardObjectInspectors, needsPostEvaluation, noOuterJoin, NOTSKIPBIGTABLE, nullsafes, numAliases, offsets, order, residualJoinFilters, residualJoinFiltersOIs, rowContainerPostFilteredOuterJoin, rowContainerStandardObjectInspectors, skipVectors, spillTableDesc, totalSz

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

abortOp, asyncInitOperations, bucketingVersion, cContext, childOperators, childOperatorsArray, childOperatorsTag, conf, CONTEXT_NAME_KEY, done, groupKeyObject, HIVE_COUNTER_CREATED_DYNAMIC_PARTITIONS, HIVE_COUNTER_CREATED_FILES, HIVE_COUNTER_FATAL, id, indexForTezUnion, inputObjInspectors, numRows, operatorId, out, outputObjInspector, parentOperators, reporter, runTimeNumRows, state, statsMap

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	VectorMapJoinBaseOperator() Kryo ctor.
	VectorMapJoinBaseOperator(CompilationOpContext ctx)
	VectorMapJoinBaseOperator(CompilationOpContext ctx, OperatorDesc conf, VectorizationContext vContext, VectorDesc vectorDesc)

Method Summary

Modifier and Type	Method and Description
void	closeOp(boolean aborted) All done.
VectorizationContext	getInputVectorizationContext()
static TypeInfo[]	getOutputTypeInfos(MapJoinDesc desc)
VectorizationContext	getOutputVectorizationContext()
VectorDesc	getVectorDesc()
void	initializeOp(org.apache.hadoop.conf.Configuration hconf) Operator specific initialization.
protected void	internalForward(Object row, ObjectInspector outputOI) 'forwards' the (row-mode) record into the (vectorized) output batch
protected void	reProcessBigTable(int partitionId) For a vectorized row batch from the rows feed from the super MapJoinOperator.

Methods inherited from class org.apache.hadoop.hive.ql.exec.MapJoinOperator

canSkipJoinProcessing, canSkipReload, cleanUpInputFileChangedOp, completeInitializationOp, endGroup, generateMapMetaData, getCacheKey, getHashTableLoader, getName, getOperatorName, getRefKey, getType, getValueObjectInspectors, isInputFileChangeSensitive, loadHashTable, process, reloadHashTable, setMapJoinKey, setTestMapJoinTableContainer, spillBigTableRow, startGroup

Methods inherited from class org.apache.hadoop.hive.ql.exec.CommonJoinOperator

checkAndGenObject, getFilteredValue, getFilterTag, getNextSize, getPosToAliasMap, hasFilter, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, reportProgress, setPosToAliasMap

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

abort, acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, augmentPlan, cleanUpInputFileChanged, clone, cloneOp, cloneRecursiveChildren, close, columnNamesRowResolvedCanBeObtained, createDummy, defaultEndGroup, defaultStartGroup, dump, dump, flush, flushRecursive, forward, forward, forward, getAdditionalCounters, getBucketingVersion, getChildOperators, getChildren, getColumnExprMap, getCompilationOpContext, getConf, getConfiguration, getCounterName, getDone, getExecContext, getGroupKeyObject, getIdentifier, getIndexForTezUnion, getInputObjInspectors, getIsReduceSink, getMarker, getNextCntr, getNumChild, getNumParent, getOperatorId, getOpTraits, getOutputObjInspector, getParentOperators, getReduceOutputName, getSchema, getStatistics, getStats, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logicalEquals, logicalEqualsTree, logStats, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, processGroup, removeChild, removeChildAndAdoptItsChildren, removeParent, removeParents, replaceChild, replaceParent, reset, setAlias, setBucketingVersion, setChildOperators, setColumnExprMap, setCompilationOpContext, setConf, setDone, setExecContext, setGroupKeyObject, setIndexForTezUnion, setInputContext, setInputObjInspectors, setMarker, setNextVectorBatchGroupStatus, setOpTraits, setOutputCollector, setParentOperators, setReporter, setSchema, setStatistics, setUseBucketizedHiveInputFormat, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

vContext

protected VectorizationContext vContext

vectorDesc

protected VectorMapJoinDesc vectorDesc

vOutContext

protected VectorizationContext vOutContext

outputBatch

protected transient org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch outputBatch

scratchBatch

protected transient org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch scratchBatch

outputVectorAssignRowMap

protected transient Map<ObjectInspector,VectorAssignRow> outputVectorAssignRowMap

vrbCtx

protected transient VectorizedRowBatchCtx vrbCtx

tag

protected transient int tag

Constructor Detail

VectorMapJoinBaseOperator

protected VectorMapJoinBaseOperator()

Kryo ctor.

VectorMapJoinBaseOperator

public VectorMapJoinBaseOperator(CompilationOpContext ctx)

VectorMapJoinBaseOperator

public VectorMapJoinBaseOperator(CompilationOpContext ctx,
                                 OperatorDesc conf,
                                 VectorizationContext vContext,
                                 VectorDesc vectorDesc)
                          throws HiveException

Throws:

HiveException

Method Detail

getInputVectorizationContext

public VectorizationContext getInputVectorizationContext()

Specified by:

getInputVectorizationContext in interface VectorizationOperator

getOutputTypeInfos

public static TypeInfo[] getOutputTypeInfos(MapJoinDesc desc)

initializeOp

public void initializeOp(org.apache.hadoop.conf.Configuration hconf)
                  throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class MapJoinOperator

Throws:

HiveException

internalForward

protected void internalForward(Object row,
                               ObjectInspector outputOI)
                        throws HiveException

'forwards' the (row-mode) record into the (vectorized) output batch

Overrides:

internalForward in class CommonJoinOperator<MapJoinDesc>

Throws:

HiveException

closeOp

public void closeOp(boolean aborted)
             throws HiveException

Description copied from class: CommonJoinOperator

All done.

Overrides:

closeOp in class MapJoinOperator

Throws:

HiveException

reProcessBigTable

protected void reProcessBigTable(int partitionId)
                          throws HiveException

For a vectorized row batch from the rows feed from the super MapJoinOperator.

Overrides:

reProcessBigTable in class MapJoinOperator

Parameters:

partitionId - the partition from which to take out spilled big table rows

Throws:

HiveException

getOutputVectorizationContext

public VectorizationContext getOutputVectorizationContext()

Specified by:

getOutputVectorizationContext in interface VectorizationContextRegion

getVectorDesc

public VectorDesc getVectorDesc()

Specified by:

getVectorDesc in interface VectorizationOperator