org.apache.hadoop.hive.ql.exec

Class MuxOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<MuxDesc>

org.apache.hadoop.hive.ql.exec.MuxOperator

All Implemented Interfaces:

Serializable, Cloneable, Node

public class MuxOperator
extends Operator<MuxDesc>
implements Serializable

MuxOperator is used in the Reduce side of MapReduce jobs optimized by Correlation Optimizer. Correlation Optimizer will remove unnecessary ReduceSinkOperaotrs, and MuxOperators are used to replace those ReduceSinkOperaotrs. Example: The original operator tree is ... JOIN2 / \ RS4 RS5 / \ GBY1 JOIN1 | / \ RS1 RS2 RS3 If GBY1, JOIN1, and JOIN2 can be executed in the same reducer (optimized by Correlation Optimizer). The new operator tree will be ... JOIN2 | MUX / \ GBY1 JOIN1 \ / DEMUX / | \ / | \ / | \ RS1 RS2 RS3 A MuxOperator has two functions. First, it will construct key, value and tag structure for the input of Join Operators. Second, it is a part of operator coordination mechanism which makes sure the operator tree in the Reducer can work correctly.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	MuxOperator.Handler Handler is used to construct the key-value structure.

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.Counter, Operator.OperatorFunc, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

abortOp, alias, asyncInitOperations, bucketingVersion, cContext, childOperators, childOperatorsArray, childOperatorsTag, conf, CONTEXT_NAME_KEY, done, groupKeyObject, HIVE_COUNTER_CREATED_DYNAMIC_PARTITIONS, HIVE_COUNTER_CREATED_FILES, HIVE_COUNTER_FATAL, id, indexForTezUnion, inputObjInspectors, numRows, operatorId, out, outputObjInspector, parentOperators, reporter, runTimeNumRows, state, statsMap

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MuxOperator() Kryo ctor.
	MuxOperator(CompilationOpContext ctx)

Method Summary

Modifier and Type	Method and Description
protected void	closeOp(boolean abort) Operator specific close routine.
void	endGroup()
void	forward(Object row, ObjectInspector rowInspector)
String	getName() Gets the name of the node.
static String	getOperatorName()
org.apache.hadoop.hive.ql.plan.api.OperatorType	getType() Return the type of the specific operator among the types in OperatorType.
protected void	initializeChildren(org.apache.hadoop.conf.Configuration hconf) Calls initialize on each of the children with outputObjetInspector as the output row format.
protected void	initializeOp(org.apache.hadoop.conf.Configuration hconf) Operator specific initialization.
boolean	logicalEquals(Operator other) Decides whether two operators are logically the same.
void	process(Object row, int tag) Process the row.
void	processGroup(int tag)
void	startGroup()

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

abort, acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, augmentPlan, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, cloneOp, cloneRecursiveChildren, close, columnNamesRowResolvedCanBeObtained, completeInitializationOp, createDummy, defaultEndGroup, defaultStartGroup, dump, dump, flush, flushRecursive, forward, forward, getAdditionalCounters, getBucketingVersion, getChildOperators, getChildren, getColumnExprMap, getCompilationOpContext, getConf, getConfiguration, getCounterName, getDone, getExecContext, getGroupKeyObject, getIdentifier, getIndexForTezUnion, getInputObjInspectors, getIsReduceSink, getMarker, getNextCntr, getNumChild, getNumParent, getOperatorId, getOpTraits, getOutputObjInspector, getParentOperators, getReduceOutputName, getSchema, getStatistics, getStats, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logicalEqualsTree, logStats, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, removeChild, removeChildAndAdoptItsChildren, removeParent, removeParents, replaceChild, replaceParent, reset, setAlias, setBucketingVersion, setChildOperators, setColumnExprMap, setCompilationOpContext, setConf, setDone, setExecContext, setGroupKeyObject, setIndexForTezUnion, setInputContext, setInputObjInspectors, setMarker, setNextVectorBatchGroupStatus, setOpTraits, setOutputCollector, setParentOperators, setReporter, setSchema, setStatistics, setUseBucketizedHiveInputFormat, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

MuxOperator

protected MuxOperator()

Kryo ctor.

MuxOperator

public MuxOperator(CompilationOpContext ctx)

Method Detail

initializeOp

protected void initializeOp(org.apache.hadoop.conf.Configuration hconf)
                     throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class Operator<MuxDesc>

Throws:

HiveException

initializeChildren

protected void initializeChildren(org.apache.hadoop.conf.Configuration hconf)
                           throws HiveException

Calls initialize on each of the children with outputObjetInspector as the output row format.

Overrides:

initializeChildren in class Operator<MuxDesc>

Throws:

HiveException

process

public void process(Object row,
                    int tag)
             throws HiveException

Description copied from class: Operator

Process the row.

Specified by:

process in class Operator<MuxDesc>

Parameters:

row - The object representing the row.

tag - The tag of the row usually means which parent this row comes from. Rows with the same tag should have exactly the same rowInspector all the time.

Throws:

HiveException

forward

public void forward(Object row,
                    ObjectInspector rowInspector)
             throws HiveException

Overrides:

forward in class Operator<MuxDesc>

Throws:

HiveException

startGroup

public void startGroup()
                throws HiveException

Overrides:

startGroup in class Operator<MuxDesc>

Throws:

HiveException

endGroup

public void endGroup()
              throws HiveException

Overrides:

endGroup in class Operator<MuxDesc>

Throws:

HiveException

processGroup

public void processGroup(int tag)
                  throws HiveException

Overrides:

processGroup in class Operator<MuxDesc>

Throws:

HiveException

closeOp

protected void closeOp(boolean abort)
                throws HiveException

Description copied from class: Operator

Operator specific close routine. Operators which inherents this class should overwrite this funtion for their specific cleanup routine.

Overrides:

closeOp in class Operator<MuxDesc>

Throws:

HiveException

getName

public String getName()

Description copied from interface: Node

Gets the name of the node. This is used in the rule dispatchers.

Specified by:

getName in interface Node

Specified by:

getName in class Operator<MuxDesc>

Returns:

the name of the operator

getOperatorName

public static String getOperatorName()

getType

public org.apache.hadoop.hive.ql.plan.api.OperatorType getType()

Description copied from class: Operator

Return the type of the specific operator among the types in OperatorType.

Specified by:

getType in class Operator<MuxDesc>

Returns:

OperatorType.*

logicalEquals

public boolean logicalEquals(Operator other)

Description copied from class: Operator

Decides whether two operators are logically the same. This can be used to merge same operators and avoid repeated computation.

Overrides:

logicalEquals in class Operator<MuxDesc>