org.apache.hadoop.hive.ql.exec

Class LateralViewJoinOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<LateralViewJoinDesc>

org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator

All Implemented Interfaces:

Serializable, Cloneable, Node

public class LateralViewJoinOperator
extends Operator<LateralViewJoinDesc>

The lateral view join operator is used for FROM src LATERAL VIEW udtf()... This operator was implemented with the following operator DAG in mind. For a query such as SELECT pageid, adid.* FROM example_table LATERAL VIEW explode(adid_list) AS adid The top of the operator DAG will look similar to [Table Scan] | [Lateral View Forward] / \ [Select](*) [Select](adid_list) | | | [UDTF] (explode) \ / [Lateral View Join] | | [Select] (pageid, adid.*) | .... Rows from the table scan operator are first to a lateral view forward operator that just forwards the row and marks the start of a LV. The select operator on the left picks all the columns while the select operator on the right picks only the columns needed by the UDTF. The output of select in the left branch and output of the UDTF in the right branch are then sent to the lateral view join (LVJ). In most cases, the UDTF will generate > 1 row for every row received from the TS, while the left select operator will generate only one. For each row output from the TS, the LVJ outputs all possible rows that can be created by joining the row from the left select and one of the rows output from the UDTF. Additional lateral views can be supported by adding a similar DAG after the previous LVJ operator.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.Counter, Operator.OperatorFunc, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
static byte	SELECT_TAG
static byte	UDTF_TAG

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

abortOp, alias, asyncInitOperations, bucketingVersion, cContext, childOperators, childOperatorsArray, childOperatorsTag, conf, CONTEXT_NAME_KEY, done, groupKeyObject, HIVE_COUNTER_CREATED_DYNAMIC_PARTITIONS, HIVE_COUNTER_CREATED_FILES, HIVE_COUNTER_FATAL, id, indexForTezUnion, inputObjInspectors, LOG, numRows, operatorId, out, outputObjInspector, parentOperators, reporter, runTimeNumRows, state, statsMap

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	LateralViewJoinOperator() Kryo ctor.
	LateralViewJoinOperator(CompilationOpContext ctx)

Method Summary

Modifier and Type	Method and Description
String	getName() Gets the name of the node.
static String	getOperatorName()
org.apache.hadoop.hive.ql.plan.api.OperatorType	getType() Return the type of the specific operator among the types in OperatorType.
protected void	initializeOp(org.apache.hadoop.conf.Configuration hconf) Operator specific initialization.
void	process(Object row, int tag) An important assumption for processOp() is that for a given row from the TS, the LVJ will first get the row from the left select operator, followed by all the corresponding rows from the UDTF operator.

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

abort, acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, augmentPlan, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, cloneOp, cloneRecursiveChildren, close, closeOp, columnNamesRowResolvedCanBeObtained, completeInitializationOp, createDummy, defaultEndGroup, defaultStartGroup, dump, dump, endGroup, flush, flushRecursive, forward, forward, forward, getAdditionalCounters, getBucketingVersion, getChildOperators, getChildren, getColumnExprMap, getCompilationOpContext, getConf, getConfiguration, getCounterName, getDone, getExecContext, getGroupKeyObject, getIdentifier, getIndexForTezUnion, getInputObjInspectors, getIsReduceSink, getMarker, getNextCntr, getNumChild, getNumParent, getOperatorId, getOpTraits, getOutputObjInspector, getParentOperators, getReduceOutputName, getSchema, getStatistics, getStats, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logicalEquals, logicalEqualsTree, logStats, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, processGroup, removeChild, removeChildAndAdoptItsChildren, removeParent, removeParents, replaceChild, replaceParent, reset, setAlias, setBucketingVersion, setChildOperators, setColumnExprMap, setCompilationOpContext, setConf, setDone, setExecContext, setGroupKeyObject, setIndexForTezUnion, setInputContext, setInputObjInspectors, setMarker, setNextVectorBatchGroupStatus, setOpTraits, setOutputCollector, setParentOperators, setReporter, setSchema, setStatistics, setUseBucketizedHiveInputFormat, startGroup, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

SELECT_TAG

public static final byte SELECT_TAG

See Also:

Constant Field Values

UDTF_TAG

public static final byte UDTF_TAG

See Also:

Constant Field Values

Constructor Detail

LateralViewJoinOperator

protected LateralViewJoinOperator()

Kryo ctor.

LateralViewJoinOperator

public LateralViewJoinOperator(CompilationOpContext ctx)

Method Detail

initializeOp

protected void initializeOp(org.apache.hadoop.conf.Configuration hconf)
                     throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class Operator<LateralViewJoinDesc>

Throws:

HiveException

process

public void process(Object row,
                    int tag)
             throws HiveException

An important assumption for processOp() is that for a given row from the TS, the LVJ will first get the row from the left select operator, followed by all the corresponding rows from the UDTF operator. And so on.

Specified by:

process in class Operator<LateralViewJoinDesc>

Parameters:

row - The object representing the row.

tag - The tag of the row usually means which parent this row comes from. Rows with the same tag should have exactly the same rowInspector all the time.

Throws:

HiveException

getName

public String getName()

Description copied from interface: Node

Gets the name of the node. This is used in the rule dispatchers.

Specified by:

getName in interface Node

Specified by:

getName in class Operator<LateralViewJoinDesc>

Returns:

String

getOperatorName

public static String getOperatorName()

getType

public org.apache.hadoop.hive.ql.plan.api.OperatorType getType()

Description copied from class: Operator

Return the type of the specific operator among the types in OperatorType.

Specified by:

getType in class Operator<LateralViewJoinDesc>

Returns:

OperatorType.*