org.apache.hadoop.hive.ql.exec.persistence

Class HashMapWrapper

java.lang.Object

org.apache.hadoop.hive.ql.exec.persistence.AbstractMapJoinTableContainer

org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper

All Implemented Interfaces:

Serializable, MemoryEstimate, MapJoinPersistableTableContainer, MapJoinTableContainer

public class HashMapWrapper
extends AbstractMapJoinTableContainer
implements Serializable

Simple wrapper for persistent Hashmap implementing only the put/get/remove/clear interface. The main memory hash table acts as a cache and all put/get will operate on it first. If the size of the main memory hash table exceeds a certain threshold, new elements will go into the persistent hash table.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hive.ql.exec.persistence.MapJoinTableContainer

MapJoinTableContainer.ReusableGetAdaptor

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hive.ql.exec.persistence.AbstractMapJoinTableContainer

LOAD_NAME, THESHOLD_NAME

Constructor Summary

Constructors

Constructor and Description
HashMapWrapper()
HashMapWrapper(org.apache.hadoop.conf.Configuration hconf, long keyCount)
HashMapWrapper(Map<String,String> metaData)

Method Summary

Modifier and Type	Method and Description
static int	calculateTableSize(float keyCountAdj, int threshold, float loadFactor, long keyCount)
void	clear() Clears the contents of the table.
MapJoinTableContainer.ReusableGetAdaptor	createGetter(MapJoinKey keyTypeFromLoader) Creates reusable get adaptor that can be used to retrieve rows from the table based on either vectorized or non-vectorized input rows to MapJoinOperator.
void	dumpMetrics()
Set<Map.Entry<MapJoinKey,MapJoinRowContainer>>	entrySet()
MapJoinRowContainer	get(MapJoinKey key)
MapJoinKey	getAnyKey()
long	getEstimatedMemorySize() Returns estimated memory size based JavaDataModel
boolean	hasSpill() Checks if the container has spilled any data onto disk.
void	put(MapJoinKey key, MapJoinRowContainer value)
MapJoinKey	putRow(org.apache.hadoop.io.Writable currentKey, org.apache.hadoop.io.Writable currentValue) Adds row from input to the table.
void	seal() Indicates to the container that the puts have ended; table is now r/o.
void	setSerde(MapJoinObjectSerDeContext keyCtx, MapJoinObjectSerDeContext valCtx)
int	size() Return the size of the hash table

Methods inherited from class org.apache.hadoop.hive.ql.exec.persistence.AbstractMapJoinTableContainer

createConstructorMetaData, getMetaData, putMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

HashMapWrapper

public HashMapWrapper(Map<String,String> metaData)

HashMapWrapper

public HashMapWrapper()

HashMapWrapper

public HashMapWrapper(org.apache.hadoop.conf.Configuration hconf,
                      long keyCount)

Method Detail

calculateTableSize

public static int calculateTableSize(float keyCountAdj,
                                     int threshold,
                                     float loadFactor,
                                     long keyCount)

get

public MapJoinRowContainer get(MapJoinKey key)

Specified by:

get in interface MapJoinPersistableTableContainer

put

public void put(MapJoinKey key,
                MapJoinRowContainer value)

Specified by:

put in interface MapJoinPersistableTableContainer

size

public int size()

Description copied from interface: MapJoinTableContainer

Return the size of the hash table

Specified by:

size in interface MapJoinPersistableTableContainer

Specified by:

size in interface MapJoinTableContainer

entrySet

public Set<Map.Entry<MapJoinKey,MapJoinRowContainer>> entrySet()

Specified by:

entrySet in interface MapJoinPersistableTableContainer

clear

public void clear()

Description copied from interface: MapJoinTableContainer

Clears the contents of the table.

Specified by:

clear in interface MapJoinTableContainer

putRow

public MapJoinKey putRow(org.apache.hadoop.io.Writable currentKey,
                         org.apache.hadoop.io.Writable currentValue)
                  throws SerDeException,
                         HiveException

Description copied from interface: MapJoinTableContainer

Adds row from input to the table.

Specified by:

putRow in interface MapJoinTableContainer

Throws:

SerDeException

HiveException

createGetter

public MapJoinTableContainer.ReusableGetAdaptor createGetter(MapJoinKey keyTypeFromLoader)

Description copied from interface: MapJoinTableContainer

Creates reusable get adaptor that can be used to retrieve rows from the table based on either vectorized or non-vectorized input rows to MapJoinOperator.

Specified by:

createGetter in interface MapJoinTableContainer

Parameters:

keyTypeFromLoader - Last key from hash table loader, to determine key type used when loading hashtable (if it can vary).

getEstimatedMemorySize

public long getEstimatedMemorySize()

Description copied from interface: MemoryEstimate

Returns estimated memory size based JavaDataModel

Specified by:

getEstimatedMemorySize in interface MemoryEstimate

Returns:

estimated memory size in bytes

seal

public void seal()

Description copied from interface: MapJoinTableContainer

Indicates to the container that the puts have ended; table is now r/o.

Specified by:

seal in interface MapJoinTableContainer

getAnyKey

public MapJoinKey getAnyKey()

Specified by:

getAnyKey in interface MapJoinTableContainer

dumpMetrics

public void dumpMetrics()

Specified by:

dumpMetrics in interface MapJoinTableContainer

hasSpill

public boolean hasSpill()

Description copied from interface: MapJoinTableContainer

Checks if the container has spilled any data onto disk. This is only applicable for HybridHashTableContainer.

Specified by:

hasSpill in interface MapJoinTableContainer

setSerde

public void setSerde(MapJoinObjectSerDeContext keyCtx,
                     MapJoinObjectSerDeContext valCtx)
              throws SerDeException

Specified by:

setSerde in interface MapJoinTableContainer

Throws:

SerDeException