org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners
Class SkewedPartitioner

java.lang.Object
  org.apache.hadoop.mapreduce.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>
      org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class SkewedPartitioner
extends org.apache.hadoop.mapreduce.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>
implements org.apache.hadoop.conf.Configurable

This class is used by skewed join. For the partitioned table, the skewedpartitioner reads the key distribution data from the sampler file and returns the reducer index in a round robin fashion. For ex: if the key distribution file contains (k1, 5, 3) as an entry, reducers from 5 to 3 are returned in a round robin manner.

Constructor Summary

SkewedPartitioner()

Method Summary

org.apache.hadoop.conf.Configuration	getConf()
int	getPartition(PigNullableWritable wrappedKey, org.apache.hadoop.io.Writable value, int numPartitions)
void	setConf(org.apache.hadoop.conf.Configuration job)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SkewedPartitioner

public SkewedPartitioner()

Method Detail

getPartition

public int getPartition(PigNullableWritable wrappedKey,
                        org.apache.hadoop.io.Writable value,
                        int numPartitions)

Specified by:

getPartition in class org.apache.hadoop.mapreduce.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>

setConf

public void setConf(org.apache.hadoop.conf.Configuration job)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable