org.apache.hadoop.hive.ql.stats.fs

Class FSStatsAggregator

java.lang.Object

org.apache.hadoop.hive.ql.stats.fs.FSStatsAggregator

All Implemented Interfaces:

StatsAggregator

public class FSStatsAggregator
extends Object
implements StatsAggregator

Constructor Summary

Constructors

Constructor and Description
FSStatsAggregator()

Method Summary

Modifier and Type	Method and Description
String	aggregateStats(String partID, String statType) This method aggregates a given statistic from all tasks (partial stats).
boolean	closeConnection(StatsCollectionContext scc) This method closes the connection to the temporary storage.
boolean	connect(StatsCollectionContext scc) This method connects to the temporary storage.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FSStatsAggregator

public FSStatsAggregator()

Method Detail

connect

public boolean connect(StatsCollectionContext scc)

Description copied from interface: StatsAggregator

This method connects to the temporary storage.

Specified by:

connect in interface StatsAggregator

Returns:

true if connection is successful, false otherwise.

aggregateStats

public String aggregateStats(String partID,
                             String statType)

Description copied from interface: StatsAggregator

This method aggregates a given statistic from all tasks (partial stats). After aggregation, this method also automatically removes all records that have been aggregated.

Specified by:

aggregateStats in interface StatsAggregator

Parameters:

partID - a prefix of the keys used in StatsPublisher to publish stats. Any rows that starts with the same prefix will be aggregated. For example, if the StatsPublisher uses the following compound key to publish stats: the output directory name (unique per FileSinkOperator) + the partition specs (only for dynamic partitions) + taskID (last component of task file) The keyPrefix for aggregation could be first 2 components. This will aggregates stats across all tasks for each partition.

statType - a string noting the key to be published. Ex: "numRows".

Returns:

a string representation of a long value, null if there are any error/exception.

closeConnection

public boolean closeConnection(StatsCollectionContext scc)

Description copied from interface: StatsAggregator

This method closes the connection to the temporary storage.

Specified by:

closeConnection in interface StatsAggregator

Returns:

true if close connection is successful, false otherwise.