org.apache.hive.hcatalog.templeton.tool

Class HDFSStorage

java.lang.Object

org.apache.hive.hcatalog.templeton.tool.HDFSStorage

All Implemented Interfaces:

TempletonStorage

public class HDFSStorage
extends Object
implements TempletonStorage

HDFS implementation of templeton storage. This implementation assumes that all keys in key/value pairs are chosen such that they don't have any newlines in them.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hive.hcatalog.templeton.tool.TempletonStorage

TempletonStorage.Type

Field Summary

Fields

Modifier and Type	Field and Description
static String	JOB_PATH
static String	JOB_TRACKINGPATH
static String	OVERHEAD_PATH
String	storage_root

Fields inherited from interface org.apache.hive.hcatalog.templeton.tool.TempletonStorage

STORAGE_CLASS, STORAGE_ROOT

Constructor Summary

Constructors

Constructor and Description
HDFSStorage()

Method Summary

Modifier and Type	Method and Description
void	closeStorage() For storage methods that require a connection, this is a hint that it's time to close the connection.
boolean	delete(TempletonStorage.Type type, String id) Delete a data grouping (all data for a jobid, all tracking data for a job, etc.).
List<String>	getAllForType(TempletonStorage.Type type) Get the id of each data grouping of a given type in the storage system.
String	getField(TempletonStorage.Type type, String id, String key) Get the value of one field for a given data type.
String	getPath(TempletonStorage.Type type) Get the path to storage based on the type.
static String	getPath(TempletonStorage.Type type, String root) Static method to get the path based on the type.
void	openStorage(org.apache.hadoop.conf.Configuration config) For storage methods that require a connection, this is a hint that it's time to open a connection.
void	saveField(TempletonStorage.Type type, String id, String key, String val) Save a single key/value pair for a specific job id.
void	startCleanup(org.apache.hadoop.conf.Configuration config) Start the cleanup process for this storage type.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

storage_root

public String storage_root

JOB_PATH

public static final String JOB_PATH

See Also:

Constant Field Values

JOB_TRACKINGPATH

public static final String JOB_TRACKINGPATH

See Also:

Constant Field Values

OVERHEAD_PATH

public static final String OVERHEAD_PATH

See Also:

Constant Field Values

Constructor Detail

HDFSStorage

public HDFSStorage()

Method Detail

startCleanup

public void startCleanup(org.apache.hadoop.conf.Configuration config)

Description copied from interface: TempletonStorage

Start the cleanup process for this storage type.

Specified by:

startCleanup in interface TempletonStorage

saveField

public void saveField(TempletonStorage.Type type,
                      String id,
                      String key,
                      String val)
               throws NotFoundException

Description copied from interface: TempletonStorage

Save a single key/value pair for a specific job id.

Specified by:

saveField in interface TempletonStorage

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

key - The name of the field to save

val - The value of the field to save

Throws:

NotFoundException

getField

public String getField(TempletonStorage.Type type,
                       String id,
                       String key)

Description copied from interface: TempletonStorage

Get the value of one field for a given data type. If the type is UNKNOWN, search for the id in all types.

Specified by:

getField in interface TempletonStorage

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

key - The name of the field to retrieve

Returns:

The value of the field requested, or null if not found.

delete

public boolean delete(TempletonStorage.Type type,
                      String id)
               throws NotFoundException

Description copied from interface: TempletonStorage

Delete a data grouping (all data for a jobid, all tracking data for a job, etc.). If the type is UNKNOWN, search for the id in all types.

Specified by:

delete in interface TempletonStorage

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

Returns:

True if successful, false if not, throws NotFoundException if the id wasn't found.

Throws:

NotFoundException

getAllForType

public List<String> getAllForType(TempletonStorage.Type type)

Description copied from interface: TempletonStorage

Get the id of each data grouping of a given type in the storage system.

Specified by:

getAllForType in interface TempletonStorage

Parameters:

type - The data type (as listed above)

Returns:

A list of ids.

openStorage

public void openStorage(org.apache.hadoop.conf.Configuration config)
                 throws IOException

Description copied from interface: TempletonStorage

For storage methods that require a connection, this is a hint that it's time to open a connection.

Specified by:

openStorage in interface TempletonStorage

Throws:

IOException

closeStorage

public void closeStorage()
                  throws IOException

Description copied from interface: TempletonStorage

For storage methods that require a connection, this is a hint that it's time to close the connection.

Specified by:

closeStorage in interface TempletonStorage

Throws:

IOException

getPath

public String getPath(TempletonStorage.Type type)

Get the path to storage based on the type.

Parameters:

type -

getPath

public static String getPath(TempletonStorage.Type type,
                             String root)

Static method to get the path based on the type.

Parameters:

type -

root -