org.apache.hive.hcatalog.templeton.tool

Class ZooKeeperStorage

java.lang.Object

org.apache.hive.hcatalog.templeton.tool.ZooKeeperStorage

All Implemented Interfaces:

TempletonStorage

public class ZooKeeperStorage
extends Object
implements TempletonStorage

A storage implementation based on storing everything in ZooKeeper. This keeps everything in a central location that is guaranteed to be available and accessible. Data is stored with each key/value pair being a node in ZooKeeper.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hive.hcatalog.templeton.tool.TempletonStorage

TempletonStorage.Type

Field Summary

Fields

Modifier and Type	Field and Description
static String	ENCODING
String	job_path
String	job_trackingpath
String	overhead_path
String	storage_root
static String	TRACKINGDIR
static String	ZK_HOSTS
static String	ZK_SESSION_TIMEOUT

Fields inherited from interface org.apache.hive.hcatalog.templeton.tool.TempletonStorage

STORAGE_CLASS, STORAGE_ROOT

Constructor Summary

Constructors

Constructor and Description
ZooKeeperStorage()

Method Summary

Modifier and Type	Method and Description
void	close() Close this ZK connection.
void	closeStorage() For storage methods that require a connection, this is a hint that it's time to close the connection.
void	create(TempletonStorage.Type type, String id) Create a node in ZooKeeper
boolean	delete(TempletonStorage.Type type, String id) Delete a data grouping (all data for a jobid, all tracking data for a job, etc.).
List<String>	getAllForType(TempletonStorage.Type type) Get the id of each data grouping of a given type in the storage system.
String	getField(TempletonStorage.Type type, String id, String key) Get the value of one field for a given data type.
String	getPath(TempletonStorage.Type type) Get the path based on the job type.
static String[]	getPaths(String fullpath)
String	makeFieldZnode(TempletonStorage.Type type, String id, String name) Make a ZK path to the named field.
String	makeZnode(TempletonStorage.Type type, String id) Make a ZK path to job
void	openStorage(org.apache.hadoop.conf.Configuration config) For storage methods that require a connection, this is a hint that it's time to open a connection.
void	saveField(TempletonStorage.Type type, String id, String key, String val) Save a single key/value pair for a specific job id.
void	startCleanup(org.apache.hadoop.conf.Configuration config) Start the cleanup process for this storage type.
static org.apache.curator.framework.CuratorFramework	zkOpen(org.apache.hadoop.conf.Configuration conf) Open a ZooKeeper connection for the JobState.
static org.apache.curator.framework.CuratorFramework	zkOpen(String zkHosts, int zkSessionTimeoutMs) Open a ZooKeeper connection for the JobState.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TRACKINGDIR

public static final String TRACKINGDIR

See Also:

Constant Field Values

storage_root

public String storage_root

job_path

public String job_path

job_trackingpath

public String job_trackingpath

overhead_path

public String overhead_path

ZK_HOSTS

public static final String ZK_HOSTS

See Also:

Constant Field Values

ZK_SESSION_TIMEOUT

public static final String ZK_SESSION_TIMEOUT

See Also:

Constant Field Values

ENCODING

public static final String ENCODING

See Also:

Constant Field Values

Constructor Detail

ZooKeeperStorage

public ZooKeeperStorage()

Method Detail

zkOpen

public static org.apache.curator.framework.CuratorFramework zkOpen(String zkHosts,
                                                                   int zkSessionTimeoutMs)
                                                            throws IOException

Open a ZooKeeper connection for the JobState.

Throws:

IOException

zkOpen

public static org.apache.curator.framework.CuratorFramework zkOpen(org.apache.hadoop.conf.Configuration conf)
                                                            throws IOException

Open a ZooKeeper connection for the JobState.

Throws:

IOException

close

public void close()
           throws IOException

Close this ZK connection.

Throws:

IOException

startCleanup

public void startCleanup(org.apache.hadoop.conf.Configuration config)

Description copied from interface: TempletonStorage

Start the cleanup process for this storage type.

Specified by:

startCleanup in interface TempletonStorage

create

public void create(TempletonStorage.Type type,
                   String id)
            throws IOException

Create a node in ZooKeeper

Throws:

IOException

getPath

public String getPath(TempletonStorage.Type type)

Get the path based on the job type.

Parameters:

type -

getPaths

public static String[] getPaths(String fullpath)

makeFieldZnode

public String makeFieldZnode(TempletonStorage.Type type,
                             String id,
                             String name)

Make a ZK path to the named field.

makeZnode

public String makeZnode(TempletonStorage.Type type,
                        String id)

Make a ZK path to job

saveField

public void saveField(TempletonStorage.Type type,
                      String id,
                      String key,
                      String val)
               throws NotFoundException

Description copied from interface: TempletonStorage

Save a single key/value pair for a specific job id.

Specified by:

saveField in interface TempletonStorage

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

key - The name of the field to save

val - The value of the field to save

Throws:

NotFoundException

getField

public String getField(TempletonStorage.Type type,
                       String id,
                       String key)

Description copied from interface: TempletonStorage

Get the value of one field for a given data type. If the type is UNKNOWN, search for the id in all types.

Specified by:

getField in interface TempletonStorage

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

key - The name of the field to retrieve

Returns:

The value of the field requested, or null if not found.

delete

public boolean delete(TempletonStorage.Type type,
                      String id)
               throws NotFoundException

Description copied from interface: TempletonStorage

Delete a data grouping (all data for a jobid, all tracking data for a job, etc.). If the type is UNKNOWN, search for the id in all types.

Specified by:

delete in interface TempletonStorage

Parameters:

type - The data type (as listed above)

id - The String id of this data grouping (jobid, etc.)

Returns:

True if successful, false if not, throws NotFoundException if the id wasn't found.

Throws:

NotFoundException

getAllForType

public List<String> getAllForType(TempletonStorage.Type type)

Description copied from interface: TempletonStorage

Get the id of each data grouping of a given type in the storage system.

Specified by:

getAllForType in interface TempletonStorage

Parameters:

type - The data type (as listed above)

Returns:

A list of ids.

openStorage

public void openStorage(org.apache.hadoop.conf.Configuration config)
                 throws IOException

Description copied from interface: TempletonStorage

For storage methods that require a connection, this is a hint that it's time to open a connection.

Specified by:

openStorage in interface TempletonStorage

Throws:

IOException

closeStorage

public void closeStorage()
                  throws IOException

Description copied from interface: TempletonStorage

For storage methods that require a connection, this is a hint that it's time to close the connection.

Specified by:

closeStorage in interface TempletonStorage

Throws:

IOException