org.apache.hadoop.hive.accumulo

Class HiveAccumuloHelper

java.lang.Object

org.apache.hadoop.hive.accumulo.HiveAccumuloHelper

public class HiveAccumuloHelper
extends Object

Helper class to hold common methods across the InputFormat, OutputFormat and StorageHandler.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.io.Text	ACCUMULO_SERVICE

Constructor Summary

Constructors

Constructor and Description
HiveAccumuloHelper()

Method Summary

Modifier and Type	Method and Description
void	addTokenFromUserToJobConf(org.apache.hadoop.security.UserGroupInformation user, org.apache.hadoop.mapred.JobConf jobConf) Extract the appropriate Token for Accumulo from the provided user and add it to the JobConf's credentials.
org.apache.hadoop.security.token.Token<?>	getAccumuloToken(org.apache.hadoop.security.UserGroupInformation user)
org.apache.accumulo.core.client.ClientConfiguration	getClientConfiguration(String zookeepers, String instanceName, boolean useSasl) Construct a ClientConfiguration instance.
org.apache.accumulo.core.client.security.tokens.AuthenticationToken	getDelegationToken(org.apache.accumulo.core.client.Connector conn) Obtain a DelegationToken from Accumulo.
org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.TokenIdentifier>	getHadoopToken(org.apache.accumulo.core.client.security.tokens.AuthenticationToken token)
boolean	hasKerberosCredentials(org.apache.hadoop.security.UserGroupInformation ugi)
void	loadDependentJars(org.apache.hadoop.conf.Configuration conf) Sets all jars requried by Accumulo input/output tasks in the configuration to be dynamically loaded when the task is executed.
void	mergeTokenIntoJobConf(org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.security.token.Token<?> accumuloToken) Merge the provided Token into the JobConf.
org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.TokenIdentifier>	setConnectorInfoForInputAndOutput(AccumuloConnectionParameters params, org.apache.accumulo.core.client.Connector conn, org.apache.hadoop.conf.Configuration conf) Obtains an Accumulo DelegationToken and sets it in the configuration for input and output jobs.
void	setInputFormatConnectorInfo(org.apache.hadoop.mapred.JobConf conf, String username, org.apache.accumulo.core.client.security.tokens.AuthenticationToken token) Calls AbstractInputFormat.setConnectorInfo(JobConf, String, AuthenticationToken), suppressing exceptions due to setting the configuration multiple times.
void	setInputFormatMockInstance(org.apache.hadoop.mapred.JobConf conf, String instanceName) Calls AbstractInputFormat.setMockInstance(JobConf, String), suppressing exceptions due to setting the configuration multiple times.
void	setInputFormatZooKeeperInstance(org.apache.hadoop.mapred.JobConf conf, String instanceName, String zookeepers, boolean isSasl) Calls AbstractInputFormat.setZooKeeperInstance(JobConf, ClientConfiguration), suppressing exceptions due to setting the configuration multiple times.
void	setOutputFormatConnectorInfo(org.apache.hadoop.mapred.JobConf conf, String username, org.apache.accumulo.core.client.security.tokens.AuthenticationToken token) Calls AccumuloOutputFormat.setConnectorInfo(JobConf, String, AuthenticationToken) suppressing exceptions due to setting the configuration multiple times.
void	setOutputFormatMockInstance(org.apache.hadoop.mapred.JobConf conf, String instanceName) Calls AccumuloOutputFormat.setMockInstance(JobConf, String), suppressing exceptions due to setting the configuration multiple times.
void	setOutputFormatZooKeeperInstance(org.apache.hadoop.mapred.JobConf conf, String instanceName, String zookeepers, boolean isSasl) Calls AccumuloOutputFormat.setZooKeeperInstance(JobConf, ClientConfiguration), suppressing exceptions due to setting the configuration multiple times.
void	updateInputFormatConfWithAccumuloToken(org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.security.UserGroupInformation currentUser, AccumuloConnectionParameters cnxnParams)
void	updateOutputFormatConfWithAccumuloToken(org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.security.UserGroupInformation currentUser, AccumuloConnectionParameters cnxnParams)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ACCUMULO_SERVICE

public static final org.apache.hadoop.io.Text ACCUMULO_SERVICE

Constructor Detail

HiveAccumuloHelper

public HiveAccumuloHelper()

Method Detail

addTokenFromUserToJobConf

public void addTokenFromUserToJobConf(org.apache.hadoop.security.UserGroupInformation user,
                                      org.apache.hadoop.mapred.JobConf jobConf)
                               throws IOException

Extract the appropriate Token for Accumulo from the provided user and add it to the JobConf's credentials.

Parameters:

user - User containing tokens

jobConf - The configuration for the job

Throws:

IOException - If the correct token is not found or the Token fails to be merged with the configuration

getAccumuloToken

public org.apache.hadoop.security.token.Token<?> getAccumuloToken(org.apache.hadoop.security.UserGroupInformation user)

mergeTokenIntoJobConf

public void mergeTokenIntoJobConf(org.apache.hadoop.mapred.JobConf jobConf,
                                  org.apache.hadoop.security.token.Token<?> accumuloToken)
                           throws IOException

Merge the provided Token into the JobConf.

Parameters:

jobConf - JobConf to merge token into

accumuloToken - The Token

Throws:

IOException - If the merging fails

getDelegationToken

public org.apache.accumulo.core.client.security.tokens.AuthenticationToken getDelegationToken(org.apache.accumulo.core.client.Connector conn)
                                                                                       throws IOException

Obtain a DelegationToken from Accumulo.

Parameters:

conn - The Accumulo connector

Returns:

The DelegationToken instance

Throws:

IOException - If the token cannot be obtained

getHadoopToken

public org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.TokenIdentifier> getHadoopToken(org.apache.accumulo.core.client.security.tokens.AuthenticationToken token)
                                                                                                                  throws IOException

Throws:

IOException

getClientConfiguration

public org.apache.accumulo.core.client.ClientConfiguration getClientConfiguration(String zookeepers,
                                                                                  String instanceName,
                                                                                  boolean useSasl)

Construct a ClientConfiguration instance.

Parameters:

zookeepers - ZooKeeper hosts

instanceName - Instance name

useSasl - Is SASL enabled

Returns:

A ClientConfiguration instance

updateInputFormatConfWithAccumuloToken

public void updateInputFormatConfWithAccumuloToken(org.apache.hadoop.mapred.JobConf jobConf,
                                                   org.apache.hadoop.security.UserGroupInformation currentUser,
                                                   AccumuloConnectionParameters cnxnParams)
                                            throws IOException

Throws:

IOException

updateOutputFormatConfWithAccumuloToken

public void updateOutputFormatConfWithAccumuloToken(org.apache.hadoop.mapred.JobConf jobConf,
                                                    org.apache.hadoop.security.UserGroupInformation currentUser,
                                                    AccumuloConnectionParameters cnxnParams)
                                             throws IOException

Throws:

IOException

hasKerberosCredentials

public boolean hasKerberosCredentials(org.apache.hadoop.security.UserGroupInformation ugi)

setInputFormatConnectorInfo

public void setInputFormatConnectorInfo(org.apache.hadoop.mapred.JobConf conf,
                                        String username,
                                        org.apache.accumulo.core.client.security.tokens.AuthenticationToken token)
                                 throws org.apache.accumulo.core.client.AccumuloSecurityException

Calls AbstractInputFormat.setConnectorInfo(JobConf, String, AuthenticationToken), suppressing exceptions due to setting the configuration multiple times.

Throws:

org.apache.accumulo.core.client.AccumuloSecurityException

setOutputFormatConnectorInfo

public void setOutputFormatConnectorInfo(org.apache.hadoop.mapred.JobConf conf,
                                         String username,
                                         org.apache.accumulo.core.client.security.tokens.AuthenticationToken token)
                                  throws org.apache.accumulo.core.client.AccumuloSecurityException

Calls AccumuloOutputFormat.setConnectorInfo(JobConf, String, AuthenticationToken) suppressing exceptions due to setting the configuration multiple times.

Throws:

org.apache.accumulo.core.client.AccumuloSecurityException

setInputFormatZooKeeperInstance

public void setInputFormatZooKeeperInstance(org.apache.hadoop.mapred.JobConf conf,
                                            String instanceName,
                                            String zookeepers,
                                            boolean isSasl)
                                     throws IOException

Calls AbstractInputFormat.setZooKeeperInstance(JobConf, ClientConfiguration), suppressing exceptions due to setting the configuration multiple times.

Throws:

IOException

setOutputFormatZooKeeperInstance

public void setOutputFormatZooKeeperInstance(org.apache.hadoop.mapred.JobConf conf,
                                             String instanceName,
                                             String zookeepers,
                                             boolean isSasl)
                                      throws IOException

Calls AccumuloOutputFormat.setZooKeeperInstance(JobConf, ClientConfiguration), suppressing exceptions due to setting the configuration multiple times.

Throws:

IOException

setInputFormatMockInstance

public void setInputFormatMockInstance(org.apache.hadoop.mapred.JobConf conf,
                                       String instanceName)

Calls AbstractInputFormat.setMockInstance(JobConf, String), suppressing exceptions due to setting the configuration multiple times.

setOutputFormatMockInstance

public void setOutputFormatMockInstance(org.apache.hadoop.mapred.JobConf conf,
                                        String instanceName)

Calls AccumuloOutputFormat.setMockInstance(JobConf, String), suppressing exceptions due to setting the configuration multiple times.

loadDependentJars

public void loadDependentJars(org.apache.hadoop.conf.Configuration conf)

Sets all jars requried by Accumulo input/output tasks in the configuration to be dynamically loaded when the task is executed.

setConnectorInfoForInputAndOutput

public org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.TokenIdentifier> setConnectorInfoForInputAndOutput(AccumuloConnectionParameters params,
                                                                                                                                            org.apache.accumulo.core.client.Connector conn,
                                                                                                                                            org.apache.hadoop.conf.Configuration conf)
                                                                                                                                     throws Exception

Obtains an Accumulo DelegationToken and sets it in the configuration for input and output jobs. The Accumulo token is converted into a Hadoop-style token and returned to the caller.

Returns:

A Hadoop-style token which contains the Accumulo DelegationToken

Throws:

Exception