org.apache.hadoop.hive.llap

Class LlapBaseInputFormat<V extends org.apache.hadoop.io.WritableComparable<?>>

java.lang.Object

org.apache.hadoop.hive.llap.LlapBaseInputFormat<V>

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.NullWritable,V>

public class LlapBaseInputFormat<V extends org.apache.hadoop.io.WritableComparable<?>>
extends Object
implements org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.NullWritable,V>

Base LLAP input format to handle requesting of splits and communication with LLAP daemon.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DB_KEY
static String	HANDLE_ID
static String	PWD_KEY
static String	QUERY_KEY
static LlapServiceInstance[]	serviceInstanceArray
String	SPLIT_QUERY
static String	URL_KEY
static String	USER_KEY

Constructor Summary

Constructors

Constructor and Description
LlapBaseInputFormat()
LlapBaseInputFormat(boolean useArrow, long arrowAllocatorLimit)
LlapBaseInputFormat(String url, String user, String pwd, String query)

Method Summary

Modifier and Type	Method and Description
static void	close(String handleId) Close the connection associated with the handle ID, if getSplits() was configured with a handle ID.
static void	closeAll() Close all outstanding connections created by getSplits() calls
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.NullWritable,V>	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)
org.apache.hadoop.mapred.InputSplit[]	getSplits(org.apache.hadoop.mapred.JobConf job, int numSplits) Calling getSplits() will open a HiveServer2 connection which should be closed by the calling application using LlapBaseInputFormat.close() when the application is done with the splits.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

URL_KEY

public static final String URL_KEY

See Also:

Constant Field Values

QUERY_KEY

public static final String QUERY_KEY

See Also:

Constant Field Values

USER_KEY

public static final String USER_KEY

See Also:

Constant Field Values

PWD_KEY

public static final String PWD_KEY

See Also:

Constant Field Values

HANDLE_ID

public static final String HANDLE_ID

See Also:

Constant Field Values

DB_KEY

public static final String DB_KEY

See Also:

Constant Field Values

SPLIT_QUERY

public final String SPLIT_QUERY

See Also:

Constant Field Values

serviceInstanceArray

public static final LlapServiceInstance[] serviceInstanceArray

Constructor Detail

LlapBaseInputFormat

public LlapBaseInputFormat(String url,
                           String user,
                           String pwd,
                           String query)

LlapBaseInputFormat

public LlapBaseInputFormat(boolean useArrow,
                           long arrowAllocatorLimit)

LlapBaseInputFormat

public LlapBaseInputFormat()

Method Detail

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.NullWritable,V> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                                                                  org.apache.hadoop.mapred.JobConf job,
                                                                                                  org.apache.hadoop.mapred.Reporter reporter)
                                                                                           throws IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.NullWritable,V extends org.apache.hadoop.io.WritableComparable<?>>

Throws:

IOException

getSplits

public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf job,
                                                       int numSplits)
                                                throws IOException

Calling getSplits() will open a HiveServer2 connection which should be closed by the calling application using LlapBaseInputFormat.close() when the application is done with the splits.

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.NullWritable,V extends org.apache.hadoop.io.WritableComparable<?>>

Throws:

IOException

close

public static void close(String handleId)
                  throws IOException

Close the connection associated with the handle ID, if getSplits() was configured with a handle ID. Call when the application is done using the splits generated by getSplits().

Parameters:

handleId - Handle ID used in configuration for getSplits()

Throws:

IOException

closeAll

public static void closeAll()

Close all outstanding connections created by getSplits() calls