org.apache.hadoop.hive.hbase

Class HiveHBaseTableOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<KEY,org.apache.hadoop.hbase.client.Mutation>

org.apache.hadoop.hbase.mapreduce.TableOutputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable>

org.apache.hadoop.hive.hbase.HiveHBaseTableOutputFormat

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,Object>

public class HiveHBaseTableOutputFormat
extends org.apache.hadoop.hbase.mapreduce.TableOutputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable>
implements org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,Object>

HiveHBaseTableOutputFormat implements HiveOutputFormat for HBase tables. We also need to implement the @deprecated org.apache.hadoop.mapred.OutFormat class to keep it compliant with Hive interfaces.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.mapreduce.TableOutputFormat

org.apache.hadoop.hbase.mapreduce.TableOutputFormat.TableRecordWriter

Field Summary

Fields

Modifier and Type	Field and Description
static String	HBASE_WAL_ENABLED

Fields inherited from class org.apache.hadoop.hbase.mapreduce.TableOutputFormat

OUTPUT_CONF_PREFIX, OUTPUT_TABLE, QUORUM_ADDRESS, QUORUM_PORT, REGION_SERVER_CLASS, REGION_SERVER_IMPL

Constructor Summary

Constructors

Constructor and Description
HiveHBaseTableOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.mapred.JobConf jc) Update the out table, and output an empty key as the key.
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.hadoop.mapred.RecordWriter<org.apache.hadoop.hbase.io.ImmutableBytesWritable,Object>	getRecordWriter(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.mapred.JobConf jobConf, String name, org.apache.hadoop.util.Progressable progressable)

Methods inherited from class org.apache.hadoop.hbase.mapreduce.TableOutputFormat

checkOutputSpecs, getConf, getRecordWriter, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HBASE_WAL_ENABLED

public static final String HBASE_WAL_ENABLED

See Also:

Constant Field Values

Constructor Detail

HiveHBaseTableOutputFormat

public HiveHBaseTableOutputFormat()

Method Detail

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem fs,
                             org.apache.hadoop.mapred.JobConf jc)
                      throws IOException

Update the out table, and output an empty key as the key.

Specified by:

checkOutputSpecs in interface org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,Object>

Parameters:

jc - the job configuration file

finalOutPath - the final output table name

valueClass - the value class

isCompressed - whether the content is compressed or not

tableProperties - the table info of the corresponding table

progress - progress used for status report

Throws:

IOException

getRecordWriter

public org.apache.hadoop.mapred.RecordWriter<org.apache.hadoop.hbase.io.ImmutableBytesWritable,Object> getRecordWriter(org.apache.hadoop.fs.FileSystem fileSystem,
                                                                                                                       org.apache.hadoop.mapred.JobConf jobConf,
                                                                                                                       String name,
                                                                                                                       org.apache.hadoop.util.Progressable progressable)
                                                                                                                throws IOException

Specified by:

getRecordWriter in interface org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,Object>

Throws:

IOException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws IOException,
                                                                      InterruptedException

Overrides:

getOutputCommitter in class org.apache.hadoop.hbase.mapreduce.TableOutputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable>

Throws:

IOException

InterruptedException