org.apache.hive.hcatalog.mapreduce

Class HCatOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>

org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat

org.apache.hive.hcatalog.mapreduce.HCatOutputFormat

@InterfaceAudience.Public
 @InterfaceStability.Stable
public class HCatOutputFormat
extends HCatBaseOutputFormat

The OutputFormat to use to write data to HCatalog. The key value is ignored and should be given as null. The value is the HCatRecord to write.

Constructor Summary

Constructors

Constructor and Description
HCatOutputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the output committer for this output format.
org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the record writer for the job.
static void	setOutput(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.security.Credentials credentials, OutputJobInfo outputJobInfo) Set the information about the output to write for the job.
static void	setOutput(org.apache.hadoop.mapreduce.Job job, OutputJobInfo outputJobInfo)
static void	setSchema(org.apache.hadoop.conf.Configuration conf, HCatSchema schema) Set the schema for the data being written out to the partition.
static void	setSchema(org.apache.hadoop.mapreduce.Job job, HCatSchema schema)

Methods inherited from class org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat

checkOutputSpecs, configureOutputStorageHandler, getJobInfo, getOutputFormat, getTableSchema, getTableSchemaWithPartitionColumns, setPartDetails

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HCatOutputFormat

public HCatOutputFormat()

Method Detail

setOutput

public static void setOutput(org.apache.hadoop.mapreduce.Job job,
                             OutputJobInfo outputJobInfo)
                      throws IOException

Throws:

IOException

See Also:

setOutput(org.apache.hadoop.conf.Configuration, Credentials, OutputJobInfo)

setOutput

public static void setOutput(org.apache.hadoop.conf.Configuration conf,
                             org.apache.hadoop.security.Credentials credentials,
                             OutputJobInfo outputJobInfo)
                      throws IOException

Set the information about the output to write for the job. This queries the metadata server to find the StorageHandler to use for the table. It throws an error if the partition is already published.

Parameters:

conf - the Configuration object

credentials - the Credentials object

outputJobInfo - the table output information for the job

Throws:

IOException - the exception in communicating with the metadata server

setSchema

public static void setSchema(org.apache.hadoop.mapreduce.Job job,
                             HCatSchema schema)
                      throws IOException

Throws:

IOException

See Also:

setSchema(org.apache.hadoop.conf.Configuration, org.apache.hive.hcatalog.data.schema.HCatSchema)

setSchema

public static void setSchema(org.apache.hadoop.conf.Configuration conf,
                             HCatSchema schema)
                      throws IOException

Set the schema for the data being written out to the partition. The table schema is used by default for the partition if this is not called.

Parameters:

conf - the job Configuration object

schema - the schema for the data

Throws:

IOException

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable<?>,HCatRecord> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                throws IOException,
                                                                                                                       InterruptedException

Get the record writer for the job. This uses the StorageHandler's default OutputFormat to get the record writer.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>

Parameters:

context - the information about the current task

Returns:

a RecordWriter to write the output for the job

Throws:

IOException

InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws IOException,
                                                                      InterruptedException

Get the output committer for this output format. This is responsible for ensuring the output is committed correctly.

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>

Parameters:

context - the task context

Returns:

an output committer

Throws:

IOException

InterruptedException