org.apache.hadoop.hive.ql.io

Class RCFileOutputFormat

java.lang.Object

org.apache.hadoop.mapred.FileOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>

org.apache.hadoop.hive.ql.io.RCFileOutputFormat

All Implemented Interfaces:

HiveOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>, org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>

public class RCFileOutputFormat
extends org.apache.hadoop.mapred.FileOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>
implements HiveOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>

RCFileOutputFormat.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileOutputFormat

org.apache.hadoop.mapred.FileOutputFormat.Counter

Constructor Summary

Constructors

Constructor and Description
RCFileOutputFormat()

Method Summary

Modifier and Type	Method and Description
static int	getColumnNumber(org.apache.hadoop.conf.Configuration conf) Returns the number of columns set in the conf for writers.
FileSinkOperator.RecordWriter	getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jc, org.apache.hadoop.fs.Path finalOutPath, Class<? extends org.apache.hadoop.io.Writable> valueClass, boolean isCompressed, Properties tableProperties, org.apache.hadoop.util.Progressable progress) create the final out file.
org.apache.hadoop.mapred.RecordWriter<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>	getRecordWriter(org.apache.hadoop.fs.FileSystem ignored, org.apache.hadoop.mapred.JobConf job, String name, org.apache.hadoop.util.Progressable progress)
static void	setColumnNumber(org.apache.hadoop.conf.Configuration conf, int columnNum) set number of columns into the given configuration.

Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat

checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath, setWorkOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.mapred.OutputFormat

checkOutputSpecs

Constructor Detail

RCFileOutputFormat

public RCFileOutputFormat()

Method Detail

setColumnNumber

public static void setColumnNumber(org.apache.hadoop.conf.Configuration conf,
                                   int columnNum)

set number of columns into the given configuration.

Parameters:

conf - configuration instance which need to set the column number

columnNum - column number for RCFile's Writer

getColumnNumber

public static int getColumnNumber(org.apache.hadoop.conf.Configuration conf)

Returns the number of columns set in the conf for writers.

Parameters:

conf -

Returns:

number of columns for RCFile's writer

getRecordWriter

public org.apache.hadoop.mapred.RecordWriter<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable> getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
                                                                                                                            org.apache.hadoop.mapred.JobConf job,
                                                                                                                            String name,
                                                                                                                            org.apache.hadoop.util.Progressable progress)
                                                                                                                     throws IOException

Specified by:

getRecordWriter in interface org.apache.hadoop.mapred.OutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>

Specified by:

getRecordWriter in class org.apache.hadoop.mapred.FileOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>

Throws:

IOException

getHiveRecordWriter

public FileSinkOperator.RecordWriter getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jc,
                                                         org.apache.hadoop.fs.Path finalOutPath,
                                                         Class<? extends org.apache.hadoop.io.Writable> valueClass,
                                                         boolean isCompressed,
                                                         Properties tableProperties,
                                                         org.apache.hadoop.util.Progressable progress)
                                                  throws IOException

create the final out file.

Specified by:

getHiveRecordWriter in interface HiveOutputFormat<org.apache.hadoop.io.WritableComparable,BytesRefArrayWritable>

Parameters:

jc - the job configuration file

finalOutPath - the final output file to be created

valueClass - the value class used for create

isCompressed - whether the content is compressed or not

tableProperties - the tableInfo of this file's corresponding table

progress - progress used for status report

Returns:

the RecordWriter for the output file

Throws:

IOException