org.apache.hadoop.hive.contrib.fileformat.base64

Class Base64TextOutputFormat<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.hadoop.mapred.FileOutputFormat<K,V>

org.apache.hadoop.mapred.TextOutputFormat<K,V>

org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat<K,V>

org.apache.hadoop.hive.contrib.fileformat.base64.Base64TextOutputFormat<K,V>

All Implemented Interfaces:

HiveOutputFormat<K,V>, org.apache.hadoop.mapred.OutputFormat<K,V>

public class Base64TextOutputFormat<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
extends HiveIgnoreKeyTextOutputFormat<K,V>

FileOutputFormat for base64 encoded text files. Each line is a base64-encoded record. The key is a LongWritable which is the offset. The value is a BytesWritable containing the base64-decoded bytes. This class accepts a configurable parameter: "base64.text.output.format.signature" The UTF-8 encoded signature will be prepended to each BytesWritable before we do base64 encoding.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Base64TextOutputFormat.Base64RecordWriter Base64RecordWriter.

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat

HiveIgnoreKeyTextOutputFormat.IgnoreKeyWriter<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.TextOutputFormat

org.apache.hadoop.mapred.TextOutputFormat.LineRecordWriter<K,V>

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileOutputFormat

org.apache.hadoop.mapred.FileOutputFormat.Counter

Constructor Summary

Constructors

Constructor and Description
Base64TextOutputFormat()

Method Summary

Modifier and Type	Method and Description
FileSinkOperator.RecordWriter	getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jc, org.apache.hadoop.fs.Path finalOutPath, Class<? extends org.apache.hadoop.io.Writable> valueClass, boolean isCompressed, Properties tableProperties, org.apache.hadoop.util.Progressable progress) create the final out file, and output row by row.

Methods inherited from class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat

getRecordWriter

Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat

checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath, setWorkOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.mapred.OutputFormat

checkOutputSpecs

Constructor Detail

Base64TextOutputFormat

public Base64TextOutputFormat()

Method Detail

getHiveRecordWriter

public FileSinkOperator.RecordWriter getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jc,
                                                         org.apache.hadoop.fs.Path finalOutPath,
                                                         Class<? extends org.apache.hadoop.io.Writable> valueClass,
                                                         boolean isCompressed,
                                                         Properties tableProperties,
                                                         org.apache.hadoop.util.Progressable progress)
                                                  throws IOException

Description copied from class: HiveIgnoreKeyTextOutputFormat

create the final out file, and output row by row. After one row is appended, a configured row separator is appended

Specified by:

getHiveRecordWriter in interface HiveOutputFormat<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>

Overrides:

getHiveRecordWriter in class HiveIgnoreKeyTextOutputFormat<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>

Parameters:

jc - the job configuration file

finalOutPath - the final output file to be created

valueClass - the value class used for create

isCompressed - whether the content is compressed or not

tableProperties - the tableProperties of this file's corresponding table

progress - progress used for status report

Returns:

the RecordWriter

Throws:

IOException