org.apache.hadoop.hive.shims

Class Hadoop23Shims

java.lang.Object

org.apache.hadoop.hive.shims.HadoopShimsSecure

org.apache.hadoop.hive.shims.Hadoop23Shims

All Implemented Interfaces:

HadoopShims

public class Hadoop23Shims
extends HadoopShimsSecure

Implemention of shims against Hadoop 0.23.0.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	Hadoop23Shims.HdfsEncryptionShim
class	Hadoop23Shims.KerberosNameShim Shim for KerberosName
class	Hadoop23Shims.MiniDFSShim MiniDFSShim.
class	Hadoop23Shims.MiniMrShim Shim for MiniMrCluster
class	Hadoop23Shims.MiniSparkShim Shim for MiniSparkOnYARNCluster
class	Hadoop23Shims.MiniTezLocalShim
class	Hadoop23Shims.MiniTezShim Shim for MiniTezCluster
static class	Hadoop23Shims.StoragePolicyShim

Nested classes/interfaces inherited from class org.apache.hadoop.hive.shims.HadoopShimsSecure

HadoopShimsSecure.CombineFileInputFormatShim<K,V>, HadoopShimsSecure.CombineFileRecordReader<K,V>, HadoopShimsSecure.InputSplitShim

Nested classes/interfaces inherited from interface org.apache.hadoop.hive.shims.HadoopShims

HadoopShims.HCatHadoopShims, HadoopShims.HdfsFileStatusWithId, HadoopShims.JobTrackerState, HadoopShims.NoopHdfsEncryptionShim, HadoopShims.StoragePolicyValue, HadoopShims.WebHCatJTShim

Field Summary

Fields

Modifier and Type	Field and Description
protected static Method	accessMethod
protected static Method	getPasswordMethod

Constructor Summary

Constructors

Constructor and Description
Hadoop23Shims()

Method Summary

Modifier and Type	Method and Description
void	addDelegationTokens(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.security.Credentials cred, String uname) Get Delegation token and add it to Credential.
void	checkFileAccess(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.permission.FsAction action) Check if the configured UGI has access to the path for the given file system action.
org.apache.hadoop.security.UserGroupInformation	cloneUgi(org.apache.hadoop.security.UserGroupInformation baseUgi) Clones the UGI and the Subject.
HadoopShims.HdfsEncryptionShim	createHdfsEncryptionShim(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf) Returns a new instance of the HdfsEncryption shim.
org.apache.hadoop.fs.FileSystem	createProxyFileSystem(org.apache.hadoop.fs.FileSystem fs, URI uri) Create a proxy file system that can serve a given scheme/authority using some other file system.
HadoopShims.CombineFileInputFormatShim	getCombineFileInputFormat()
org.apache.hadoop.conf.Configuration	getConfiguration(org.apache.hadoop.mapreduce.JobContext context) Get configuration from JobContext
org.apache.hadoop.fs.Path	getCurrentTrashPath(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs)
long	getDefaultBlockSize(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) Get the default block size for the path.
short	getDefaultReplication(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) Get the default replication for a path.
long	getFileId(org.apache.hadoop.fs.FileSystem fs, String path) Gets file ID.
HadoopShims.HCatHadoopShims	getHCatShim()
org.apache.hadoop.mapred.JobConf	getJobConf(org.apache.hadoop.mapred.JobContext context) Get job conf from the old style JobContext.
String	getJobLauncherHttpAddress(org.apache.hadoop.conf.Configuration conf) All references to jobtracker/resource manager http address in the configuration should be done through this shim
String	getJobLauncherRpcAddress(org.apache.hadoop.conf.Configuration conf) All retrieval of jobtracker/resource manager rpc address in the configuration should be done through this shim
HadoopShims.JobTrackerState	getJobTrackerState(org.apache.hadoop.mapred.ClusterStatus clusterStatus) Convert the ClusterStatus to its Thrift equivalent: JobTrackerState.
Hadoop23Shims.KerberosNameShim	getKerberosNameShim(String name) Returns a shim to wrap KerberosName
HadoopShims.MiniMrShim	getLocalMiniTezCluster(org.apache.hadoop.conf.Configuration conf, boolean usingLlap)
org.apache.hadoop.fs.BlockLocation[]	getLocations(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus status) For file status returned by listLocatedStatus, convert them into a list of block locations.
TreeMap<Long,org.apache.hadoop.fs.BlockLocation>	getLocationsWithOffset(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus status) For the block locations returned by getLocations() convert them into a Treemap <Offset,blockLocation> by iterating over the list of blockLocation.
Comparator<org.apache.hadoop.io.LongWritable>	getLongComparator()
void	getMergedCredentials(org.apache.hadoop.mapred.JobConf jobConf)
HadoopShims.MiniDFSShim	getMiniDfs(org.apache.hadoop.conf.Configuration conf, int numDataNodes, boolean format, String[] racks) Returns a shim to wrap MiniDFSCluster.
HadoopShims.MiniDFSShim	getMiniDfs(org.apache.hadoop.conf.Configuration conf, int numDataNodes, boolean format, String[] racks, boolean isHA)
Hadoop23Shims.MiniMrShim	getMiniMrCluster(org.apache.hadoop.conf.Configuration conf, int numberOfTaskTrackers, String nameNode, int numDir) Returns a shim to wrap MiniMrCluster
Hadoop23Shims.MiniMrShim	getMiniSparkCluster(org.apache.hadoop.conf.Configuration conf, int numberOfTaskTrackers, String nameNode, int numDir) Returns a shim to wrap MiniSparkOnYARNCluster
Hadoop23Shims.MiniMrShim	getMiniTezCluster(org.apache.hadoop.conf.Configuration conf, int numberOfTaskTrackers, String nameNode, boolean usingLlap) Returns a shim to wrap MiniMrTez
org.apache.hadoop.fs.FileSystem	getNonCachedFileSystem(URI uri, org.apache.hadoop.conf.Configuration conf)
String	getPassword(org.apache.hadoop.conf.Configuration conf, String name) Use password API (if available) to fetch credentials/password
org.apache.hadoop.fs.Path	getPathWithoutSchemeAndAuthority(org.apache.hadoop.fs.Path path)
HadoopShims.StoragePolicyShim	getStoragePolicyShim(org.apache.hadoop.fs.FileSystem fs) obtain a storage policy shim associated with the filesystem.
String	getTaskAttemptLogUrl(org.apache.hadoop.mapred.JobConf conf, String taskTrackerHttpAddress, String taskAttemptId) Constructs and Returns TaskAttempt Logger Url or null if the TaskLogServlet is not available
HadoopShims.WebHCatJTShim	getWebHCatShim(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.security.UserGroupInformation ugi) Provides a Hadoop JobTracker shim.
boolean	hasStickyBit(org.apache.hadoop.fs.permission.FsPermission permission) Check stick bit in the permission
void	hflush(org.apache.hadoop.fs.FSDataOutputStream stream) Flush and make visible to other users the changes to the given stream.
boolean	isDirectory(org.apache.hadoop.fs.FileStatus fileStatus) Check whether file is directory.
static boolean	isHdfsEncryptionSupported()
boolean	isLocalMode(org.apache.hadoop.conf.Configuration conf) Check wether MR is configured to run in local-mode
List<HadoopShims.HdfsFileStatusWithId>	listLocatedHdfsStatus(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.PathFilter filter)
void	mergeCredentials(org.apache.hadoop.mapred.JobConf dest, org.apache.hadoop.mapred.JobConf src)
org.apache.hadoop.mapreduce.JobContext	newJobContext(org.apache.hadoop.mapreduce.Job job)
org.apache.hadoop.mapreduce.TaskAttemptContext	newTaskAttemptContext(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.util.Progressable progressable)
org.apache.hadoop.mapreduce.TaskAttemptID	newTaskAttemptID(org.apache.hadoop.mapreduce.JobID jobId, boolean isMap, int taskId, int id)
int	readByteBuffer(org.apache.hadoop.fs.FSDataInputStream file, ByteBuffer dest) Reads data into ByteBuffer.
void	refreshDefaultQueue(org.apache.hadoop.conf.Configuration conf, String userName) Load the fair scheduler queue for given user if available.
boolean	runDistCp(List<org.apache.hadoop.fs.Path> srcPaths, org.apache.hadoop.fs.Path dst, org.apache.hadoop.conf.Configuration conf) Copies a source dir/file to a destination by orchestrating the copy between hdfs nodes.
boolean	runDistCpAs(List<org.apache.hadoop.fs.Path> srcPaths, org.apache.hadoop.fs.Path dst, org.apache.hadoop.conf.Configuration conf, String doAsUser) Copies a source dir/file to a destination by orchestrating the copy between hdfs nodes.
void	setJobLauncherRpcAddress(org.apache.hadoop.conf.Configuration conf, String val) All updates to jobtracker/resource manager rpc address in the configuration should be done through this shim
void	setTotalOrderPartitionFile(org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.fs.Path partitionFile) The method sets to set the partition file has a different signature between hadoop versions.
boolean	supportStickyBit() check whether current hadoop supports sticky bit
boolean	supportTrashFeature()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

accessMethod

protected static final Method accessMethod

getPasswordMethod

protected static final Method getPasswordMethod

Constructor Detail

Hadoop23Shims

public Hadoop23Shims()

Method Detail

getCombineFileInputFormat

public HadoopShims.CombineFileInputFormatShim getCombineFileInputFormat()

getTaskAttemptLogUrl

public String getTaskAttemptLogUrl(org.apache.hadoop.mapred.JobConf conf,
                                   String taskTrackerHttpAddress,
                                   String taskAttemptId)
                            throws MalformedURLException

Description copied from interface: HadoopShims

Constructs and Returns TaskAttempt Logger Url or null if the TaskLogServlet is not available

Returns:

TaskAttempt Logger Url

Throws:

MalformedURLException

getJobTrackerState

public HadoopShims.JobTrackerState getJobTrackerState(org.apache.hadoop.mapred.ClusterStatus clusterStatus)
                                               throws Exception

Description copied from interface: HadoopShims

Convert the ClusterStatus to its Thrift equivalent: JobTrackerState. See MAPREDUCE-2455 for why this is a part of the shim.

Specified by:

getJobTrackerState in interface HadoopShims

Specified by:

getJobTrackerState in class HadoopShimsSecure

Returns:

the matching JobTrackerState

Throws:

Exception - if no equivalent JobTrackerState exists

newTaskAttemptContext

public org.apache.hadoop.mapreduce.TaskAttemptContext newTaskAttemptContext(org.apache.hadoop.conf.Configuration conf,
                                                                            org.apache.hadoop.util.Progressable progressable)

Specified by:

newTaskAttemptContext in interface HadoopShims

Specified by:

newTaskAttemptContext in class HadoopShimsSecure

newTaskAttemptID

public org.apache.hadoop.mapreduce.TaskAttemptID newTaskAttemptID(org.apache.hadoop.mapreduce.JobID jobId,
                                                                  boolean isMap,
                                                                  int taskId,
                                                                  int id)

newJobContext

public org.apache.hadoop.mapreduce.JobContext newJobContext(org.apache.hadoop.mapreduce.Job job)

Specified by:

newJobContext in interface HadoopShims

Specified by:

newJobContext in class HadoopShimsSecure

isLocalMode

public boolean isLocalMode(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: HadoopShims

Check wether MR is configured to run in local-mode

Specified by:

isLocalMode in interface HadoopShims

Specified by:

isLocalMode in class HadoopShimsSecure

Returns:

getJobLauncherRpcAddress

public String getJobLauncherRpcAddress(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: HadoopShims

All retrieval of jobtracker/resource manager rpc address in the configuration should be done through this shim

Specified by:

getJobLauncherRpcAddress in interface HadoopShims

Specified by:

getJobLauncherRpcAddress in class HadoopShimsSecure

Returns:

setJobLauncherRpcAddress

public void setJobLauncherRpcAddress(org.apache.hadoop.conf.Configuration conf,
                                     String val)

Description copied from interface: HadoopShims

All updates to jobtracker/resource manager rpc address in the configuration should be done through this shim

Specified by:

setJobLauncherRpcAddress in interface HadoopShims

Specified by:

setJobLauncherRpcAddress in class HadoopShimsSecure

getJobLauncherHttpAddress

public String getJobLauncherHttpAddress(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: HadoopShims

All references to jobtracker/resource manager http address in the configuration should be done through this shim

Specified by:

getJobLauncherHttpAddress in interface HadoopShims

Specified by:

getJobLauncherHttpAddress in class HadoopShimsSecure

Returns:

getDefaultBlockSize

public long getDefaultBlockSize(org.apache.hadoop.fs.FileSystem fs,
                                org.apache.hadoop.fs.Path path)

Description copied from interface: HadoopShims

Get the default block size for the path. FileSystem alone is not sufficient to determine the same, as in case of CSMT the underlying file system determines that.

Specified by:

getDefaultBlockSize in interface HadoopShims

Specified by:

getDefaultBlockSize in class HadoopShimsSecure

Returns:

getDefaultReplication

public short getDefaultReplication(org.apache.hadoop.fs.FileSystem fs,
                                   org.apache.hadoop.fs.Path path)

Description copied from interface: HadoopShims

Get the default replication for a path. In case of CSMT the given path will be used to locate the actual filesystem.

Specified by:

getDefaultReplication in interface HadoopShims

Specified by:

getDefaultReplication in class HadoopShimsSecure

Returns:

setTotalOrderPartitionFile

public void setTotalOrderPartitionFile(org.apache.hadoop.mapred.JobConf jobConf,
                                       org.apache.hadoop.fs.Path partitionFile)

Description copied from interface: HadoopShims

The method sets to set the partition file has a different signature between hadoop versions.

getLongComparator

public Comparator<org.apache.hadoop.io.LongWritable> getLongComparator()

refreshDefaultQueue

public void refreshDefaultQueue(org.apache.hadoop.conf.Configuration conf,
                                String userName)
                         throws IOException

Load the fair scheduler queue for given user if available.

userName - end user name

Throws:

IOException

getMiniMrCluster

public Hadoop23Shims.MiniMrShim getMiniMrCluster(org.apache.hadoop.conf.Configuration conf,
                                                 int numberOfTaskTrackers,
                                                 String nameNode,
                                                 int numDir)
                                          throws IOException

Returns a shim to wrap MiniMrCluster

Throws:

IOException

getLocalMiniTezCluster

public HadoopShims.MiniMrShim getLocalMiniTezCluster(org.apache.hadoop.conf.Configuration conf,
                                                     boolean usingLlap)

getMiniTezCluster

public Hadoop23Shims.MiniMrShim getMiniTezCluster(org.apache.hadoop.conf.Configuration conf,
                                                  int numberOfTaskTrackers,
                                                  String nameNode,
                                                  boolean usingLlap)
                                           throws IOException

Returns a shim to wrap MiniMrTez

Throws:

IOException

getMiniSparkCluster

public Hadoop23Shims.MiniMrShim getMiniSparkCluster(org.apache.hadoop.conf.Configuration conf,
                                                    int numberOfTaskTrackers,
                                                    String nameNode,
                                                    int numDir)
                                             throws IOException

Returns a shim to wrap MiniSparkOnYARNCluster

Throws:

IOException

getMiniDfs

public HadoopShims.MiniDFSShim getMiniDfs(org.apache.hadoop.conf.Configuration conf,
                                          int numDataNodes,
                                          boolean format,
                                          String[] racks)
                                   throws IOException

Description copied from interface: HadoopShims

Returns a shim to wrap MiniDFSCluster. This is necessary since this class was moved from org.apache.hadoop.dfs to org.apache.hadoop.hdfs

Throws:

IOException

getMiniDfs

public HadoopShims.MiniDFSShim getMiniDfs(org.apache.hadoop.conf.Configuration conf,
                                          int numDataNodes,
                                          boolean format,
                                          String[] racks,
                                          boolean isHA)
                                   throws IOException

Throws:

IOException

getHCatShim

public HadoopShims.HCatHadoopShims getHCatShim()

getWebHCatShim

public HadoopShims.WebHCatJTShim getWebHCatShim(org.apache.hadoop.conf.Configuration conf,
                                                org.apache.hadoop.security.UserGroupInformation ugi)
                                         throws IOException

Description copied from interface: HadoopShims

Provides a Hadoop JobTracker shim.

Parameters:

conf - not null

Throws:

IOException

listLocatedHdfsStatus

public List<HadoopShims.HdfsFileStatusWithId> listLocatedHdfsStatus(org.apache.hadoop.fs.FileSystem fs,
                                                                    org.apache.hadoop.fs.Path p,
                                                                    org.apache.hadoop.fs.PathFilter filter)
                                                             throws IOException

Throws:

IOException

getLocations

public org.apache.hadoop.fs.BlockLocation[] getLocations(org.apache.hadoop.fs.FileSystem fs,
                                                         org.apache.hadoop.fs.FileStatus status)
                                                  throws IOException

Description copied from interface: HadoopShims

For file status returned by listLocatedStatus, convert them into a list of block locations.

Parameters:

fs - the file system

status - the file information

Returns:

the block locations of the file

Throws:

IOException

getLocationsWithOffset

public TreeMap<Long,org.apache.hadoop.fs.BlockLocation> getLocationsWithOffset(org.apache.hadoop.fs.FileSystem fs,
                                                                               org.apache.hadoop.fs.FileStatus status)
                                                                        throws IOException

Description copied from interface: HadoopShims

For the block locations returned by getLocations() convert them into a Treemap <Offset,blockLocation> by iterating over the list of blockLocation. Using TreeMap from offset to blockLocation, makes it O(logn) to get a particular block based upon offset.

Parameters:

fs - the file system

status - the file information

Returns:

TreeMap<Long, BlockLocation>

Throws:

IOException

hflush

public void hflush(org.apache.hadoop.fs.FSDataOutputStream stream)
            throws IOException

Description copied from interface: HadoopShims

Flush and make visible to other users the changes to the given stream.

Parameters:

stream - the stream to hflush.

Throws:

IOException

createProxyFileSystem

public org.apache.hadoop.fs.FileSystem createProxyFileSystem(org.apache.hadoop.fs.FileSystem fs,
                                                             URI uri)

Description copied from interface: HadoopShims

Create a proxy file system that can serve a given scheme/authority using some other file system.

Specified by:

createProxyFileSystem in interface HadoopShims

Specified by:

createProxyFileSystem in class HadoopShimsSecure

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration(org.apache.hadoop.mapreduce.JobContext context)

Description copied from interface: HadoopShims

Get configuration from JobContext

getJobConf

public org.apache.hadoop.mapred.JobConf getJobConf(org.apache.hadoop.mapred.JobContext context)

Description copied from interface: HadoopShims

Get job conf from the old style JobContext.

Parameters:

context - job context

Returns:

job conf

getNonCachedFileSystem

public org.apache.hadoop.fs.FileSystem getNonCachedFileSystem(URI uri,
                                                              org.apache.hadoop.conf.Configuration conf)
                                                       throws IOException

Specified by:

getNonCachedFileSystem in interface HadoopShims

Specified by:

getNonCachedFileSystem in class HadoopShimsSecure

Throws:

IOException

getMergedCredentials

public void getMergedCredentials(org.apache.hadoop.mapred.JobConf jobConf)
                          throws IOException

Throws:

IOException

mergeCredentials

public void mergeCredentials(org.apache.hadoop.mapred.JobConf dest,
                             org.apache.hadoop.mapred.JobConf src)
                      throws IOException

Throws:

IOException

checkFileAccess

public void checkFileAccess(org.apache.hadoop.fs.FileSystem fs,
                            org.apache.hadoop.fs.FileStatus stat,
                            org.apache.hadoop.fs.permission.FsAction action)
                     throws IOException,
                            AccessControlException,
                            Exception

Description copied from interface: HadoopShims

Check if the configured UGI has access to the path for the given file system action. Method will return successfully if action is permitted. AccessControlExceptoin will be thrown if user does not have access to perform the action. Other exceptions may be thrown for non-access related errors.

Specified by:

checkFileAccess in interface HadoopShims

Overrides:

checkFileAccess in class HadoopShimsSecure

Throws:

IOException

AccessControlException

Exception

getPassword

public String getPassword(org.apache.hadoop.conf.Configuration conf,
                          String name)
                   throws IOException

Description copied from interface: HadoopShims

Use password API (if available) to fetch credentials/password

Returns:

Throws:

IOException

supportStickyBit

public boolean supportStickyBit()

Description copied from interface: HadoopShims

check whether current hadoop supports sticky bit

Returns:

hasStickyBit

public boolean hasStickyBit(org.apache.hadoop.fs.permission.FsPermission permission)

Description copied from interface: HadoopShims

Check stick bit in the permission

Returns:

sticky bit

supportTrashFeature

public boolean supportTrashFeature()

Returns:

True if the current hadoop supports trash feature.

getCurrentTrashPath

public org.apache.hadoop.fs.Path getCurrentTrashPath(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.fs.FileSystem fs)

Returns:

Path to HDFS trash, if current hadoop supports trash feature. Null otherwise.

getKerberosNameShim

public Hadoop23Shims.KerberosNameShim getKerberosNameShim(String name)
                                                   throws IOException

Returns a shim to wrap KerberosName

Throws:

IOException

isDirectory

public boolean isDirectory(org.apache.hadoop.fs.FileStatus fileStatus)

Description copied from interface: HadoopShims

Check whether file is directory.

getStoragePolicyShim

public HadoopShims.StoragePolicyShim getStoragePolicyShim(org.apache.hadoop.fs.FileSystem fs)

Description copied from interface: HadoopShims

obtain a storage policy shim associated with the filesystem. Returns null when the filesystem has no storage policies.

runDistCpAs

public boolean runDistCpAs(List<org.apache.hadoop.fs.Path> srcPaths,
                           org.apache.hadoop.fs.Path dst,
                           org.apache.hadoop.conf.Configuration conf,
                           String doAsUser)
                    throws IOException

Description copied from interface: HadoopShims

Copies a source dir/file to a destination by orchestrating the copy between hdfs nodes. This distributed process is meant to copy huge files that could take some time if a single copy is done. This is a variation which allows proxying as a different user to perform the distcp, and requires that the caller have requisite proxy user privileges.

Parameters:

srcPaths - List of Path to the source files or directories to copy

dst - Path to the destination file or directory

conf - The hadoop configuration object

doAsUser - The user to perform the distcp as

Returns:

True if it is successfull; False otherwise.

Throws:

IOException

runDistCp

public boolean runDistCp(List<org.apache.hadoop.fs.Path> srcPaths,
                         org.apache.hadoop.fs.Path dst,
                         org.apache.hadoop.conf.Configuration conf)
                  throws IOException

Description copied from interface: HadoopShims

Copies a source dir/file to a destination by orchestrating the copy between hdfs nodes. This distributed process is meant to copy huge files that could take some time if a single copy is done.

Parameters:

srcPaths - List of Path to the source files or directories to copy

dst - Path to the destination file or directory

conf - The hadoop configuration object

Returns:

True if it is successfull; False otherwise.

Throws:

IOException

isHdfsEncryptionSupported

public static boolean isHdfsEncryptionSupported()

createHdfsEncryptionShim

public HadoopShims.HdfsEncryptionShim createHdfsEncryptionShim(org.apache.hadoop.fs.FileSystem fs,
                                                               org.apache.hadoop.conf.Configuration conf)
                                                        throws IOException

Description copied from interface: HadoopShims

Returns a new instance of the HdfsEncryption shim.

Parameters:

fs - A FileSystem object to HDFS

conf - A Configuration object

Returns:

A new instance of the HdfsEncryption shim.

Throws:

IOException - If an error occurred while creating the instance.

getPathWithoutSchemeAndAuthority

public org.apache.hadoop.fs.Path getPathWithoutSchemeAndAuthority(org.apache.hadoop.fs.Path path)

readByteBuffer

public int readByteBuffer(org.apache.hadoop.fs.FSDataInputStream file,
                          ByteBuffer dest)
                   throws IOException

Description copied from interface: HadoopShims

Reads data into ByteBuffer.

Parameters:

file - File.

dest - Buffer.

Returns:

Number of bytes read, just like file.read. If any bytes were read, dest position will be set to old position + number of bytes read.

Throws:

IOException

addDelegationTokens

public void addDelegationTokens(org.apache.hadoop.fs.FileSystem fs,
                                org.apache.hadoop.security.Credentials cred,
                                String uname)
                         throws IOException

Description copied from interface: HadoopShims

Get Delegation token and add it to Credential.

Specified by:

addDelegationTokens in interface HadoopShims

Specified by:

addDelegationTokens in class HadoopShimsSecure

Parameters:

fs - FileSystem object to HDFS

cred - Credentials object to add the token to.

uname - user name.

Throws:

IOException - If an error occurred on adding the token.

getFileId

public long getFileId(org.apache.hadoop.fs.FileSystem fs,
                      String path)
               throws IOException

Description copied from interface: HadoopShims

Gets file ID. Only supported on hadoop-2.

Returns:

inode ID of the file.

Throws:

IOException

cloneUgi

public org.apache.hadoop.security.UserGroupInformation cloneUgi(org.apache.hadoop.security.UserGroupInformation baseUgi)
                                                         throws IOException

Description copied from interface: HadoopShims

Clones the UGI and the Subject.

Throws:

IOException