org.apache.hadoop.hdfs.server.namenode

类 BlockPlacementPolicy

java.lang.Object

org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy

直接已知子类:

BlockPlacementPolicyDefault

@InterfaceAudience.Private
public abstract class BlockPlacementPolicy
extends Object

This interface is used for choosing the desired number of targets for placing block replicas.

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	BlockPlacementPolicy.NotEnoughReplicasException

构造器概要

构造器

构造器和说明
BlockPlacementPolicy()

方法概要

方法

限定符和类型	方法和说明
void	adjustSetsWithChosenReplica(Map<String,List<DatanodeDescriptor>> rackMap, List<DatanodeDescriptor> moreThanOne, List<DatanodeDescriptor> exactlyOne, DatanodeInfo cur) Adjust rackmap, moreThanOne, and exactlyOne after removing replica on cur.
abstract DatanodeDescriptor	chooseReplicaToDelete(FSInodeInfo srcInode, Block block, short replicationFactor, Collection<DatanodeDescriptor> existingReplicas, Collection<DatanodeDescriptor> moreExistingReplicas) Decide whether deleting the specified replica of the block still makes the block conform to the configured block placement policy.
DatanodeDescriptor[]	chooseTarget(String srcPath, int numOfReplicas, DatanodeDescriptor writer, HashMap<Node,Node> excludedNodes, long blocksize) choose numOfReplicas nodes for writer to replicate a block with size blocksize If not, return as many as we can.
abstract DatanodeDescriptor[]	chooseTarget(String srcPath, int numOfReplicas, DatanodeDescriptor writer, List<DatanodeDescriptor> chosenNodes, HashMap<Node,Node> excludedNodes, long blocksize) choose numOfReplicas data nodes for writer to re-replicate a block with size blocksize If not, return as many as we can.
static BlockPlacementPolicy	getInstance(Configuration conf, FSClusterStats stats, NetworkTopology clusterMap) Get an instance of the configured Block Placement Policy based on the value of the configuration paramater dfs.block.replicator.classname.
protected String	getRack(DatanodeInfo datanode) Get rack string from a data node
protected abstract void	initialize(Configuration conf, FSClusterStats stats, NetworkTopology clusterMap) Used to setup a BlockPlacementPolicy object.
void	splitNodesWithRack(Collection<DatanodeDescriptor> dataNodes, Map<String,List<DatanodeDescriptor>> rackMap, List<DatanodeDescriptor> moreThanOne, List<DatanodeDescriptor> exactlyOne) Split data nodes into two sets, one set includes nodes on rack with more than one replica, the other set contains the remaining nodes.
abstract int	verifyBlockPlacement(String srcPath, LocatedBlock lBlk, int minRacks) Verify that the block is replicated on at least minRacks different racks if there is more than minRacks rack in the system.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

BlockPlacementPolicy

public BlockPlacementPolicy()

方法详细资料

chooseTarget

public abstract DatanodeDescriptor[] chooseTarget(String srcPath,
                                int numOfReplicas,
                                DatanodeDescriptor writer,
                                List<DatanodeDescriptor> chosenNodes,
                                HashMap<Node,Node> excludedNodes,
                                long blocksize)

choose numOfReplicas data nodes for writer to re-replicate a block with size blocksize If not, return as many as we can.

参数:

srcPath - the file to which this chooseTargets is being invoked.

numOfReplicas - additional number of replicas wanted.

writer - the writer's machine, null if not in the cluster.

chosenNodes - datanodes that have been chosen as targets.

excludedNodes: - datanodes that should not be considered as targets.

blocksize - size of the data to be written.

返回:

array of DatanodeDescriptor instances chosen as target and sorted as a pipeline.

verifyBlockPlacement

public abstract int verifyBlockPlacement(String srcPath,
                       LocatedBlock lBlk,
                       int minRacks)

Verify that the block is replicated on at least minRacks different racks if there is more than minRacks rack in the system.

参数:

srcPath - the full pathname of the file to be verified

lBlk - block with locations

minRacks - number of racks the block should be replicated to

返回:

the difference between the required and the actual number of racks the block is replicated to.

chooseReplicaToDelete

public abstract DatanodeDescriptor chooseReplicaToDelete(FSInodeInfo srcInode,
                                       Block block,
                                       short replicationFactor,
                                       Collection<DatanodeDescriptor> existingReplicas,
                                       Collection<DatanodeDescriptor> moreExistingReplicas)

Decide whether deleting the specified replica of the block still makes the block conform to the configured block placement policy.

参数:

srcInode - The inode of the file to which the block-to-be-deleted belongs

block - The block to be deleted

replicationFactor - The required number of replicas for this block

existingReplicas - The replica locations of this block that are present on at least two unique racks.

moreExistingReplicas - Replica locations of this block that are not listed in the previous parameter.

返回:

the replica that is the best candidate for deletion

initialize

protected abstract void initialize(Configuration conf,
              FSClusterStats stats,
              NetworkTopology clusterMap)

Used to setup a BlockPlacementPolicy object. This should be defined by all implementations of a BlockPlacementPolicy.

参数:

conf - the configuration object

stats - retrieve cluster status from here

clusterMap - cluster topology

getInstance

public static BlockPlacementPolicy getInstance(Configuration conf,
                               FSClusterStats stats,
                               NetworkTopology clusterMap)

Get an instance of the configured Block Placement Policy based on the value of the configuration paramater dfs.block.replicator.classname.

参数:

conf - the configuration to be used

stats - an object that is used to retrieve the load on the cluster

clusterMap - the network topology of the cluster

返回:

an instance of BlockPlacementPolicy

chooseTarget

public DatanodeDescriptor[] chooseTarget(String srcPath,
                                int numOfReplicas,
                                DatanodeDescriptor writer,
                                HashMap<Node,Node> excludedNodes,
                                long blocksize)

choose numOfReplicas nodes for writer to replicate a block with size blocksize If not, return as many as we can.

参数:

srcPath - a string representation of the file for which chooseTarget is invoked

numOfReplicas - number of replicas wanted.

writer - the writer's machine, null if not in the cluster.

blocksize - size of the data to be written.

excludedNodes - datanodes that should not be considered as targets.

返回:

array of DatanodeDescriptor instances chosen as targets and sorted as a pipeline.

adjustSetsWithChosenReplica

public void adjustSetsWithChosenReplica(Map<String,List<DatanodeDescriptor>> rackMap,
                               List<DatanodeDescriptor> moreThanOne,
                               List<DatanodeDescriptor> exactlyOne,
                               DatanodeInfo cur)

Adjust rackmap, moreThanOne, and exactlyOne after removing replica on cur.

参数:

rackMap - a map from rack to replica

moreThanOne - The List of replica nodes on rack which has more than one replica

exactlyOne - The List of replica nodes on rack with only one replica

cur - current replica to remove

getRack

protected String getRack(DatanodeInfo datanode)

Get rack string from a data node

参数:

datanode -

返回:

rack of data node

splitNodesWithRack

public void splitNodesWithRack(Collection<DatanodeDescriptor> dataNodes,
                      Map<String,List<DatanodeDescriptor>> rackMap,
                      List<DatanodeDescriptor> moreThanOne,
                      List<DatanodeDescriptor> exactlyOne)

Split data nodes into two sets, one set includes nodes on rack with more than one replica, the other set contains the remaining nodes.

参数:

dataNodes -

rackMap - a map from rack to datanodes

moreThanOne - contains nodes on rack with more than one replica

exactlyOne - remains contains the remaining nodes