org.apache.hadoop.mapreduce.lib.partition

クラス KeyFieldBasedPartitioner<K2,V2>

java.lang.Object

org.apache.hadoop.mapreduce.Partitioner<K2,V2>

org.apache.hadoop.mapreduce.lib.partition.KeyFieldBasedPartitioner<K2,V2>

すべての実装されたインタフェース:

Configurable

直系の既知のサブクラス:

KeyFieldBasedPartitioner

@InterfaceAudience.Public
 @InterfaceStability.Stable
public class KeyFieldBasedPartitioner<K2,V2>
extends Partitioner<K2,V2>
implements Configurable

Defines a way to partition keys based on certain key fields (also see KeyFieldBasedComparator. The key specification supported is of the form -k pos1[,pos2], where, pos is of the form f[.c][opts], where f is the number of the key field to use, and c is the number of the first character from the beginning of the field. Fields and character posns are numbered starting with 1; a character position of zero in pos2 indicates the field's last character. If '.c' is omitted from pos1, it defaults to 1 (the beginning of the field); if omitted from pos2, it defaults to 0 (the end of the field).

フィールドのサマリー

フィールド

修飾子とタイプ	フィールドと説明
static String	PARTITIONER_OPTIONS

コンストラクタのサマリー

コンストラクタ

コンストラクタと説明
KeyFieldBasedPartitioner()

メソッドのサマリー

修飾子とタイプ	メソッドと説明
Configuration	getConf() Return the configuration used by this object.
String	getKeyFieldPartitionerOption(JobContext job) Get the KeyFieldBasedPartitioner options
protected int	getPartition(int hash, int numReduceTasks)
int	getPartition(K2 key, V2 value, int numReduceTasks) Get the partition number for a given key (hence record) given the total number of partitions i.e. number of reduce-tasks for the job.
protected int	hashCode(byte[] b, int start, int end, int currentHash)
void	setConf(Configuration conf) Set the configuration to be used by this object.
void	setKeyFieldPartitionerOptions(Job job, String keySpec) Set the KeyFieldBasedPartitioner options used for Partitioner

クラスから継承されたメソッド java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

フィールドの詳細

PARTITIONER_OPTIONS

public static String PARTITIONER_OPTIONS

コンストラクタの詳細

KeyFieldBasedPartitioner

public KeyFieldBasedPartitioner()

メソッドの詳細

setConf

public void setConf(Configuration conf)

インタフェースからコピーされた説明: Configurable

Set the configuration to be used by this object.

定義:

setConf インタフェース内 Configurable

getConf

public Configuration getConf()

インタフェースからコピーされた説明: Configurable

Return the configuration used by this object.

定義:

getConf インタフェース内 Configurable

getPartition

public int getPartition(K2 key,
                        V2 value,
                        int numReduceTasks)

クラスからコピーされた説明: Partitioner

Get the partition number for a given key (hence record) given the total number of partitions i.e. number of reduce-tasks for the job.

Typically a hash function on a all or a subset of the key.

定義:

getPartition クラス内 Partitioner<K2,V2>

パラメータ:

key - the key to be partioned.

value - the entry value.

numReduceTasks - the total number of partitions.

戻り値:

the partition number for the key.

hashCode

protected int hashCode(byte[] b,
                       int start,
                       int end,
                       int currentHash)

getPartition

protected int getPartition(int hash,
                           int numReduceTasks)

setKeyFieldPartitionerOptions

public void setKeyFieldPartitionerOptions(Job job,
                                          String keySpec)

Set the KeyFieldBasedPartitioner options used for Partitioner

パラメータ:

keySpec - the key specification of the form -k pos1[,pos2], where, pos is of the form f[.c][opts], where f is the number of the key field to use, and c is the number of the first character from the beginning of the field. Fields and character posns are numbered starting with 1; a character position of zero in pos2 indicates the field's last character. If '.c' is omitted from pos1, it defaults to 1 (the beginning of the field); if omitted from pos2, it defaults to 0 (the end of the field).

getKeyFieldPartitionerOption

public String getKeyFieldPartitionerOption(JobContext job)

Get the KeyFieldBasedPartitioner options