org.apache.hadoop.mapreduce.lib.input

クラス FixedLengthInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<LongWritable,BytesWritable>

org.apache.hadoop.mapreduce.lib.input.FixedLengthInputFormat

@InterfaceAudience.Public
 @InterfaceStability.Stable
public class FixedLengthInputFormat
extends FileInputFormat<LongWritable,BytesWritable>

FixedLengthInputFormat is an input format used to read input files which contain fixed length records. The content of a record need not be text. It can be arbitrary binary data. Users must configure the record length property by calling: FixedLengthInputFormat.setRecordLength(conf, recordLength);

or conf.setInt(FixedLengthInputFormat.FIXED_RECORD_LENGTH, recordLength);

関連項目:

FixedLengthRecordReader

フィールドのサマリー

フィールド

修飾子とタイプ	フィールドと説明
static String	FIXED_RECORD_LENGTH

クラスから継承されたフィールド org.apache.hadoop.mapreduce.lib.input.FileInputFormat

DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

コンストラクタのサマリー

コンストラクタ

コンストラクタと説明
FixedLengthInputFormat()

メソッドのサマリー

修飾子とタイプ	メソッドと説明
RecordReader<LongWritable,BytesWritable>	createRecordReader(InputSplit split, TaskAttemptContext context) Create a record reader for a given split.
static int	getRecordLength(Configuration conf) Get record length value
protected boolean	isSplitable(JobContext context, Path file) Is the given filename splittable?
static void	setRecordLength(Configuration conf, int recordLength) Set the length of each record

クラスから継承されたメソッド org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, listStatus, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

クラスから継承されたメソッド java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

フィールドの詳細

FIXED_RECORD_LENGTH

public static final String FIXED_RECORD_LENGTH

関連項目:

定数フィールド値

コンストラクタの詳細

FixedLengthInputFormat

public FixedLengthInputFormat()

メソッドの詳細

setRecordLength

public static void setRecordLength(Configuration conf,
                                   int recordLength)

Set the length of each record

パラメータ:

conf - configuration

recordLength - the length of a record

getRecordLength

public static int getRecordLength(Configuration conf)

Get record length value

パラメータ:

conf - configuration

戻り値:

the record length, zero means none was set

createRecordReader

public RecordReader<LongWritable,BytesWritable> createRecordReader(InputSplit split,
                                                                   TaskAttemptContext context)
                                                            throws IOException,
                                                                   InterruptedException

クラスからコピーされた説明: InputFormat

Create a record reader for a given split. The framework will call RecordReader.initialize(InputSplit, TaskAttemptContext) before the split is used.

定義:

createRecordReader クラス内 InputFormat<LongWritable,BytesWritable>

パラメータ:

split - the split to be read

context - the information about the task

戻り値:

a new record reader

例外:

IOException

InterruptedException

isSplitable

protected boolean isSplitable(JobContext context,
                              Path file)

クラスからコピーされた説明: FileInputFormat

Is the given filename splittable? Usually, true, but if the file is stream compressed, it will not be. The default implementation in FileInputFormat always returns true. Implementations that may deal with non-splittable files must override this method. FileInputFormat implementations can override this and return false to ensure that individual input files are never split-up so that Mappers process entire files.

オーバーライド:

isSplitable クラス内 FileInputFormat<LongWritable,BytesWritable>

パラメータ:

context - the job context

file - the file name to check

戻り値:

is this file splitable?