FileOutputCommitter (Apache Hadoop Main 3.0.0-alpha2-SNAPSHOT API)

java.lang.Object
- org.apache.hadoop.mapreduce.OutputCommitter
- - org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

直系の既知のサブクラス:

PartialFileOutputCommitter
```
@InterfaceAudience.Public
 @InterfaceStability.Stable
public class FileOutputCommitter
extends OutputCommitter
```
An OutputCommitter that commits files specified in job output directory i.e. ${mapreduce.output.fileoutputformat.outputdir}.

フィールドのサマリー

フィールド
修飾子とタイプ	フィールドと説明
`static String`	`FILEOUTPUTCOMMITTER_ALGORITHM_VERSION`
`static int`	`FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT`
`static String`	`FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED`
`static boolean`	`FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT`
`static String`	`FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED`
`static boolean`	`FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT`
`static String`	`FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS`
`static int`	`FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS_DEFAULT`
`static String`	`PENDING_DIR_NAME` Name of directory where pending data is placed.
`static String`	`SUCCEEDED_FILE_NAME`
`static String`	`SUCCESSFUL_JOB_OUTPUT_DIR_MARKER`
`protected static String`	`TEMP_DIR_NAME` 非推奨です。

コンストラクタのサマリー

コンストラクタ
コンストラクタと説明
`FileOutputCommitter(Path outputPath, JobContext context)` Create a file output committer
`FileOutputCommitter(Path outputPath, TaskAttemptContext context)` Create a file output committer

メソッドのサマリー

すべてのメソッド staticメソッドインスタンス・メソッド concreteメソッド非推奨のメソッド
修飾子とタイプ	メソッドと説明
`void`	`abortJob(JobContext context, org.apache.hadoop.mapreduce.JobStatus.State state)` Delete the temporary directory, including all of the work directories.
`void`	`abortTask(TaskAttemptContext context)` Delete the work directory
`void`	`cleanupJob(JobContext context)` 非推奨です。
`void`	`commitJob(JobContext context)` The job has completed, so do works in commitJobInternal().
`protected void`	`commitJobInternal(JobContext context)` The job has completed, so do following commit job, include: Move all committed tasks to the final output dir (algorithm 1 only).
`void`	`commitTask(TaskAttemptContext context)` Move the files from the work directory to the job output directory
`protected Path`	`getCommittedTaskPath(int appAttemptId, TaskAttemptContext context)` Compute the path where the output of a committed task is stored until the entire job is committed for a specific application attempt.
`Path`	`getCommittedTaskPath(TaskAttemptContext context)` Compute the path where the output of a committed task is stored until the entire job is committed.
`static Path`	`getCommittedTaskPath(TaskAttemptContext context, Path out)`
`protected Path`	`getJobAttemptPath(int appAttemptId)` Compute the path where the output of a given job attempt will be placed.
`Path`	`getJobAttemptPath(JobContext context)` Compute the path where the output of a given job attempt will be placed.
`static Path`	`getJobAttemptPath(JobContext context, Path out)` Compute the path where the output of a given job attempt will be placed.
`Path`	`getTaskAttemptPath(TaskAttemptContext context)` Compute the path where the output of a task attempt is stored until that task is committed.
`static Path`	`getTaskAttemptPath(TaskAttemptContext context, Path out)` Compute the path where the output of a task attempt is stored until that task is committed.
`Path`	`getWorkPath()` Get the directory that the task should write results into.
`boolean`	`isCommitJobRepeatable(JobContext context)` Returns true if an in-progress job commit can be retried.
`boolean`	`isRecoverySupported()` 非推奨です。
`boolean`	`needsTaskCommit(TaskAttemptContext context)` Did this task write any files in the work directory?
`void`	`recoverTask(TaskAttemptContext context)` Recover the task output.
`void`	`setupJob(JobContext context)` Create the temporary directory that is the root of all of the task work directories.
`void`	`setupTask(TaskAttemptContext context)` No task setup required.

クラスから継承されたメソッド org.apache.hadoop.mapreduce.OutputCommitter
isRecoverySupported

クラスから継承されたメソッド java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- フィールドの詳細
  - PENDING_DIR_NAME
```
public static final String PENDING_DIR_NAME
```
    Name of directory where pending data is placed. Data that has not been committed yet.
    
    関連項目:
    
    定数フィールド値
  - TEMP_DIR_NAME
```
@Deprecated
protected static final String TEMP_DIR_NAME
```
    非推奨です。
    
    Temporary directory name The static variable to be compatible with M/R 1.x
    
    関連項目:
    
    定数フィールド値
  - SUCCEEDED_FILE_NAME
```
public static final String SUCCEEDED_FILE_NAME
```
    関連項目:
    
    定数フィールド値
  - SUCCESSFUL_JOB_OUTPUT_DIR_MARKER
```
public static final String SUCCESSFUL_JOB_OUTPUT_DIR_MARKER
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_ALGORITHM_VERSION
```
public static final String FILEOUTPUTCOMMITTER_ALGORITHM_VERSION
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT
```
public static final int FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED
```
public static final String FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT
```
public static final boolean FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED
```
public static final String FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT
```
public static final boolean FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS
```
public static final String FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS
```
    関連項目:
    
    定数フィールド値
  - FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS_DEFAULT
```
public static final int FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS_DEFAULT
```
    関連項目:
    
    定数フィールド値
- コンストラクタの詳細
  - FileOutputCommitter
```
public FileOutputCommitter(Path outputPath,
                           TaskAttemptContext context)
                    throws IOException
```
    Create a file output committer
    
    パラメータ:
    
    outputPath - the job's output path, or null if you want the output committer to act as a noop.
    
    context - the task's context
    
    例外:
    
    IOException
  - FileOutputCommitter
```
@InterfaceAudience.Private
public FileOutputCommitter(Path outputPath,
                                                      JobContext context)
                                               throws IOException
```
    Create a file output committer
    
    パラメータ:
    
    outputPath - the job's output path, or null if you want the output committer to act as a noop.
    
    context - the task's context
    
    例外:
    
    IOException
- メソッドの詳細
  - getJobAttemptPath
```
public Path getJobAttemptPath(JobContext context)
```
    Compute the path where the output of a given job attempt will be placed.
    
    パラメータ:
    
    context - the context of the job. This is used to get the application attempt id.
    
    戻り値:
    
    the path to store job attempt data.
  - getJobAttemptPath
```
public static Path getJobAttemptPath(JobContext context,
                                     Path out)
```
    Compute the path where the output of a given job attempt will be placed.
    
    パラメータ:
    
    context - the context of the job. This is used to get the application attempt id.
    
    out - the output path to place these in.
    
    戻り値:
    
    the path to store job attempt data.
  - getJobAttemptPath
```
protected Path getJobAttemptPath(int appAttemptId)
```
    Compute the path where the output of a given job attempt will be placed.
    
    パラメータ:
    
    appAttemptId - the ID of the application attempt for this job.
    
    戻り値:
    
    the path to store job attempt data.
  - getTaskAttemptPath
```
public Path getTaskAttemptPath(TaskAttemptContext context)
```
    Compute the path where the output of a task attempt is stored until that task is committed.
    
    パラメータ:
    
    context - the context of the task attempt.
    
    戻り値:
    
    the path where a task attempt should be stored.
  - getTaskAttemptPath
```
public static Path getTaskAttemptPath(TaskAttemptContext context,
                                      Path out)
```
    Compute the path where the output of a task attempt is stored until that task is committed.
    
    パラメータ:
    
    context - the context of the task attempt.
    
    out - The output path to put things in.
    
    戻り値:
    
    the path where a task attempt should be stored.
  - getCommittedTaskPath
```
public Path getCommittedTaskPath(TaskAttemptContext context)
```
    Compute the path where the output of a committed task is stored until the entire job is committed.
    
    パラメータ:
    
    context - the context of the task attempt
    
    戻り値:
    
    the path where the output of a committed task is stored until the entire job is committed.
  - getCommittedTaskPath
```
public static Path getCommittedTaskPath(TaskAttemptContext context,
                                        Path out)
```
  - getCommittedTaskPath
```
protected Path getCommittedTaskPath(int appAttemptId,
                                    TaskAttemptContext context)
```
    Compute the path where the output of a committed task is stored until the entire job is committed for a specific application attempt.
    
    パラメータ:
    
    appAttemptId - the id of the application attempt to use
    
    context - the context of any task.
    
    戻り値:
    
    the path where the output of a committed task is stored.
  - getWorkPath
```
public Path getWorkPath()
                 throws IOException
```
    Get the directory that the task should write results into.
    
    戻り値:
    
    the work directory
    
    例外:
    
    IOException
  - setupJob
```
public void setupJob(JobContext context)
              throws IOException
```
    Create the temporary directory that is the root of all of the task work directories.
    
    定義:
    
    setupJob クラス内 OutputCommitter
    
    パラメータ:
    
    context - the job's context
    
    例外:
    
    IOException - if temporary output could not be created
  - commitJob
```
public void commitJob(JobContext context)
               throws IOException
```
    The job has completed, so do works in commitJobInternal(). Could retry on failure if using algorithm 2.
    
    オーバーライド:
    
    commitJob クラス内 OutputCommitter
    
    パラメータ:
    
    context - the job's context
    
    例外:
    
    IOException
  - commitJobInternal
```
protected void commitJobInternal(JobContext context)
                          throws IOException
```
    The job has completed, so do following commit job, include: Move all committed tasks to the final output dir (algorithm 1 only). Delete the temporary directory, including all of the work directories. Create a _SUCCESS file to make it as successful.
    
    パラメータ:
    
    context - the job's context
    
    例外:
    
    IOException
  - cleanupJob
```
@Deprecated
public void cleanupJob(JobContext context)
                            throws IOException
```
    非推奨です。
    
    クラスからコピーされた説明: OutputCommitter
    
    For cleaning up the job's output after job completion. This is called from the application master process for the entire job. This may be called multiple times.
    
    オーバーライド:
    
    cleanupJob クラス内 OutputCommitter
    
    パラメータ:
    
    context - Context of the job whose output is being written.
    
    例外:
    
    IOException
  - abortJob
```
public void abortJob(JobContext context,
                     org.apache.hadoop.mapreduce.JobStatus.State state)
              throws IOException
```
    Delete the temporary directory, including all of the work directories.
    
    オーバーライド:
    
    abortJob クラス内 OutputCommitter
    
    パラメータ:
    
    context - the job's context
    
    state - final runstate of the job
    
    例外:
    
    IOException
  - setupTask
```
public void setupTask(TaskAttemptContext context)
               throws IOException
```
    No task setup required.
    
    定義:
    
    setupTask クラス内 OutputCommitter
    
    パラメータ:
    
    context - Context of the task whose output is being written.
    
    例外:
    
    IOException
  - commitTask
```
public void commitTask(TaskAttemptContext context)
                throws IOException
```
    Move the files from the work directory to the job output directory
    
    定義:
    
    commitTask クラス内 OutputCommitter
    
    パラメータ:
    
    context - the task context
    
    例外:
    
    IOException - if commit is not successful.
  - abortTask
```
public void abortTask(TaskAttemptContext context)
               throws IOException
```
    Delete the work directory
    
    定義:
    
    abortTask クラス内 OutputCommitter
    
    例外:
    
    IOException
  - needsTaskCommit
```
public boolean needsTaskCommit(TaskAttemptContext context)
                        throws IOException
```
    Did this task write any files in the work directory?
    
    定義:
    
    needsTaskCommit クラス内 OutputCommitter
    
    パラメータ:
    
    context - the task's context
    
    戻り値:
    
    true/false
    
    例外:
    
    IOException
  - isRecoverySupported
```
@Deprecated
public boolean isRecoverySupported()
```
    非推奨です。
    
    クラスからコピーされた説明: OutputCommitter
    
    Is task output recovery supported for restarting jobs? If task output recovery is supported, job restart can be done more efficiently.
    
    オーバーライド:
    
    isRecoverySupported クラス内 OutputCommitter
    
    戻り値:
    
    true if task output recovery is supported, false otherwise
    
    関連項目:
    
    OutputCommitter.recoverTask(TaskAttemptContext)
  - isCommitJobRepeatable
```
public boolean isCommitJobRepeatable(JobContext context)
                              throws IOException
```
    クラスからコピーされた説明: OutputCommitter
    
    Returns true if an in-progress job commit can be retried. If the MR AM is re-run then it will check this value to determine if it can retry an in-progress commit that was started by a previous version. Note that in rare scenarios, the previous AM version might still be running at that time, due to system anomalies. Hence if this method returns true then the retry commit operation should be able to run concurrently with the previous operation. If repeatable job commit is supported, job restart can tolerate previous AM failures during job commit. By default, it is not supported. Extended classes (like: FileOutputCommitter) should explicitly override it if provide support.
    
    オーバーライド:
    
    isCommitJobRepeatable クラス内 OutputCommitter
    
    パラメータ:
    
    context - Context of the job whose output is being written.
    
    戻り値:
    
    true repeatable job commit is supported, false otherwise
    
    例外:
    
    IOException
  - recoverTask
```
public void recoverTask(TaskAttemptContext context)
                 throws IOException
```
    クラスからコピーされた説明: OutputCommitter
    
    Recover the task output. The retry-count for the job will be passed via the MRJobConfig.APPLICATION_ATTEMPT_ID key in JobContext.getConfiguration() for the OutputCommitter. This is called from the application master process, but it is called individually for each task. If an exception is thrown the task will be attempted again. This may be called multiple times for the same task. But from different application attempts.
    
    オーバーライド:
    
    recoverTask クラス内 OutputCommitter
    
    パラメータ:
    
    context - Context of the task whose output is being recovered
    
    例外:
    
    IOException

クラス FileOutputCommitter

フィールドのサマリー

コンストラクタのサマリー

メソッドのサマリー

クラスから継承されたメソッド org.apache.hadoop.mapreduce.OutputCommitter

クラスから継承されたメソッド java.lang.Object

フィールドの詳細

PENDING_DIR_NAME

TEMP_DIR_NAME

SUCCEEDED_FILE_NAME

SUCCESSFUL_JOB_OUTPUT_DIR_MARKER

FILEOUTPUTCOMMITTER_ALGORITHM_VERSION

FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT

FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED

FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT

FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED

FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT

FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS

FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS_DEFAULT

コンストラクタの詳細

FileOutputCommitter

FileOutputCommitter

メソッドの詳細

getJobAttemptPath

getJobAttemptPath

getJobAttemptPath

getTaskAttemptPath

getTaskAttemptPath

getCommittedTaskPath

getCommittedTaskPath

getCommittedTaskPath

getWorkPath

setupJob

commitJob

commitJobInternal

cleanupJob

abortJob

setupTask

commitTask

abortTask

needsTaskCommit

isRecoverySupported

isCommitJobRepeatable

recoverTask