1234

-client-core-2.2.0.jar:$HADOOP_HOME/share/hadoop/common/lib/commons-cli-1.2.jar -d wordcount_classes WordCount.java
/usr/local/hadoop/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/Path.class): warning: Cannot find annotation method 'value()' in type 'LimitedPrivate': class file for org.apache.hadoop.classification.InterfaceAudience not found
Note: WordCount.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
1 warning
hduser@qwe-VirtualBox:~/javacode/wordcount$ jar cvf mywordcount.jar -C wordcount_classes/ .
added manifest
adding: hadoop/(in = 0) (out= 0)(stored 0%)
adding: hadoop/mytest/(in = 0) (out= 0)(stored 0%)
adding: hadoop/mytest/WordCount$TokenizerMapper.class(in = 1855) (out= 806)(deflated 56%)
adding: hadoop/mytest/WordCount$IntSumCombiner.class(in = 2680) (out= 1135)(deflated 57%)
adding: hadoop/mytest/WordCount.class(in = 2010) (out= 1040)(deflated 48%)
adding: hadoop/mytest/WordCount$ExpPartitioner.class(in = 896) (out= 464)(deflated 48%)
adding: hadoop/mytest/WordCount$IntSumReducer.class(in = 2655) (out= 1110)(deflated 58%)
hduser@qwe-VirtualBox:~/javacode/wordcount$ hadoop jar mywordcount.jar hadoop.mytest.WordCount myinput/input /home/hduser/output
14/01/30 05:04:17 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/01/30 05:04:17 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
14/01/30 05:04:19 INFO input.FileInputFormat: Total input paths to process : 2
14/01/30 05:04:20 INFO mapreduce.JobSubmitter: number of splits:2
14/01/30 05:04:20 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
14/01/30 05:04:20 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
14/01/30 05:04:20 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
14/01/30 05:04:20 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
14/01/30 05:04:20 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
14/01/30 05:04:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local837299323_0001
14/01/30 05:04:21 WARN conf.Configuration: file:/app/hadoop/tmp/mapred/staging/hduser837299323/.staging/job_local837299323_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
14/01/30 05:04:21 WARN conf.Configuration: file:/app/hadoop/tmp/mapred/staging/hduser837299323/.staging/job_local837299323_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
14/01/30 05:04:21 WARN conf.Configuration: file:/app/hadoop/tmp/mapred/local/localRunner/hduser/job_local837299323_0001/job_local837299323_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
14/01/30 05:04:21 WARN conf.Configuration: file:/app/hadoop/tmp/mapred/local/localRunner/hduser/job_local837299323_0001/job_local837299323_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
14/01/30 05:04:21 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
14/01/30 05:04:21 INFO mapreduce.Job: Running job: job_local837299323_0001
14/01/30 05:04:22 INFO mapred.LocalJobRunner: OutputCommitter set in config null
14/01/30 05:04:22 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
14/01/30 05:04:22 INFO mapred.LocalJobRunner: Waiting for map tasks
14/01/30 05:04:22 INFO mapred.LocalJobRunner: Starting task: attempt_local837299323_0001_m_000000_0
14/01/30 05:04:22 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
14/01/30 05:04:22 INFO mapreduce.Job: Job job_local837299323_0001 running in uber mode : false
14/01/30 05:04:23 INFO mapred.MapTask: Processing split: hdfs://localhost:54310/user/hduser/myinput/input/text1:0+78
14/01/30 05:04:23 INFO mapreduce.Job: map 0% reduce 0%
14/01/30 05:04:23 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
14/01/30 05:04:26 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
14/01/30 05:04:26 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
14/01/30 05:04:26 INFO mapred.MapTask: soft limit at 83886080
14/01/30 05:04:26 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
14/01/30 05:04:26 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
14/01/30 05:04:27 INFO mapred.LocalJobRunner:
14/01/30 05:04:27 INFO mapred.MapTask: Starting flush of map output
14/01/30 05:04:27 INFO mapred.MapTask: Spilling map output
14/01/30 05:04:27 INFO mapred.MapTask: bufstart = 0; bufend = 166; bufvoid = 104857600
14/01/30 05:04:27 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214312(104857248); length = 85/6553600
14/01/30 05:04:28 INFO mapred.MapTask: Finished spill 0
14/01/30 05:04:28 INFO mapred.Task: Task:attempt_local837299323_0001_m_000000_0 is done. And is in the process of committing
14/01/30 05:04:28 INFO mapred.LocalJobRunner: map
14/01/30 05:04:28 INFO mapred.Task: Task 'attempt_local837299323_0001_m_000000_0' done.
14/01/30 05:04:28 INFO mapred.LocalJobRunner: Finishing task: attempt_local837299323_0001_m_000000_0
14/01/30 05:04:28 INFO mapred.LocalJobRunner: Starting task: attempt_local837299323_0001_m_000001_0
14/01/30 05:04:28 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
14/01/30 05:04:28 INFO mapred.MapTask: Processing split: hdfs://localhost:54310/user/hduser/myinput/input/text2:0+47
14/01/30 05:04:28 INFO mapreduce.Job: map 100% reduce 0%
14/01/30 05:04:28 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
14/01/30 05:04:28 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
14/01/30 05:04:28 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
14/01/30 05:04:28 INFO mapred.MapTask: soft limit at 83886080
14/01/30 05:04:28 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
14/01/30 05:04:28 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
14/01/30 05:04:28 INFO mapred.LocalJobRunner:
14/01/30 05:04:28 INFO mapred.MapTask: Starting flush of map output
14/01/30 05:04:28 INFO mapred.MapTask: Spilling map output
14/01/30 05:04:28 INFO mapred.MapTask: bufstart = 0; bufend = 87; bufvoid = 104857600
14/01/30 05:04:28 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
14/01/30 05:04:28 INFO mapred.MapTask: Finished spill 0
14/01/30 05:04:28 INFO mapred.Task: Task:attempt_local837299323_0001_m_000001_0 is done. And is in the process of committing
14/01/30 05:04:28 INFO mapred.LocalJobRunner: map
14/01/30 05:04:28 INFO mapred.Task: Task 'attempt_local837299323_0001_m_000001_0' done.
14/01/30 05:04:28 INFO mapred.LocalJobRunner: Finishing task: attempt_local837299323_0001_m_000001_0
14/01/30 05:04:28 INFO mapred.LocalJobRunner: Map task executor complete.
14/01/30 05:04:29 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
14/01/30 05:04:29 INFO mapred.Merger: Merging 2 sorted segments
14/01/30 05:04:29 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 406 bytes
14/01/30 05:04:29 INFO mapred.LocalJobRunner:
14/01/30 05:04:30 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
14/01/30 05:04:31 INFO mapred.Task: Task:attempt_local837299323_0001_r_000000_0 is done. And is in the process of committing
14/01/30 05:04:31 INFO mapred.LocalJobRunner:
14/01/30 05:04:31 INFO mapred.Task: Task attempt_local837299323_0001_r_000000_0 is allowed to commit now
14/01/30 05:04:31 INFO output.FileOutputCommitter: Saved output of task 'attempt_local837299323_0001_r_000000_0' to hdfs://localhost:54310/home/hduser/output/_temporary/0/task_local837299323_0001_r_000000
14/01/30 05:04:31 INFO mapred.LocalJobRunner: reduce > reduce
14/01/30 05:04:31 INFO mapred.Task: Task 'attempt_local837299323_0001_r_000000_0' done.
14/01/30 05:04:32 INFO mapreduce.Job: map 100% reduce 100%
14/01/30 05:04:32 INFO mapreduce.Job: Job job_local837299323_0001 completed successfully
14/01/30 05:04:32 INFO mapreduce.Job: Counters: 32
File System Counters
FILE: Number of bytes read=19809
FILE: Number of bytes written=575368
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=328
HDFS: Number of bytes written=335
HDFS: Number of read operations=25
HDFS: Number of large read operations=0
HDFS: Number of write operations=5
Map-Reduce Framework
Map input records=8
Map output records=32
Map output bytes=253
Map output materialized bytes=470
Input split bytes=238
Combine input records=32
Combine output records=34
Reduce input groups=30
Reduce shuffle bytes=0
Reduce input records=34
Reduce output records=34
Spilled Records=68
Shuffled Maps =0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=302
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Total committed heap usage (bytes)=437968896
File Input Format Counters
Bytes Read=125
File Output Format Counters
Bytes Written=335