Hadoop分析Tomcat访问日志实例：命令的使用

概述：分析对象Tomcat日志，讲述了如何将日志放入hadoop,如何分析日志，如何查看结果
日志样例：
Tomcat日志代码

172.16.2.12,-,-,[06/Sep/2011:10:03:13 +0800],GET /icestat/jpivot/toolbar/sort-asc-up.png HTTP/1.1,200,336,-,Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2)
172.16.2.12,-,-,[06/Sep/2011:09:48:17 +0800],GET /icestat/ HTTP/1.1,200,171,http://10.65.11.241:8080/icestat/,Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.43 Safari/534.24

172.16.2.12,-,-,[06/Sep/2011:10:03:13 +0800],GET /icestat/jpivot/toolbar/sort-asc-up.png HTTP/1.1,200,336,-,Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2)172.16.2.12,-,-,[06/Sep/2011:09:48:17 +0800],GET /icestat/ HTTP/1.1,200,171,http://10.65.11.241:8080/icestat/,Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.43 Safari/534.24
把日志放入hdfs
命令行代码

Hadoop fs -put icestat_access_log.2011-09-06.txt icestat_access_log.2011-09-06.txt

hadoop fs -put icestat_access_log.2011-09-06.txt icestat_access_log.2011-09-06.txt 分析日志
命令行代码

hadoop jar tomcatLog.jar icestat_access_log.2011-09-06.txt output6

hadoop jar tomcatLog.jar icestat_access_log.2011-09-06.txt output6 查看输出
命令行代码

[root@www.linuxidc.com hadoop-0.20.2]# hadoop fs -cat output6/part-r-00000

11/09/06 00:18:54 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively

11/09/06 00:18:54 WARN fs.FileSystem: "xxx :9000" is a deprecated filesystem name. Use "hdfs://xxx :9000/" instead.

chrome 58

firefox 23

other 49

safari 5

图文精华

Hadoop分析Tomcat访问日志实例：命令的使用

推荐 /2