图文精华

About云-梭伦科技»专题 › 技术学习(版主发帖区) › 大数据学习 › Hadoop|YARN › 总结型› hadoop2.7 64位编译插件下载及编译过程记录、问题总结
分享

hadoop2.7 64位编译插件下载及编译过程记录、问题总结

nettman 2015-4-23 00:38:17 发表于 总结型 [显示全部楼层] 只看大图 回帖奖励 阅读模式 关闭右栏 10 144200
本帖最后由 nettman 于 2015-4-23 00:43 编辑
问题导读
1.什么情况下会发生Could not resolve dependencies？
2.什么情况下会出现No such file or directory？





1.hadoop2.7  64位包下载：

链接：http://pan.baidu.com/s/1o6kNDGY 密码：jpuj

2.jpg

2.编译按照这篇帖子即可成功：
从零教你在Linux环境下（ubuntu）如何编译hadoop2.4
最好在root下编译，否则可能会遇到权限问题




3.总结编译中遇到的两大问题：
1.插件下载不完全

表现：Could not resolve dependencies
例如：
Could not resolve dependencies for project org.apache.hadoop:hadoop-minikdc:jar:2.7.0:

解决办法：
这种情况很常见，而且很多都碰到了，他们也是完全按照文档来配置的，但是就不成功，这就是因为插件没有下载完毕造成的。所以尽量多执行几次下面命令
  1. mvn package -Pdist,native -DskipTests -Dtar
复制代码
2.权限问题

权限一般报找不到文件
表现
No such file or directory

例如：
Error executing ant tasks: /home/aboutyun/hadoop-2.7.0-src/hadoop-project/target/antrun/build-main.xml (No such file or directory) -> [Help 1]
解决办法：
在root下编译。然后确保所属权限属于root。
剩下的就是多编译几次


4.相关工具：

1.vmware10
链接：http://pan.baidu.com/s/1bIBuA 密码：3ss9

2.winscp516setup

链接：http://pan.baidu.com/s/1nt64rzj 密码：2109


3.jdk-7u67-linux-x64.tar
链接：http://pan.baidu.com/s/1pJ60syf 密码：9kee

4.protobuf-2.5.0.tar.gz
链接：http://pan.baidu.com/s/1dDgWe6P 密码：xgiu

5.apache-ant-1.9.4-bin.tar.gz
http://pan.baidu.com/s/1c0vjhBy





5.编译遇到问题
1.权限问题
本来已经改变了文件夹的权限，可是文件夹里面的还是没有改变
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [4:34.701s]
[INFO] Apache Hadoop Project POM ......................... FAILURE [29.710s]
[INFO] Apache Hadoop Annotations ......................... SKIPPED
[INFO] Apache Hadoop Assemblies .......................... SKIPPED
[INFO] Apache Hadoop Project Dist POM .................... SKIPPED
[INFO] Apache Hadoop Maven Plugins ....................... SKIPPED
[INFO] Apache Hadoop MiniKDC ............................. SKIPPED
[INFO] Apache Hadoop Auth ................................ SKIPPED
[INFO] Apache Hadoop Auth Examples ....................... SKIPPED
[INFO] Apache Hadoop Common .............................. SKIPPED
[INFO] Apache Hadoop NFS ................................. SKIPPED
[INFO] Apache Hadoop KMS ................................. SKIPPED
[INFO] Apache Hadoop Common Project ...................... SKIPPED
[INFO] Apache Hadoop HDFS ................................ SKIPPED
[INFO] Apache Hadoop HttpFS .............................. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................ SKIPPED
[INFO] Apache Hadoop HDFS Project ........................ SKIPPED
[INFO] hadoop-yarn ....................................... SKIPPED
[INFO] hadoop-yarn-api ................................... SKIPPED
[INFO] hadoop-yarn-common ................................ SKIPPED
[INFO] hadoop-yarn-server ................................ SKIPPED
[INFO] hadoop-yarn-server-common ......................... SKIPPED
[INFO] hadoop-yarn-server-nodemanager .................... SKIPPED
[INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
[INFO] hadoop-yarn-server-tests .......................... SKIPPED
[INFO] hadoop-yarn-client ................................ SKIPPED
[INFO] hadoop-yarn-server-sharedcachemanager ............. SKIPPED
[INFO] hadoop-yarn-applications .......................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SKIPPED
[INFO] hadoop-yarn-site .................................. SKIPPED
[INFO] hadoop-yarn-registry .............................. SKIPPED
[INFO] hadoop-yarn-project ............................... SKIPPED
[INFO] hadoop-mapreduce-client ........................... SKIPPED
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED
[INFO] hadoop-mapreduce-client-common .................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
[INFO] Apache Hadoop Distributed Copy .................... SKIPPED
[INFO] Apache Hadoop Archives ............................ SKIPPED
[INFO] Apache Hadoop Rumen ............................... SKIPPED
[INFO] Apache Hadoop Gridmix ............................. SKIPPED
[INFO] Apache Hadoop Data Join ........................... SKIPPED
[INFO] Apache Hadoop Ant Tasks ........................... SKIPPED
[INFO] Apache Hadoop Extras .............................. SKIPPED
[INFO] Apache Hadoop Pipes ............................... SKIPPED
[INFO] Apache Hadoop OpenStack support ................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support ......... SKIPPED
[INFO] Apache Hadoop Azure support ....................... SKIPPED
[INFO] Apache Hadoop Client .............................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............ SKIPPED
[INFO] Apache Hadoop Tools Dist .......................... SKIPPED
[INFO] Apache Hadoop Tools ............................... SKIPPED
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8:34.724s
[INFO] Finished at: Wed Apr 22 03:40:41 PDT 2015
[INFO] Final Memory: 28M/67M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (create-testdirs) on project hadoop-project: Error executing ant tasks: /home/aboutyun/hadoop-2.7.0-src/hadoop-project/target/antrun/build-main.xml (No such file or directory) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluen ... oExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-project

查看下权限，都是root
  1. aboutyun@ubuntu:~/hadoop-2.7.0-src$ ll
复制代码

  1. total 120

  2. drwxr-xr-x 15 aboutyun aboutyun  4096 Apr 10 11:39 ./

  3. drwxr-xr-x 24 aboutyun aboutyun  4096 Apr 22 03:32 ../

  4. -rw-r--r--  1 root     root     13110 Apr 10 11:35 BUILDING.txt

  5. drwxr-xr-x  2 root     root      4096 Apr 22 03:31 dev-support/

  6. drwxr-xr-x  3 root     root      4096 Apr 22 03:31 hadoop-assemblies/

  7. drwxr-xr-x  2 root     root      4096 Apr 22 03:31 hadoop-client/

  8. drwxr-xr-x 10 root     root      4096 Apr 22 03:31 hadoop-common-project/

  9. drwxr-xr-x  2 root     root      4096 Apr 22 03:31 hadoop-dist/

  10. drwxr-xr-x  6 root     root      4096 Apr 22 03:31 hadoop-hdfs-project/

  11. drwxr-xr-x  9 root     root      4096 Apr 22 03:31 hadoop-mapreduce-project/

  12. drwxr-xr-x  3 root     root      4096 Apr 22 03:31 hadoop-maven-plugins/

  13. drwxr-xr-x  2 root     root      4096 Apr 22 03:31 hadoop-minicluster/

  14. drwxr-xr-x  3 root     root      4096 Apr 22 03:31 hadoop-project/

  15. drwxr-xr-x  2 root     root      4096 Apr 22 03:31 hadoop-project-dist/

  16. drwxr-xr-x 16 root     root      4096 Apr 22 03:31 hadoop-tools/

  17. drwxr-xr-x  3 root     root      4096 Apr 22 03:31 hadoop-yarn-project/

  18. -rw-r--r--  1 root     root     15429 Apr 10 11:35 LICENSE.txt

  19. -rw-r--r--  1 root     root       101 Apr 10 11:35 NOTICE.txt

  20. -rw-r--r--  1 root     root     18893 Apr 10 11:35 pom.xml

  21. -rw-r--r--  1 root     root      1366 Apr 10 11:35 README.txt
复制代码
修改为aboutyun.
执行下面命令：
  1.  sudo chown aboutyun:aboutyun hadoop-2.7.0-src/*

复制代码


1.jpg


修改完毕

错误2
  1. Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/2.4.4/plexus-archiver-2.4.4.jar (161 KB at 34.7 KB/sec)

  2. [INFO] ------------------------------------------------------------------------

  3. [INFO] Reactor Summary:

  4. [INFO] 

  5. [INFO] Apache Hadoop Main ................................ SUCCESS [2.569s]

  6. [INFO] Apache Hadoop Project POM ......................... FAILURE [2:16.305s]

  7. [INFO] Apache Hadoop Annotations ......................... SKIPPED

  8. [INFO] Apache Hadoop Assemblies .......................... SKIPPED

  9. [INFO] Apache Hadoop Project Dist POM .................... SKIPPED

  10. [INFO] Apache Hadoop Maven Plugins ....................... SKIPPED

  11. [INFO] Apache Hadoop MiniKDC ............................. SKIPPED

  12. [INFO] Apache Hadoop Auth ................................ SKIPPED

  13. [INFO] Apache Hadoop Auth Examples ....................... SKIPPED

  14. [INFO] Apache Hadoop Common .............................. SKIPPED

  15. [INFO] Apache Hadoop NFS ................................. SKIPPED

  16. [INFO] Apache Hadoop KMS ................................. SKIPPED

  17. [INFO] Apache Hadoop Common Project ...................... SKIPPED

  18. [INFO] Apache Hadoop HDFS ................................ SKIPPED

  19. [INFO] Apache Hadoop HttpFS .............................. SKIPPED

  20. [INFO] Apache Hadoop HDFS BookKeeper Journal ............. SKIPPED

  21. [INFO] Apache Hadoop HDFS-NFS ............................ SKIPPED

  22. [INFO] Apache Hadoop HDFS Project ........................ SKIPPED

  23. [INFO] hadoop-yarn ....................................... SKIPPED

  24. [INFO] hadoop-yarn-api ................................... SKIPPED

  25. [INFO] hadoop-yarn-common ................................ SKIPPED

  26. [INFO] hadoop-yarn-server ................................ SKIPPED

  27. [INFO] hadoop-yarn-server-common ......................... SKIPPED

  28. [INFO] hadoop-yarn-server-nodemanager .................... SKIPPED

  29. [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED

  30. [INFO] hadoop-yarn-server-applicationhistoryservice ...... SKIPPED

  31. [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED

  32. [INFO] hadoop-yarn-server-tests .......................... SKIPPED

  33. [INFO] hadoop-yarn-client ................................ SKIPPED

  34. [INFO] hadoop-yarn-server-sharedcachemanager ............. SKIPPED

  35. [INFO] hadoop-yarn-applications .......................... SKIPPED

  36. [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED

  37. [INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SKIPPED

  38. [INFO] hadoop-yarn-site .................................. SKIPPED

  39. [INFO] hadoop-yarn-registry .............................. SKIPPED

  40. [INFO] hadoop-yarn-project ............................... SKIPPED

  41. [INFO] hadoop-mapreduce-client ........................... SKIPPED

  42. [INFO] hadoop-mapreduce-client-core ...................... SKIPPED

  43. [INFO] hadoop-mapreduce-client-common .................... SKIPPED

  44. [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED

  45. [INFO] hadoop-mapreduce-client-app ....................... SKIPPED

  46. [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED

  47. [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED

  48. [INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED

  49. [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED

  50. [INFO] hadoop-mapreduce .................................. SKIPPED

  51. [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED

  52. [INFO] Apache Hadoop Distributed Copy .................... SKIPPED

  53. [INFO] Apache Hadoop Archives ............................ SKIPPED

  54. [INFO] Apache Hadoop Rumen ............................... SKIPPED

  55. [INFO] Apache Hadoop Gridmix ............................. SKIPPED

  56. [INFO] Apache Hadoop Data Join ........................... SKIPPED

  57. [INFO] Apache Hadoop Ant Tasks ........................... SKIPPED

  58. [INFO] Apache Hadoop Extras .............................. SKIPPED

  59. [INFO] Apache Hadoop Pipes ............................... SKIPPED

  60. [INFO] Apache Hadoop OpenStack support ................... SKIPPED

  61. [INFO] Apache Hadoop Amazon Web Services support ......... SKIPPED

  62. [INFO] Apache Hadoop Azure support ....................... SKIPPED

  63. [INFO] Apache Hadoop Client .............................. SKIPPED

  64. [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED

  65. [INFO] Apache Hadoop Scheduler Load Simulator ............ SKIPPED

  66. [INFO] Apache Hadoop Tools Dist .......................... SKIPPED

  67. [INFO] Apache Hadoop Tools ............................... SKIPPED

  68. [INFO] Apache Hadoop Distribution ........................ SKIPPED

  69. [INFO] ------------------------------------------------------------------------

  70. [INFO] BUILD FAILURE

  71. [INFO] ------------------------------------------------------------------------

  72. [INFO] Total time: 2:25.054s

  73. [INFO] Finished at: Wed Apr 22 03:50:50 PDT 2015

  74. [INFO] Final Memory: 30M/71M

  75. [INFO] ------------------------------------------------------------------------

  76. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) on project hadoop-project: Execution hadoop-java-sources of goal org.apache.maven.plugins:maven-source-plugin:2.3:jar-no-fork failed: Plugin org.apache.maven.plugins:maven-source-plugin:2.3 or one of its dependencies could not be resolved: Could not transfer artifact org.apache.maven.reporting:maven-reporting-api:jar:2.0.6 from/to central (http://repo.maven.apache.org/maven2): GET request of: org/apache/maven/reporting/maven-reporting-api/2.0.6/maven-reporting-api-2.0.6.jar from central failed: Read timed out -> [Help 1]

  77. [ERROR] 

  78. [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

  79. [ERROR] Re-run Maven using the -X switch to enable full debug logging.

  80. [ERROR] 

  81. [ERROR] For more information about the errors and possible solutions, please read the following articles:

  82. [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

  83. [ERROR] 

  84. [ERROR] After correcting the problems, you can resume the build with the command

  85. [ERROR]   mvn <goals> -rf :hadoop-project

复制代码

执行了下
  1. mvn install
复制代码

第二个过了，后面又遇到了问题，文件夹里，还有文件里，而且还是root，最后直接切换root，免得在遇到权限问题

切换root遇到问题

  1. [ERROR] The build could not read 1 project -> [Help 1]

  2. [ERROR]   

  3. [ERROR]   The project org.apache.hadoop:hadoop-main:2.7.0 (/root/hadoop-2.7.0-src/pom.xml) has 1 error

  4. [ERROR]     Unresolveable build extension: Plugin org.apache.felix:maven-bundle-plugin:2.5.0 or one of its dependencies could not be resolved: Failed to collect dependencies for org.apache.felix:maven-bundle-plugin:jar:2.5.0 (): Failed to read artifact descriptor for org.apache.maven:maven-plugin-parameter-documenter:jar:2.0.6: Could not transfer artifact org.apache.maven:maven-plugin-parameter-documenter:pom:2.0.6 from/to central (http://repo.maven.apache.org/maven2): Read timed out -> [Help 2]

  5. [ERROR] 

  6. [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

  7. [ERROR] Re-run Maven using the -X switch to enable full debug logging.

  8. [ERROR] 

  9. [ERROR] For more information about the errors and possible solutions, please read the following articles:

  10. [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException

  11. [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

复制代码

又遇到
  1. [INFO] ------------------------------------------------------------------------

  2. [ERROR] Failed to execute goal on project hadoop-minikdc: Could not resolve dependencies for project org.apache.hadoop:hadoop-minikdc:jar:2.7.0: The following artifacts could not be resolved: org.apache.directory.server:apacheds-core-api:jar:2.0.0-M15, org.apache.directory.api:api-ldap-codec-core:jar:1.0.0-M20, org.apache.directory.api:api-ldap-extras-aci:jar:1.0.0-M20, org.apache.directory.api:api-ldap-extras-util:jar:1.0.0-M20, org.apache.directory.api:api-ldap-model:jar:1.0.0-M20, org.apache.directory.api:api-util:jar:1.0.0-M20, org.apache.mina:mina-core:jar:2.0.0-M5, net.sf.ehcache:ehcache-core:jar:2.4.4, org.apache.directory.server:apacheds-interceptors-admin:jar:2.0.0-M15: Could not transfer artifact org.apache.directory.server:apacheds-core-api:jar:2.0.0-M15 from/to central (http://repo.maven.apache.org/maven2): GET request of: org/apache/directory/server/apacheds-core-api/2.0.0-M15/apacheds-core-api-2.0.0-M15.jar from central failed: Read timed out -> [Help 1]

复制代码


总结：
遇到这种情况，一般都是插件没有下载
  1. Could not resolve dependencies for project org.apache.hadoop:hadoop-minikdc:jar:2.7.0:
复制代码
可以多次执行，下载完毕。

1.jpg


参考：

从零教你在Linux环境下（ubuntu 12.04）如何编译hadoop2.4






加微信w3aboutyun，可拉入技术爱好者群
回复

使用道具 举报

自定义广告语

已有(10)人评论

跳转到指定楼层
正序浏览
QQo0oBIQ 发表于 2015-4-23 09:05:21
回复

使用道具 举报

lgwhwyhd 发表于 2015-4-23 09:15:53
正需要,谢共享!
回复

使用道具 举报

kanaka10 发表于 2015-4-23 12:15:09
收藏了   谢谢分享  
回复

使用道具 举报

rossette 发表于 2015-4-23 16:01:37
谢谢分享O(∩_∩)O哈！
回复

使用道具 举报

hb1984 发表于 2015-4-23 18:12:54
谢谢楼主分享。        
回复

使用道具 举报

masterrest 发表于 2015-4-23 22:14:17
看看呢，学习下
回复

使用道具 举报

awenkidz 发表于 2015-4-29 21:32:24
等了很久了，疯狂啦。
回复

使用道具 举报

12下一页
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

发表新帖
nettman

超级版主

关注

  • 2911

    主题

  • 3204

    帖子

  • 478

    粉丝

TA的主题
关闭

推荐上一条 /2 下一条

关于我们· 联系我们· 加入我们· 小黑屋· 合作伙伴   

Copyright © 2001-2025 About云-梭伦科技 Powered by Discuz! X3.4 Licensed Discuz Team.

简书 / Smrz 京ICP备2020039040号 Wxb 简书网举报电话：021-34700000