分享

新手求助~jo执行没有问题,但是status job是失败的,数据也没导入Mysql..........

任务执行:
sqoop:000> start job --name FisrtJob
Submission details
Job Name: FisrtJob
Server URL: http://localhost:12000/sqoop/
Created by: hadoop
Creation date: 2017-04-06 17:57:43 CST
Lastly updated by: hadoop
External ID: job_1491448007523_0009
        http://namenode:8088/proxy/application_1491448007523_0009/
Target Connector schema: Schema{name= test . people ,columns=[
        FixedPoint{name=id,nullable=true,type=FIXED_POINT,byteSize=8,signed=true},
        Text{name=name,nullable=true,type=TEXT,charSize=null},
        Text{name=gender,nullable=true,type=TEXT,charSize=null}]}
2017-04-06 17:57:43 CST: BOOTING  - Progress is not available
sqoop:000>


任务属性失败:
sqoop:000> status job -n FisrtJob
Submission details
Job Name: FisrtJob
Server URL: http://localhost:12000/sqoop/
Created by: hadoop
Creation date: 2017-04-06 17:57:43 CST
Lastly updated by: hadoop
External ID: job_1491448007523_0009
        http://namenode:8088/proxy/application_1491448007523_0009/
2017-04-06 17:58:59 CST: FAILED
Exception: Job Failed with status:3
Stack trace:


任务信息:
sqoop:000> show job -all
1 job(s) to show:
Job with name FisrtJob (Enabled: true, Created by hadoop at 17-4-6 下午4:31, Updated by hadoop at 17-4-6 下午4:31)
  Throttling resources
    Extractors:
    Loaders:
  Classpath configuration
    Extra mapper jars:
From link: HDFS
  Input configuration
    Input directory: /toMysql
    Override null value:
    Null value:
  Incremental import
    Incremental type: NONE
    Last imported date:
To link: MYSQL
  Database target
    Schema name: test
    Table name: people
    Column names:
    Staging table:
    Clear stage table:
sqoop:000>

hdfs输入 数据:
[hadoop@namenode @LOGDIR@]$ hadoop fs -cat /toMysql/people.csv
1,Signal,male
2,Hathway,female
3,lee,beauty

mysql输出的表:
mysql> create table people( id bigint, name varchar(50), gender varchar(10) );




mysql people
里没有任何数据,求各位指教

已有(10)人评论

跳转到指定楼层
dsy198816 发表于 2017-4-6 18:04:52
日志里就两个警告也看不出来:
2017-04-06 17:57:43,928 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:57:43,942 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:57:43,943 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:57:43,944 INFO  [org.apache.sqoop.connector.hdfs.HdfsUtils.createConfiguration(HdfsUtils.java:66)] Found Hadoop configuration file yarn-site.xml
2017-04-06 17:57:43,944 INFO  [org.apache.sqoop.connector.hdfs.HdfsUtils.createConfiguration(HdfsUtils.java:66)] Found Hadoop configuration file httpfs-site.xml
2017-04-06 17:57:43,945 INFO  [org.apache.sqoop.connector.hdfs.HdfsUtils.createConfiguration(HdfsUtils.java:66)] Found Hadoop configuration file kms-site.xml
2017-04-06 17:57:43,945 INFO  [org.apache.sqoop.connector.hdfs.HdfsUtils.createConfiguration(HdfsUtils.java:66)] Found Hadoop configuration file hdfs-site.xml
2017-04-06 17:57:43,945 INFO  [org.apache.sqoop.connector.hdfs.HdfsUtils.createConfiguration(HdfsUtils.java:66)] Found Hadoop configuration file mapred-site.xml
2017-04-06 17:57:43,945 INFO  [org.apache.sqoop.connector.hdfs.HdfsUtils.createConfiguration(HdfsUtils.java:66)] Found Hadoop configuration file core-site.xml
2017-04-06 17:57:43,983 INFO  [org.apache.hadoop.conf.Configuration.warnOnceIfDeprecated(Configuration.java:1049)] fs.default.name is deprecated. Instead, use fs.defaultFS
2017-04-06 17:57:43,985 INFO  [org.apache.sqoop.connector.hdfs.HdfsFromInitializer$1.run(HdfsFromInitializer.java:68)] Input directory: /toMysql
2017-04-06 17:57:43,992 INFO  [org.apache.sqoop.connector.hadoop.security.SecurityUtils.generateDelegationTokens(SecurityUtils.java:75)] Running on unsecured cluster, skipping delegation token generation.
2017-04-06 17:57:44,000 INFO  [org.apache.sqoop.connector.jdbc.GenericJdbcToInitializer.configureTableProperties(GenericJdbcToInitializer.java:152)] Using query to insert data: INSERT INTO  test . people  VALUES (?,?,?)
2017-04-06 17:57:44,026 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:57:44,114 INFO  [org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:98)] Connecting to ResourceManager at namenode/192.168.0.137:8032
2017-04-06 17:57:44,168 WARN  [org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:153)] Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2017-04-06 17:57:53,825 WARN  [org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:261)] No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2017-04-06 17:57:53,857 INFO  [org.apache.hadoop.conf.Configuration.warnOnceIfDeprecated(Configuration.java:1049)] fs.default.name is deprecated. Instead, use fs.defaultFS
2017-04-06 17:57:53,857 INFO  [org.apache.sqoop.connector.hadoop.security.SecurityUtils.loadDelegationTokensToUGI(SecurityUtils.java:101)] No delegation tokens found
2017-04-06 17:57:53,864 INFO  [org.apache.sqoop.connector.hdfs.HdfsPartitioner$1.run(HdfsPartitioner.java:153)] Will process input file: hdfs://192.168.0.137:9000/toMysql/people.csv
2017-04-06 17:57:53,934 INFO  [org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:494)] number of splits:8
2017-04-06 17:57:54,447 INFO  [org.apache.hadoop.mapreduce.JobSubmitter.printTokens(JobSubmitter.java:583)] Submitting tokens for job: job_1491448007523_0009
2017-04-06 17:57:54,468 INFO  [org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:371)] Job jar is not present. Not adding any jar to the list of resources.
2017-04-06 17:57:54,550 INFO  [org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:251)] Submitted application application_1491448007523_0009
2017-04-06 17:57:54,563 INFO  [org.apache.hadoop.mapreduce.Job.submit(Job.java:1300)] The url to track the job: http://namenode:8088/proxy/application_1491448007523_0009/
2017-04-06 17:57:54,879 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:58:59,671 INFO  [org.apache.sqoop.handler.JobRequestHandler.handleEvent(JobRequestHandler.java:96)] Got job request
2017-04-06 17:58:59,673 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:58:59,675 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:58:59,676 INFO  [org.apache.sqoop.audit.FileAuditLogger.logAuditEvent(FileAuditLogger.java:61)] user=hadoop       ip=127.0.0.1    op=status       obj=job objId=FisrtJob
2017-04-06 17:58:59,691 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:58:59,705 INFO  [org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:276)] Application state is completed. FinalApplicationStatus=FAILED. Redirecting to job history server
2017-04-06 17:58:59,862 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:59:37,055 INFO  [org.apache.sqoop.handler.JobRequestHandler.handleEvent(JobRequestHandler.java:96)] Got job request
2017-04-06 17:59:37,055 INFO  [org.apache.sqoop.audit.FileAuditLogger.logAuditEvent(FileAuditLogger.java:61)] user=hadoop       ip=127.0.0.1    op=get  obj=job objId=all
2017-04-06 17:59:37,060 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 17:59:37,062 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2017-04-06 18:01:43,831 INFO  [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
                                                                                                                                                                          
回复

使用道具 举报

2017 发表于 2017-4-6 18:21:37
dsy198816 发表于 2017-4-6 18:04
日志里就两个警告也看不出来:
2017-04-06 17:57:43,928 INFO  [org.apache.sqoop.repository.JdbcReposito ...

可能数据类型不一致造成的,具体需要贴出你的job内容来
回复

使用道具 举报

dsy198816 发表于 2017-4-6 18:27:50
楼上朋友job内容是job的消息吗?上面不是吗?
sqoop:000> show job -all
1 job(s) to show:
Job with name FirstJob (Enabled: true, Created by hadoop at 17-4-6 下午6:24, Updated by hadoop at 17-4-6 下午6:24)
  Throttling resources
    Extractors:
    Loaders:
  Classpath configuration
    Extra mapper jars:
From link: HDFS
  Input configuration
    Input directory: /toMysql
    Override null value:
    Null value:
  Incremental import
    Incremental type: NONE
    Last imported date:
To link: MYSQL
  Database target
    Schema name: test
    Table name: people
    Column names:
    Staging table:
    Clear stage table:
sqoop:000>
回复

使用道具 举报

dsy198816 发表于 2017-4-6 18:29:07
输入内容:
[hadoop@namenode @LOGDIR@]$ hadoop fs -cat /toMysql/people.csv
2,dfc,axc
3,lkv,zxs

输出表格式:
mysql> desc people
    -> ;
+--------+-------------+------+-----+---------+-------+
| Field  | Type        | Null | Key | Default | Extra |
+--------+-------------+------+-----+---------+-------+
| id     | bigint(20)  | YES  |     | NULL    |       |
| name   | varchar(50) | YES  |     | NULL    |       |
| gender | varchar(10) | YES  |     | NULL    |       |
+--------+-------------+------+-----+---------+-------+
3 rows in set (0.00 sec)

回复

使用道具 举报

dsy198816 发表于 2017-4-6 18:43:55
运行时候非常奇怪,一个几十字节的很小的输入文件,输入mysql,running %0的状态持续了几分钟,才失败的。      
sqoop:000> status job -n FirstJob
Submission details
Job Name: FirstJob
Server URL: http://localhost:12000/sqoop/
Created by: hadoop
Creation date: 2017-04-06 18:41:19 CST
Lastly updated by: hadoop
External ID: job_1491475256133_0001
        http://namenode:8088/proxy/application_1491475256133_0001/
2017-04-06 18:41:58 CST: RUNNING  - 0.00 %
sqoop:000> status job -n FirstJob
Submission details
Job Name: FirstJob
Server URL: http://localhost:12000/sqoop/
Created by: hadoop
Creation date: 2017-04-06 18:41:19 CST
Lastly updated by: hadoop
External ID: job_1491475256133_0001
        http://namenode:8088/proxy/application_1491475256133_0001/
2017-04-06 18:42:30 CST: FAILED
Exception: Job Failed with status:3
sqoop:000>
回复

使用道具 举报

tanxl 发表于 2017-4-6 18:48:46
和我一样的问题 加个好友一起渡过难关 953292085
回复

使用道具 举报

2017 发表于 2017-4-6 18:51:19
dsy198816 发表于 2017-4-6 18:43
运行时候非常奇怪,一个几十字节的很小的输入文件,输入mysql,running %0的状态持续了几分钟,才失败的。  ...

创建的job对象,里面的内容。如下

[mw_shl_code=bash,true]sqoop job –create FirstJob— import –connectjdbc:mysql://localhost:3306/test –table
person[/mw_shl_code]

回复

使用道具 举报

dsy198816 发表于 2017-4-6 22:37:43
楼上的朋友。我用的sqoop2啊  /bin目录下都没有sqoop这个命令啊
回复

使用道具 举报

dsy198816 发表于 2017-4-6 22:38:51
求大牛来,解决我们的问题啊。
回复

使用道具 举报

12下一页
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条