运用CDH5.7.5 sqoop 1.4.6从sqlserver 导入数据到hive表中,报错,报错信息如下。
[root@node3 imp_sqoop]# sqoop --options-file alipay_systemlog
Warning: /opt/cloudera/parcels/CDH-5.7.5-1.cdh5.7.5.p0.3/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 17/05/04 16:01:58 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.7.5 17/05/04 16:01:58 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 17/05/04 16:01:58 INFO manager.SqlManager: Using default fetchSize of 1000 17/05/04 16:01:58 INFO tool.CodeGenTool: Beginning code generation 17/05/04 16:01:58 INFO tool.CodeGenTool: Will generate java class as codegen_alipay_systemlog 17/05/04 16:01:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [alipay_systemlog] AS t WHERE 1=0 17/05/04 16:01:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce Note: /tmp/sqoop-root/compile/2b31d0e634e6b8dcd9f287f88b7f8f33/codegen_alipay_systemlog.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 17/05/04 16:01:59 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/2b31d0e634e6b8dcd9f287f88b7f8f33/codegen_alipay_systemlog.jar 17/05/04 16:01:59 INFO tool.ImportTool: Destination directory /dianmei/sqoop/alipay_systemlog is not present, hence not deleting. 17/05/04 16:01:59 INFO mapreduce.ImportJobBase: Beginning import of alipay_systemlog 17/05/04 16:01:59 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 17/05/04 16:01:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [alipay_systemlog] AS t WHERE 1=0 17/05/04 16:01:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [alipay_systemlog] AS t WHERE 1=0 17/05/04 16:01:59 ERROR tool.ImportTool: Imported Failed: Cannot convert SQL type 2005
alipay_systemlog文件内容如下:
import --connect jdbc:jtds:sqlserver://10.97.0.144:1433;DatabaseName=dianmei --username sa --password root --table alipay_systemlog --map-column-hive createdate=string,type=string --as-parquetfile --fields-terminated-by '\t' --hive-import --hive-drop-import-delims --hive-overwrite --hive-database sqoop --hive-table alipay_systemlog --delete-target-dir --target-dir /dianmei/sqoop/alipay_systemlog --null-string '' --null-non-string '\\N' -m 1 --compression-codec org.apache.hadoop.io.compress.SnappyCodec
sqlserver中表alipay_systemlog表的结构如下所示:
id varchar no 50 no no no Chinese_PRC_CI_AS actionname varchar no 100 yes no yes Chinese_PRC_CI_AS compid varchar no 50 yes no yes Chinese_PRC_CI_AS compname varchar no 200 yes no yes Chinese_PRC_CI_AS alipaypartner varchar no 50 yes no yes Chinese_PRC_CI_AS out_trade_no varchar no 100 yes no yes Chinese_PRC_CI_AS trade_no varchar no 200 yes no yes Chinese_PRC_CI_AS message varchar no -1 yes no yes Chinese_PRC_CI_AS remark varchar no 2000 yes no yes Chinese_PRC_CI_AS createdate datetime no 8 yes (n/a) (n/a)
type int no 4 10 0 yes (n/a) (n/a)
把 --map-column-hive createdate=string,type=string 这个参数去掉或者随便删除一个都还是报这个错,搞了大半天了,有哪位高手能帮忙看下么,谢谢了。
|