图文精华

About云-梭伦科技»专题 › 交流区 › 技术交流 › CM 安装的oozie调用hive任务报找不到数据库

CM 安装的oozie调用hive任务报找不到数据库

查看数: 21041 | 评论数: 13 | 收藏 0
关灯 | 提示：支持键盘翻页<-左 右->
帖子模式
    组图打开中，请稍候......
liuzhixin137
发布时间: 2016-7-12 14:29

正文摘要:

oozie调用简单的hive语句， use mytest99; select * from test where id > 500; 但是很奇怪的报错， Script [script.q] content: ---------------------------- Licensed to the Apache Software Founda ...

回复

liuzhixin137 发表于 2016-7-18 09:24:48
http://blog.sina.com.cn/s/blog_e699b42b0102xg4e.html
liuzhixin137 发表于 2016-7-13 14:46:25
问题已解决，是我自己弄错了metastore server的端口。
liuzhixin137 发表于 2016-7-12 20:23:51
tntzbzc 发表于 2016-7-12 18:17
补充：
最好去掉这个属性

好的 ，我试试 。多谢！
tntzbzc 发表于 2016-7-12 18:17:27
tntzbzc 发表于 2016-7-12 18:03
这是服务端的配置文件，这里设置为
  
        hive.metastore.local  

补充：
最好去掉这个属性
tntzbzc 发表于 2016-7-12 18:03:37
本帖最后由 tntzbzc 于 2016-7-12 18:17 编辑
liuzhixin137 发表于 2016-7-12 17:23
这是我的hive-metastore 的日志 2016-07-12 14:19:29,512 INFO  org.apache.hadoop.hive.ql.log.PerfLogger ...

这是服务端的配置文件，这里设置为
<property>  
        <name>hive.metastore.local</name>  
        <value>true</value>
  </property>

liuzhixin137 发表于 2016-7-12 17:23:13
这是我的hive-metastore 的日志 2016-07-12 14:19:29,512 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_all_databases from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:19:29,513 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_all_databases
2016-07-12 14:19:29,513 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_all_databases       
2016-07-12 14:19:29,521 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_all_databases start=1468304369512 end=1468304369521 duration=9 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:00,244 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=create_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:00,245 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 create_database: Database(name:mytest99, description:null, locationUri:null, parameters:null, ownerName:root, ownerType:USER)
2016-07-12 14:20:00,245 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 create_database: Database(name:mytest99, description:null, locationUri:null, parameters:null, ownerName:root, ownerType:USER)       
2016-07-12 14:20:00,267 INFO  org.apache.hadoop.hive.common.FileUtils: [pool-4-thread-2]: Creating directory if it doesn't exist: hdfs://hadoop2:8020/user/hive/warehouse/mytest99.db
2016-07-12 14:20:00,296 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=create_database start=1468304400244 end=1468304400296 duration=52 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:04,129 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:04,130 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_database: mytest99
2016-07-12 14:20:04,131 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_database: mytest99       
2016-07-12 14:20:04,138 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_database start=1468304404129 end=1468304404138 duration=9 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:04,162 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:04,163 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_database: mytest99
2016-07-12 14:20:04,163 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_database: mytest99       
2016-07-12 14:20:04,170 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_database start=1468304404162 end=1468304404170 duration=8 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:04,172 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:04,172 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_database: mytest99
2016-07-12 14:20:04,173 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_database: mytest99       
2016-07-12 14:20:04,179 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_database start=1468304404172 end=1468304404179 duration=7 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:10,634 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_database from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:10,635 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_database: mytest99
2016-07-12 14:20:10,635 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_database: mytest99       
2016-07-12 14:20:10,643 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_database start=1468304410634 end=1468304410643 duration=9 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:10,668 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=create_table_with_environment_context from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:10,668 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 create_table: Table(tableName:test, dbName:mytest99, owner:root, createTime:1468304410, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:id, type:int, comment:null)], location:null, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format= , field.delim= }), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{}, groupPrivileges:null, rolePrivileges:null), temporary:false)
2016-07-12 14:20:10,668 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 create_table: Table(tableName:test, dbName:mytest99, owner:root, createTime:1468304410, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:id, type:int, comment:null)], location:null, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format= , field.delim= }), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{}, groupPrivileges:null, rolePrivileges:null), temporary:false)       
2016-07-12 14:20:10,688 INFO  org.apache.hadoop.hive.common.FileUtils: [pool-4-thread-2]: Creating directory if it doesn't exist: hdfs://hadoop2:8020/user/hive/warehouse/mytest99.db/test
2016-07-12 14:20:10,828 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=create_table_with_environment_context start=1468304410667 end=1468304410828 duration=161 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:15,289 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:15,290 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_table : db=mytest99 tbl=test
2016-07-12 14:20:15,290 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_table : db=mytest99 tbl=test       
2016-07-12 14:20:15,323 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_table start=1468304415289 end=1468304415323 duration=34 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:15,332 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:15,332 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_table : db=mytest99 tbl=test
2016-07-12 14:20:15,332 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_table : db=mytest99 tbl=test       
2016-07-12 14:20:15,350 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_table start=1468304415332 end=1468304415350 duration=18 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:15,352 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:15,353 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_table : db=mytest99 tbl=test
2016-07-12 14:20:15,353 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_table : db=mytest99 tbl=test       
2016-07-12 14:20:15,368 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_table start=1468304415352 end=1468304415368 duration=16 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:15,409 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=alter_table_with_cascade from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:15,409 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 alter_table: db=mytest99 tbl=test newtbl=test
2016-07-12 14:20:15,409 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 alter_table: db=mytest99 tbl=test newtbl=test       
2016-07-12 14:20:15,447 INFO  hive.log: [pool-4-thread-2]: Updating table stats fast for test
2016-07-12 14:20:15,447 INFO  hive.log: [pool-4-thread-2]: Updated size of table test to 151
2016-07-12 14:20:15,457 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=alter_table_with_cascade start=1468304415409 end=1468304415457 duration=48 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:15,459 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:15,459 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_table : db=mytest99 tbl=test
2016-07-12 14:20:15,460 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_table : db=mytest99 tbl=test       
2016-07-12 14:20:15,474 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_table start=1468304415459 end=1468304415474 duration=15 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:15,475 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:15,475 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_table : db=mytest99 tbl=test
2016-07-12 14:20:15,476 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_table : db=mytest99 tbl=test       
2016-07-12 14:20:15,491 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_table start=1468304415475 end=1468304415491 duration=16 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:15,499 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=alter_table_with_cascade from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:15,500 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 alter_table: db=mytest99 tbl=test newtbl=test
2016-07-12 14:20:15,500 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 alter_table: db=mytest99 tbl=test newtbl=test       
2016-07-12 14:20:15,536 INFO  hive.log: [pool-4-thread-2]: Updating table stats fast for test
2016-07-12 14:20:15,536 INFO  hive.log: [pool-4-thread-2]: Updated size of table test to 151
2016-07-12 14:20:15,541 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=alter_table_with_cascade start=1468304415499 end=1468304415541 duration=42 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>
2016-07-12 14:20:33,008 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: <PERFLOG method=get_table from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
2016-07-12 14:20:33,009 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-2]: 2: source:192.168.1.104 get_table : db=mytest99 tbl=test
2016-07-12 14:20:33,010 INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]: ugi=root        ip=192.168.1.104        cmd=source:192.168.1.104 get_table : db=mytest99 tbl=test       
2016-07-12 14:20:33,056 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [pool-4-thread-2]: </PERFLOG method=get_table start=1468304433008 end=1468304433056 duration=48 from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2 retryCount=0 error=false>

看了一下，里面的配置都正常，也没有报错
liuzhixin137 发表于 2016-7-12 17:12:49
easthome001 发表于 2016-7-12 17:01
楼主的意思是创建数据库，hdfs文件有了，但是mysql数据库中没有这个数据？

对啊，很奇怪。
tntzbzc 发表于 2016-7-12 17:08:07
贴下服务端和客户端hive-site.xml的配置文件
关闭

推荐上一条 /2 下一条

关于我们· 联系我们· 加入我们· 小黑屋· 合作伙伴   

Copyright © 2001-2024 About云-梭伦科技 Powered by Discuz! X3.4 Licensed Discuz Team.

简书 / Smrz 京ICP备2020039040号 Wxb 简书网举报电话：021-34700000