kettle连接hive-Sqoop-About云-梭伦科技 - Powered by Discuz! Archiver

廉兵兵 发表于 2017-6-12 13:59:08

kettle连接hive

环境：
kettle 6.1
hivecdh5.3
hive驱动该用哪个？放哪里？还有能不能看出别的错？求大神们指教。。。

错误连接数据库 : org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class org.apache.hadoop.hive.jdbc.HiveDriver)
org/apache/hadoop/hive/metastore/api/MetaException

org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class org.apache.hadoop.hive.jdbc.HiveDriver)
org/apache/hadoop/hive/metastore/api/MetaException

at org.pentaho.di.core.database.Database.normalConnect(Database.java:466)
at org.pentaho.di.core.database.Database.connect(Database.java:364)
at org.pentaho.di.core.database.Database.connect(Database.java:335)
at org.pentaho.di.core.database.Database.connect(Database.java:325)
at org.pentaho.di.core.database.DatabaseFactory.getConnectionTestReport(DatabaseFactory.java:80)
at org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2734)
at org.pentaho.ui.database.event.DataHandler.testDatabaseConnection(DataHandler.java:591)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at org.pentaho.ui.xul.swt.tags.SwtButton.access$500(SwtButton.java:43)
at org.pentaho.ui.xul.swt.tags.SwtButton$4.widgetSelected(SwtButton.java:137)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:820)
at org.eclipse.jface.window.Window.open(Window.java:796)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:88)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:55)
at org.pentaho.di.ui.core.database.dialog.XulDatabaseDialog.open(XulDatabaseDialog.java:116)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:60)
at org.pentaho.di.ui.trans.step.BaseStepDialog.showDbDialogUnlessCancelledOrValid(BaseStepDialog.java:776)
at org.pentaho.di.ui.trans.step.BaseStepDialog$AddConnectionListener.widgetSelected(BaseStepDialog.java:1371)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.trans.steps.databasejoin.DatabaseJoinDialog.open(DatabaseJoinDialog.java:415)
at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:125)
at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:8783)
at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:3072)
at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:755)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1347)
at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7989)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9269)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:662)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)
Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
Error connecting to database: (using class org.apache.hadoop.hive.jdbc.HiveDriver)
org/apache/hadoop/hive/metastore/api/MetaException

at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:581)
at org.pentaho.di.core.database.Database.normalConnect(Database.java:450)
... 52 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException
at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at org.pentaho.di.core.database.DelegatingDriver.connect(DelegatingDriver.java:53)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:565)
... 53 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.MetaException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.pentaho.di.core.plugins.KettleURLClassLoader.loadClassFromParent(KettleURLClassLoader.java:89)
at org.pentaho.di.core.plugins.KettleURLClassLoader.loadClass(KettleURLClassLoader.java:108)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 58 more

主机名 : 192.168.111.13
端口 : 10000
数据库名:lianbingbing

desehawk 发表于 2017-6-12 14:50:04

将对应的mysql驱动包放到下kettle的目录：
data-integration\libext\JDBC下。
然后在看看mysql的权限，是否授权
GRANT ALL PRIVILEGES ON *.* TO ' 用户名'@'%' identified by '密码 ';

nextuser 发表于 2017-6-12 14:52:29

确认下hive是否启动
hive --service hiveserver2使用hiveserver2

廉兵兵 发表于 2017-6-12 15:34:27

nextuser 发表于 2017-6-12 14:52
确认下hive是否启动
hive --service hiveserver2使用hiveserver2

hiveserver2已启动，我感觉是hive driver的问题，cdh5.3的hive驱动包对应的是哪个？

廉兵兵 发表于 2017-6-12 15:35:37

desehawk 发表于 2017-6-12 14:50
将对应的mysql驱动包放到下kettle的目录：
data-integration\libext\JDBC下。
然后在看看mysql的权限，是 ...

亲，是kettle连接hive，不是mysql，mysql已成功连接。

desehawk 发表于 2017-6-12 16:04:36

廉兵兵发表于 2017-6-12 15:35
亲，是kettle连接hive，不是mysql，mysql已成功连接。

连接hive，也需要加入jdbc包。其它的应该没有需要加入的了。直接连接hive即可

ledasion 发表于 2017-6-23 17:52:38

连接方式使用 HiveServer2

页: [1]

About云-梭伦科技's Archiver

kettle连接hive