Cloudera 5.10.0安装遇到的问题和解决方案
本帖最后由 醉半城 于 2017-4-19 17:51 编辑1. Unable to retrieve non-local non-loopback IP address. Seeing address: cm/127.0.0.1
2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive: Unable to retrieve non-local non-loopback IP address. Seeing address: cm/127.0.0.1. 2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive failed. Bootup = false2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive was not able to access CM identity to validate it.2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive is deferring the validation to the next run in 15 seconds.2017-04-18 09:40:29,308 WARN ScmActive-0:com.cloudera.enterprise.AbstractWrappedEntityManager: Rolling back transaction that wasn't marked for rollback-only.java.lang.Exception: Non-thrown exception for stack trace.
at com.cloudera.enterprise.AbstractWrappedEntityManager.close(AbstractWrappedEntityManager.java:161)
at com.cloudera.cmf.persist.CmfEntityManager.close(CmfEntityManager.java:356)
at com.cloudera.server.cmf.components.ScmActive.markScmActive(ScmActive.java:224)
at com.cloudera.server.cmf.components.ScmActive.run(ScmActive.java:87)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
解决方案：
是由于host文件导致，我的host文件多了一行127.0.0.1 cm,删除这行就OK了
2. Unable to create new directory at /var/lib/cloudera-host-monitor/ts/ts_entity_metadata
2017-04-19 09:44:37,403 INFO com.cloudera.cmon.tstore.leveldb.LDBUtils: Creating directory /var/lib/cloudera-host-monitor/ts/ts_entity_metadata
2017-04-19 09:44:37,404 ERROR com.cloudera.cmon.firehose.Main: Error creating LevelDB timeseries store in directory /var/lib/cloudera-host-monitor/ts
java.io.IOException: Unable to create new directory at /var/lib/cloudera-host-monitor/ts/ts_entity_metadata
at com.cloudera.cmon.tstore.leveldb.LDBUtils.openVersionedDB(LDBUtils.java:239)
at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesMetadataStore.openMetadataDB(LDBTimeSeriesMetadataStore.java:148)
at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesMetadataStore.<init>(LDBTimeSeriesMetadataStore.java:138)
at com.cloudera.cmon.firehose.Main.main(Main.java:452)
解决方案：
是由于文件夹权限导致，修改文件夹拥有人chown cloudera-scm：cloudera-scm /var/lib/cloudera-host-monitor和/var/lib/cloudera-service-monitor
3. ValueError: too many values to unpack
出现这个问题会导致parcel一直处于分发阶段
25170 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels
25170 MainThread parcel INFO Agent does create users/groups and apply file permissions
25170 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache
25170 MainThread agent ERROR Caught unexpected exception in main loop.
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 710, in __issue_heartbeat
self._init_after_first_heartbeat_response(resp_data)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 947, in _init_after_first_heartbeat_response
self.client_configs.load()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 682, in load
new_deployed.update(self._lookup_alternatives(fname))
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 432, in _lookup_alternatives
return self._parse_alternatives(alt_name, out)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 444, in _parse_alternatives
path, _, _, priority_str = line.rstrip().split(" ")
ValueError: too many values to unpack
解决方案：
修改/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py脚本的第444行代码.
修改为：
for line in output.splitlines():
if line.startswith("/"):
if len(line.rstrip().split(" "))<=4:
path, _, _, priority_str = line.rstrip().split(" ")
# Ignore the alternative if it's not managed by CM.
if CM_MAGIC_PREFIX not in os.path.basename(path):
continue
try:
priority = int(priority_str)
except ValueError:
THROTTLED_LOG.info("Failed to parse %s: %s", name, line)
key = ClientConfigKey(name, path)
value = ClientConfigValue(priority, self._read_generation(path))
ret = value
else:
pass
return ret
具体可以参考：http://blog.csdn.net/qq_23660243/article/details/60870527 感谢这位同学的解决方案.
4. Error, CM server guid updated
3700 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels
3700 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels
3700 MainThread agent WARNINGExpected user root for /opt/cloudera/parcels but was cloudera-scm
3700 MainThread agent WARNINGExpected group root for /opt/cloudera/parcels but was cloudera-scm
3700 MainThread parcel INFO Agent does create users/groups and apply file permissions
3700 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache
3700 MainThread agent ERROR Error, CM server guid updated, expected 0fd6eac4-d1dc-4b46-90bb-b58c87fa3d1f, received c9bb909f-e7c0-4d56-b33e-26e3764adae8
解决方案：
rm -f /var/lib/cloudera-scm-agent/cm_guid 然后 service cloudera-scm-agent restart
5. Failed request to SCM: 302
2017-04-19 15:01:36,610 INFO com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
2017-04-19 15:01:37,610 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2017-04-19 15:01:37,650 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2017-04-19 15:01:37,653 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.
解决方案：
官方回复是
The "Failed request to SCM: 302" message occurs when the Host Monitor attempts to communicate to Cloudera Manager but the session has expired.The Host Monitor acts as a client that authenticates to SCM, so it is subject to session restrictions.
The error can be ignored as the Host Monitor will re-authenticate and we do see that in your case that occurs.
In fact, a code change to make the message an INFO message rather than ERROR is slated for Cloudera Manager 5.7.2 and 5.8.2.
没啥大的影响，但我用的是5.10.0......
6. Unable to find the JDBC database jar on host
这个属于安装oozie和hive时用mysql作为元数据库时遇到的
解决方案：将mysql的连接驱动放到 /usr/share/java/mysql-connector-java.jar
7. 安装Kafka时java.lang.OutOfMemoryError: Java heap space
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="/usr/lib64/cmf/service/common/killparent.sh"
# Executing /bin/sh -c "/usr/lib64/cmf/service/common/killparent.sh"...
Wed Apr 19 16:15:33 CST 2017
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Using -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/kafka_kafka-KAFKA_BROKER-c07d94c57862124dad7ce85095d99c9f_pid22663.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh as CSD_JAVA_OPTS
Using /run/cloudera-scm-agent/process/114-kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script
CONF_DIR=/run/cloudera-scm-agent/process/114-kafka-KAFKA_BROKER
CMF_CONF_DIR=/etc/cloudera-scm-agent
解决方案： 修改Kafka的Java Heap Size of Broker 设置为broker_max_heap_size=256
感谢分享 levycui 发表于 2017-4-20 10:15
感谢分享
相互学习。
谢谢分享！按照你的方法果然分发通过了，谢谢。
页:
[1]