专题 › Cloudera系列 › 已经一周勒，网上看帖子都没能解决，求大神指点

liubiao 发表于 2018-11-22 19:16:02

已经一周勒，网上看帖子都没能解决，求大神指点

3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent/supervis
or/include
3361 MainThread agent      ERROR    Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1636, in find_or_start_supervisor
    self.get_supervisor_process_info()
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1845, in get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/xmlrpc.py", line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
File "/usr/lib64/python2.6/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
File "/usr/lib64/python2.6/httplib.py", line 1010, in _send_request
    self.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 967, in endheaders
    self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 831, in _send_output
    self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 790, in send
    self.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread tmpfs      INFO   Reusing mounted tmpfs at /opt/cm-5.5.0/run/cloudera-scm-agent/process
3361 MainThread agent      INFO   Trying to connect to newly launched supervisor (Attempt 1)
3361 MainThread agent      INFO   Supervisor version: 3.0
3361 MainThread agent      INFO   Successfully connected to supervisor
3361 MainThread status_server INFO   Using maximum impala profile bundle size of 1073741824 bytes.
3361 MainThread status_server INFO   Using maximum stacks log bundle size of 1073741824 bytes.
3361 MainThread _cplogging   INFO    ENGINE Bus STARTING
3361 MainThread _cplogging   INFO    ENGINE Started monitor thread '_TimeoutMonitor'.
3361 MainThread _cplogging   INFO    ENGINE Serving on CentOS165:9000
3361 MainThread _cplogging   INFO    ENGINE Bus STARTED
3361 MainThread __init__   INFO   New monitor: (<cmf.monitor.host.HostMonitor object at 0x2cfb3d0>,)
3361 MainThread agent      INFO   Setting default socket timeout to 30
3361 MonitorDaemon-Scheduler __init__   INFO   Monitor ready to report: ('HostMonitor',)
3361 Monitor-HostMonitor network_interfaces INFO   NIC iface eth0 doesn't support ETHTOOL (95)
3361 Monitor-HostMonitor network_interfaces INFO   NIC iface pan0 doesn't support ETHTOOL (95)
3361 Monitor-HostMonitor throttling_logger ERROR    Error getting directory attributes for /opt/cm-5.5.0/log/clouder
a-scm-agent
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/dir_monitor.py", line 90, in _get_directory_attributes
    name = pwd.getpwuid(uid)
KeyError: 'getpwuid(): uid not found: 1106'
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      INFO   Using parcels directory from server provided value: /opt/cloudera/parcels
3361 MainThread agent      INFO   Created /opt/cloudera/parcels
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcels to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcels to 0755
3361 MainThread agent      INFO   Created /opt/cloudera/parcel-cache
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcel-cache to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcel-cache to 0755
3361 MainThread parcel       INFO   Agent does create users/groups and apply file permissions
3361 MainThread downloader   INFO   Downloader path: /opt/cloudera/parcel-cache
3361 MainThread parcel_cache INFO   Using /opt/cloudera/parcel-cache for parcel cache
3361 MainThread agent      INFO   Active parcel list updated; recalculating component info.
3361 DnsResolutionMonitor throttling_logger INFO   Using java location: '/usr/local/jdk1.7/bin/java'.
3361 Monitor-HostMonitor throttling_logger ERROR    Timeout with args ['ntpdc', '-np']
None
3361 Monitor-HostMonitor throttling_logger ERROR    Failed to collect NTP metrics
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 37, in collect
    result, stdout, stderr = self._subprocess_with_timeout(args, self._timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 30, in _subprocess_with_timeout
    return subprocess_with_timeout(args, timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/subprocess_timeout.py", line 94, in subprocess_with_timeout
    raise Exception("timeout with args %s" % args)
Exception: timeout with args ['ntpdc', '-np']
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
# more cloudera-scm-agent.log
3361 MainThread agent      INFO   SCM Agent Version: 5.5.0
3361 MainThread agent      INFO   Agent Protocol Version: 4
3361 MainThread agent      INFO   Using Host ID: 78441a1b-4d95-4f6c-8e91-f5b12fbc27cd
3361 MainThread agent      INFO   Using directory: /opt/cm-5.5.0/run/cloudera-scm-agent
3361 MainThread agent      INFO   Using supervisor binary path: /opt/cm-5.5.0/lib64/cmf/agent/src/cmf/../../buil
d/env/bin/supervisord
3361 MainThread agent      INFO   Neither verify_cert_file nor verify_cert_dir are configured. Not performing va
lidation of server certificates in HTTPS communication. These options can be configured in this agent's config.ini file to enable certificate val
idation.
3361 MainThread agent      INFO   Agent Logging Level: INFO
3361 MainThread agent      INFO   No command line vars
3361 MainThread agent      INFO   Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if you
're not using this database type)
3361 MainThread agent      INFO   Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if yo
u're not using this database type)
3361 MainThread agent      INFO   Found database jar: /opt/cm-5.5.0/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
3361 MainThread agent      INFO   Agent starting as pid 3361 user root(0) group root(0).
3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent
3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups
3361 MainThread cgroups      INFO   Found cgroups subsystem: cpu
3361 MainThread cgroups      INFO   Found cgroups subsystem: cpuacct
3361 MainThread cgroups      INFO   Found cgroups subsystem: memory
3361 MainThread cgroups      INFO   Found cgroups subsystem: blkio
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/memory
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/cpu
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/cpuacct
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/blkio
3361 MainThread agent      INFO   Found cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_byte
s': -1, 'default_memory_soft_limit_in_bytes': -1, 'writable_cgroup_dot_procs': True, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True, 'defau
lt_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct': True, 'has_blkio': True}
3361 MainThread agent      INFO   Setting up supervisord event monitor.
3361 MainThread filesystem_map INFO   Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
3361 MainThread filesystem_map INFO   Using timeout of 2.000000
3361 MainThread filesystem_map INFO   Using join timeout of 0.100000
3361 MainThread filesystem_map INFO   Using tolerance of 60.000000
3361 MainThread filesystem_map INFO   Local filesystem types whitelist: ['ext2', 'ext3', 'ext4']
3361 MainThread kt_renewer   INFO   Agent wide credential cache set to /opt/cm-5.5.0/run/cloudera-scm-agent/krb5cc
_cm_agent_0
3361 MainThread agent      INFO   Using metrics_url_timeout_seconds of 30.000000
3361 MainThread agent      INFO   Using task_metrics_timeout_seconds of 5.000000
3361 MainThread agent      INFO   Using max_collection_wait_seconds of 10.000000
3361 MainThread metrics      INFO   Importing tasktracker metric schema from file /opt/cm-5.5.0/lib64/cmf/agent/sr
c/cmf/monitor/tasktracker/schema.json
3361 MainThread ntp_monitorINFO   Using timeout of 2.000000
3361 MainThread dns_names    INFO   Using timeout of 30.000000
3361 MainThread __init__   INFO   Created DNS monitor.
3361 MainThread stacks_collection_manager INFO   Using max_uncompressed_file_size_bytes: 5242880
3361 MainThread __init__   INFO   Importing metric schema from file /opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monito
r/schema.json

3361 MainThread agent      ERROR    Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1636, in find_or_start_supervisor
    self.get_supervisor_process_info()
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1845, in get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/xmlrpc.py", line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
File "/usr/lib64/python2.6/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
File "/usr/lib64/python2.6/httplib.py", line 1010, in _send_request
    self.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 967, in endheaders
    self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 831, in _send_output
    self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 790, in send
    self.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread tmpfs      INFO   Reusing mounted tmpfs at /opt/cm-5.5.0/run/cloudera-scm-agent/process
3361 Monitor-HostMonitor throttling_logger ERROR    Error getting directory attributes for /opt/cm-5.5.0/log/clouder
a-scm-agent
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/dir_monitor.py", line 90, in _get_directory_attributes
    name = pwd.getpwuid(uid)
KeyError: 'getpwuid(): uid not found: 1106'
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      INFO   Using parcels directory from server provided value: /opt/cloudera/parcels
3361 MainThread agent      INFO   Created /opt/cloudera/parcels
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcels to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcels to 0755
3361 MainThread agent      INFO   Created /opt/cloudera/parcel-cache
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcel-cache to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcel-cache to 0755
3361 MainThread parcel       INFO   Agent does create users/groups and apply file permissions
3361 MainThread downloader   INFO   Downloader path: /opt/cloudera/parcel-cache
3361 MainThread parcel_cache INFO   Using /opt/cloudera/parcel-cache for parcel cache
3361 MainThread agent      INFO   Active parcel list updated; recalculating component info.
3361 DnsResolutionMonitor throttling_logger INFO   Using java location: '/usr/local/jdk1.7/bin/java'.
3361 Monitor-HostMonitor throttling_logger ERROR    Timeout with args ['ntpdc', '-np']
None
3361 Monitor-HostMonitor throttling_logger ERROR    Failed to collect NTP metrics
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 37, in collect
    result, stdout, stderr = self._subprocess_with_timeout(args, self._timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 30, in _subprocess_with_timeout
    return subprocess_with_timeout(args, timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/subprocess_timeout.py", line 94, in subprocess_with_timeout
    raise Exception("timeout with args %s" % args)
Exception: timeout with args ['ntpdc', '-np'] 3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent/supervis
or/include
3361 MainThread agent      ERROR    Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1636, in find_or_start_supervisor
    self.get_supervisor_process_info()
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1845, in get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/xmlrpc.py", line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
File "/usr/lib64/python2.6/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
File "/usr/lib64/python2.6/httplib.py", line 1010, in _send_request
    self.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 967, in endheaders
    self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 831, in _send_output
    self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 790, in send
    self.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread tmpfs      INFO   Reusing mounted tmpfs at /opt/cm-5.5.0/run/cloudera-scm-agent/process
3361 MainThread agent      INFO   Trying to connect to newly launched supervisor (Attempt 1)
3361 MainThread agent      INFO   Supervisor version: 3.0
3361 MainThread agent      INFO   Successfully connected to supervisor
3361 MainThread status_server INFO   Using maximum impala profile bundle size of 1073741824 bytes.
3361 MainThread status_server INFO   Using maximum stacks log bundle size of 1073741824 bytes.
3361 MainThread _cplogging   INFO    ENGINE Bus STARTING
3361 MainThread _cplogging   INFO    ENGINE Started monitor thread '_TimeoutMonitor'.
3361 MainThread _cplogging   INFO    ENGINE Serving on CentOS165:9000
3361 MainThread _cplogging   INFO    ENGINE Bus STARTED
3361 MainThread __init__   INFO   New monitor: (<cmf.monitor.host.HostMonitor object at 0x2cfb3d0>,)
3361 MainThread agent      INFO   Setting default socket timeout to 30
3361 MonitorDaemon-Scheduler __init__   INFO   Monitor ready to report: ('HostMonitor',)
3361 Monitor-HostMonitor network_interfaces INFO   NIC iface eth0 doesn't support ETHTOOL (95)
3361 Monitor-HostMonitor network_interfaces INFO   NIC iface pan0 doesn't support ETHTOOL (95)
3361 Monitor-HostMonitor throttling_logger ERROR    Error getting directory attributes for /opt/cm-5.5.0/log/clouder
a-scm-agent
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/dir_monitor.py", line 90, in _get_directory_attributes
    name = pwd.getpwuid(uid)
KeyError: 'getpwuid(): uid not found: 1106'
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      INFO   Using parcels directory from server provided value: /opt/cloudera/parcels
3361 MainThread agent      INFO   Created /opt/cloudera/parcels
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcels to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcels to 0755
3361 MainThread agent      INFO   Created /opt/cloudera/parcel-cache
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcel-cache to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcel-cache to 0755
3361 MainThread parcel       INFO   Agent does create users/groups and apply file permissions
3361 MainThread downloader   INFO   Downloader path: /opt/cloudera/parcel-cache
3361 MainThread parcel_cache INFO   Using /opt/cloudera/parcel-cache for parcel cache
3361 MainThread agent      INFO   Active parcel list updated; recalculating component info.
3361 DnsResolutionMonitor throttling_logger INFO   Using java location: '/usr/local/jdk1.7/bin/java'.
3361 Monitor-HostMonitor throttling_logger ERROR    Timeout with args ['ntpdc', '-np']
None
3361 Monitor-HostMonitor throttling_logger ERROR    Failed to collect NTP metrics
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 37, in collect
    result, stdout, stderr = self._subprocess_with_timeout(args, self._timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 30, in _subprocess_with_timeout
    return subprocess_with_timeout(args, timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/subprocess_timeout.py", line 94, in subprocess_with_timeout
    raise Exception("timeout with args %s" % args)
Exception: timeout with args ['ntpdc', '-np']
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
# more cloudera-scm-agent.log
3361 MainThread agent      INFO   SCM Agent Version: 5.5.0
3361 MainThread agent      INFO   Agent Protocol Version: 4
3361 MainThread agent      INFO   Using Host ID: 78441a1b-4d95-4f6c-8e91-f5b12fbc27cd
3361 MainThread agent      INFO   Using directory: /opt/cm-5.5.0/run/cloudera-scm-agent
3361 MainThread agent      INFO   Using supervisor binary path: /opt/cm-5.5.0/lib64/cmf/agent/src/cmf/../../buil
d/env/bin/supervisord
3361 MainThread agent      INFO   Neither verify_cert_file nor verify_cert_dir are configured. Not performing va
lidation of server certificates in HTTPS communication. These options can be configured in this agent's config.ini file to enable certificate val
idation.
3361 MainThread agent      INFO   Agent Logging Level: INFO
3361 MainThread agent      INFO   No command line vars
3361 MainThread agent      INFO   Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if you
're not using this database type)
3361 MainThread agent      INFO   Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if yo
u're not using this database type)
3361 MainThread agent      INFO   Found database jar: /opt/cm-5.5.0/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
3361 MainThread agent      INFO   Agent starting as pid 3361 user root(0) group root(0).
3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent
3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups
3361 MainThread cgroups      INFO   Found cgroups subsystem: cpu
3361 MainThread cgroups      INFO   Found cgroups subsystem: cpuacct
3361 MainThread cgroups      INFO   Found cgroups subsystem: memory
3361 MainThread cgroups      INFO   Found cgroups subsystem: blkio
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/memory
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/cpu
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/cpuacct
3361 MainThread cgroups      INFO   Reusing /opt/cm-5.5.0/run/cloudera-scm-agent/cgroups/blkio
3361 MainThread agent      INFO   Found cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_byte
s': -1, 'default_memory_soft_limit_in_bytes': -1, 'writable_cgroup_dot_procs': True, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True, 'defau
lt_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct': True, 'has_blkio': True}
3361 MainThread agent      INFO   Setting up supervisord event monitor.
3361 MainThread filesystem_map INFO   Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
3361 MainThread filesystem_map INFO   Using timeout of 2.000000
3361 MainThread filesystem_map INFO   Using join timeout of 0.100000
3361 MainThread filesystem_map INFO   Using tolerance of 60.000000
3361 MainThread filesystem_map INFO   Local filesystem types whitelist: ['ext2', 'ext3', 'ext4']
3361 MainThread kt_renewer   INFO   Agent wide credential cache set to /opt/cm-5.5.0/run/cloudera-scm-agent/krb5cc
_cm_agent_0
3361 MainThread agent      INFO   Using metrics_url_timeout_seconds of 30.000000
3361 MainThread agent      INFO   Using task_metrics_timeout_seconds of 5.000000
3361 MainThread agent      INFO   Using max_collection_wait_seconds of 10.000000
3361 MainThread metrics      INFO   Importing tasktracker metric schema from file /opt/cm-5.5.0/lib64/cmf/agent/sr
c/cmf/monitor/tasktracker/schema.json
3361 MainThread ntp_monitorINFO   Using timeout of 2.000000
3361 MainThread dns_names    INFO   Using timeout of 30.000000
3361 MainThread __init__   INFO   Created DNS monitor.
3361 MainThread stacks_collection_manager INFO   Using max_uncompressed_file_size_bytes: 5242880
3361 MainThread __init__   INFO   Importing metric schema from file /opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monito
r/schema.json
3361 MainThread agent      INFO   Supervised processes will add the following to their environment (in addition
to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/opt/cm-5.5.0/li
b64/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/opt/cm-5.5.0/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HO
ME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/
hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_S
TANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.
jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUSTEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce',
'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/opt/cm-5.5.0/share/cmf/lib/postgresql-9.0-801.jdbc4
.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/et
c/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/
usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_M
R1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry
', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'ROOT': '/opt/cm-5.5.0
/lib64/cmf', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME':
'/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
3361 MainThread agent      INFO   To override these variables, use /etc/cloudera-scm-agent/config.ini. Environme
nt variables for CDH locations are not used when CDH is installed from parcels.
3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent/process
3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent/supervis
or
3361 MainThread agent      INFO   Re-using pre-existing directory: /opt/cm-5.5.0/run/cloudera-scm-agent/supervis
or/include
3361 MainThread agent      ERROR    Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1636, in find_or_start_supervisor
    self.get_supervisor_process_info()
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1845, in get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/xmlrpc.py", line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
File "/usr/lib64/python2.6/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
File "/usr/lib64/python2.6/httplib.py", line 1010, in _send_request
    self.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 967, in endheaders
    self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 831, in _send_output
    self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 790, in send
    self.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread tmpfs      INFO   Reusing mounted tmpfs at /opt/cm-5.5.0/run/cloudera-scm-agent/process
3361 MainThread agent      INFO   Trying to connect to newly launched supervisor (Attempt 1)
3361 MainThread agent      INFO   Supervisor version: 3.0
3361 MainThread agent      INFO   Successfully connected to supervisor
3361 MainThread status_server INFO   Using maximum impala profile bundle size of 1073741824 bytes.
3361 MainThread status_server INFO   Using maximum stacks log bundle size of 1073741824 bytes.
3361 MainThread _cplogging   INFO    ENGINE Bus STARTING
3361 MainThread _cplogging   INFO    ENGINE Started monitor thread '_TimeoutMonitor'.
3361 MainThread _cplogging   INFO    ENGINE Serving on CentOS165:9000
3361 MainThread _cplogging   INFO    ENGINE Bus STARTED
3361 MainThread __init__   INFO   New monitor: (<cmf.monitor.host.HostMonitor object at 0x2cfb3d0>,)
3361 MainThread agent      INFO   Setting default socket timeout to 30
3361 MonitorDaemon-Scheduler __init__   INFO   Monitor ready to report: ('HostMonitor',)
3361 Monitor-HostMonitor network_interfaces INFO   NIC iface eth0 doesn't support ETHTOOL (95)
3361 Monitor-HostMonitor network_interfaces INFO   NIC iface pan0 doesn't support ETHTOOL (95)
3361 Monitor-HostMonitor throttling_logger ERROR    Error getting directory attributes for /opt/cm-5.5.0/log/clouder
a-scm-agent
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/dir_monitor.py", line 90, in _get_directory_attributes
    name = pwd.getpwuid(uid)
KeyError: 'getpwuid(): uid not found: 1106'
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      ERROR    Heartbeating to CentOS165:7182 failed.
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/agent.py", line 1062, in send_heartbeat
    self.master_port)
File "/opt/cm-5.5.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
File "/usr/lib64/python2.6/httplib.py", line 771, in connect
    self.timeout)
File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
error: Connection refused
3361 MainThread agent      INFO   Using parcels directory from server provided value: /opt/cloudera/parcels
3361 MainThread agent      INFO   Created /opt/cloudera/parcels
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcels to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcels to 0755
3361 MainThread agent      INFO   Created /opt/cloudera/parcel-cache
3361 MainThread agent      INFO   Chowning /opt/cloudera/parcel-cache to root (0) root (0)
3361 MainThread agent      INFO   Chmod'ing /opt/cloudera/parcel-cache to 0755
3361 MainThread parcel       INFO   Agent does create users/groups and apply file permissions
3361 MainThread downloader   INFO   Downloader path: /opt/cloudera/parcel-cache
3361 MainThread parcel_cache INFO   Using /opt/cloudera/parcel-cache for parcel cache
3361 MainThread agent      INFO   Active parcel list updated; recalculating component info.
3361 DnsResolutionMonitor throttling_logger INFO   Using java location: '/usr/local/jdk1.7/bin/java'.
3361 Monitor-HostMonitor throttling_logger ERROR    Timeout with args ['ntpdc', '-np']
None
3361 Monitor-HostMonitor throttling_logger ERROR    Failed to collect NTP metrics
Traceback (most recent call last):
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 37, in collect
    result, stdout, stderr = self._subprocess_with_timeout(args, self._timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/monitor/host/ntp_monitor.py", line 30, in _subprocess_with_timeout
    return subprocess_with_timeout(args, timeout)
File "/opt/cm-5.5.0/lib64/cmf/agent/src/cmf/subprocess_timeout.py", line 94, in subprocess_with_timeout
    raise Exception("timeout with args %s" % args)
Exception: timeout with args ['ntpdc', '-np']

s060403072 发表于 2018-11-22 20:09:23

只知道你在做cloudera遇到异常，描述的也不详细。上面其实也没什么有价值的信息。在去看看其他日志。或则描述清晰，你在干什么？你的环境是否干净？是否重装过等

liubiao 发表于 2018-11-24 13:54:56

都是重新赚的系统，环境干净，启动agent日志报错（ERROR    Failed to connect to previous supervisor.）（ERROR    Heartbeating to CentOS171:7182 failed.）重装系统多次，不知道哪里出了问题

liubiao 发表于 2018-11-24 13:56:07

都是重新赚的系统，环境干净，启动代理日志报错（ERROR无法连接到以前的主管。）（ERROR Heartbeating to CentOS171：7182失败。）重装系统多次，不知道哪里出了问题

阿飞 发表于 2018-11-24 16:45:22

liubiao 发表于 2018-11-24 13:56


那就是通信失败了。属于网络问题，比如ssh，防火墙，hosts，各个进程是否都在等等。

建议，看下About云Cloudera课程，这是一线资深老师出的cloudera安装部署教程，出现问题可以像老师请教。个人安装部署，确实挺耗费时间的，一个星期算是短的，个人摸索，可能一个月，甚至几个月。
Cloudera实战课程介绍
http://www.aboutyun.com/forum.php?mod=viewthread&tid=26351

查看完整版本: 已经一周勒，网上看帖子都没能解决，求大神指点