分享

hive启动报警告太多了,请教如何消除

zeroone 发表于 2016-5-1 08:23:36 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 4 19970
      hive启动就报一大堆警告 , 可是去hive-site.xml,mapred-site.xml里面有没有对应的项改,求大神指教。启动过程如下
[root@slave1 ~]# hive
16/05/01 07:57:33 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
16/05/01 07:57:33 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
16/05/01 07:57:33 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
16/05/01 07:57:33 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
16/05/01 07:57:33 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
16/05/01 07:57:33 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
16/05/01 07:57:33 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
16/05/01 07:57:34 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.1.0-1.cdh5.1.0.p0.53/lib/hive/lib/hive-common-0.12.0-cdh5.1.0.jar!/hive-log4j.properties
hive>
虽然不影响运行 , 但每次进入hive跳这么多信息看着头疼。


已有(4)人评论

跳转到指定楼层
Alkaloid0515 发表于 2016-5-1 08:56:47
 Hadoop自升级到2.x版本之后,有很多属性的名称已经被遗弃了,虽然这些被遗弃的属性名称目前还可以用.所以去除警告,使用新的属性名。

这些都是hadoop的,所以在hadoop配置文件中,并非hive配置文件中
回复

使用道具 举报

Alkaloid0515 发表于 2016-5-1 08:57:30

Hadoop升级到2.x版本之后,有很多属性的名称已经被遗弃了,虽然这些被遗弃的属性名称目前还可以用,主要遗弃的属性名称主要见下面表格:
已经被遗弃属性的名称新的属性名称
create.empty.dir.if.nonexistmapreduce.jobcontrol.createdir.ifnotexist
dfs.access.time.precisiondfs.namenode.accesstime.precision
dfs.backup.addressdfs.namenode.backup.address
dfs.backup.http.addressdfs.namenode.backup.http-address
dfs.balance.bandwidthPerSecdfs.datanode.balance.bandwidthPerSec
dfs.block.sizedfs.blocksize
dfs.data.dirdfs.datanode.data.dir
dfs.datanode.max.xcieversdfs.datanode.max.transfer.threads
dfs.df.intervalfs.df.interval
dfs.federation.nameservice.iddfs.nameservice.id
dfs.federation.nameservicesdfs.nameservices
dfs.http.addressdfs.namenode.http-address
dfs.https.addressdfs.namenode.https-address
dfs.https.client.keystore.resourcedfs.client.https.keystore.resource
dfs.https.need.client.authdfs.client.https.need-auth
dfs.max.objectsdfs.namenode.max.objects
dfs.max-repl-streamsdfs.namenode.replication.max-streams
dfs.name.dirdfs.namenode.name.dir
dfs.name.dir.restoredfs.namenode.name.dir.restore
dfs.name.edits.dirdfs.namenode.edits.dir
dfs.permissionsdfs.permissions.enabled
dfs.permissions.supergroupdfs.permissions.superusergroup
dfs.read.prefetch.sizedfs.client.read.prefetch.size
dfs.replication.considerLoaddfs.namenode.replication.considerLoad
dfs.replication.intervaldfs.namenode.replication.interval
dfs.replication.mindfs.namenode.replication.min
dfs.replication.pending.timeout.secdfs.namenode.replication.pending.timeout-sec
dfs.safemode.extensiondfs.namenode.safemode.extension
dfs.safemode.threshold.pctdfs.namenode.safemode.threshold-pct
dfs.secondary.http.addressdfs.namenode.secondary.http-address
dfs.socket.timeoutdfs.client.socket-timeout
dfs.umaskmodefs.permissions.umask-mode
dfs.write.packet.sizedfs.client-write-packet-size
fs.checkpoint.dirdfs.namenode.checkpoint.dir
fs.checkpoint.edits.dirdfs.namenode.checkpoint.edits.dir
fs.checkpoint.perioddfs.namenode.checkpoint.period
fs.default.namefs.defaultFS
hadoop.configured.node.mappingnet.topology.configured.node.mapping
hadoop.job.history.locationmapreduce.jobtracker.jobhistory.location
hadoop.native.libio.native.lib.available
hadoop.net.static.resolutionsmapreduce.tasktracker.net.static.resolutions
hadoop.pipes.command-file.keepmapreduce.pipes.commandfile.preserve
hadoop.pipes.executable.interpretormapreduce.pipes.executable.interpretor
hadoop.pipes.executablemapreduce.pipes.executable
hadoop.pipes.java.mappermapreduce.pipes.isjavamapper
hadoop.pipes.java.recordreadermapreduce.pipes.isjavarecordreader
hadoop.pipes.java.recordwritermapreduce.pipes.isjavarecordwriter
hadoop.pipes.java.reducermapreduce.pipes.isjavareducer
hadoop.pipes.partitionermapreduce.pipes.partitioner
heartbeat.recheck.intervaldfs.namenode.heartbeat.recheck-interval
io.bytes.per.checksumdfs.bytes-per-checksum
io.sort.factormapreduce.task.io.sort.factor
io.sort.mbmapreduce.task.io.sort.mb
io.sort.spill.percentmapreduce.map.sort.spill.percent
jobclient.completion.poll.intervalmapreduce.client.completion.pollinterval
jobclient.output.filtermapreduce.client.output.filter
jobclient.progress.monitor.poll.intervalmapreduce.client.progressmonitor.pollinterval
job.end.notification.urlmapreduce.job.end-notification.url
job.end.retry.attemptsmapreduce.job.end-notification.retry.attempts
job.end.retry.intervalmapreduce.job.end-notification.retry.interval
job.local.dirmapreduce.job.local.dir
keep.failed.task.filesmapreduce.task.files.preserve.failedtasks
keep.task.files.patternmapreduce.task.files.preserve.filepattern
key.value.separator.in.input.linemapreduce.input.keyvaluelinerecordreader.key.value.separator
local.cache.sizemapreduce.tasktracker.cache.local.size
map.input.filemapreduce.map.input.file
map.input.lengthmapreduce.map.input.length
map.input.startmapreduce.map.input.start
map.output.key.field.separatormapreduce.map.output.key.field.separator
map.output.key.value.fields.specmapreduce.fieldsel.map.output.key.value.fields.spec
mapred.acls.enabledmapreduce.cluster.acls.enabled
mapred.binary.partitioner.left.offsetmapreduce.partition.binarypartitioner.left.offset
mapred.binary.partitioner.right.offsetmapreduce.partition.binarypartitioner.right.offset
mapred.cache.archivesmapreduce.job.cache.archives
mapred.cache.archives.timestampsmapreduce.job.cache.archives.timestamps
mapred.cache.filesmapreduce.job.cache.files
mapred.cache.files.timestampsmapreduce.job.cache.files.timestamps
mapred.cache.localArchivesmapreduce.job.cache.local.archives
mapred.cache.localFilesmapreduce.job.cache.local.files
mapred.child.tmpmapreduce.task.tmp.dir
mapred.cluster.average.blacklist.thresholdmapreduce.jobtracker.blacklist.average.threshold
mapred.cluster.map.memory.mbmapreduce.cluster.mapmemory.mb
mapred.cluster.max.map.memory.mbmapreduce.jobtracker.maxmapmemory.mb
mapred.cluster.max.reduce.memory.mbmapreduce.jobtracker.maxreducememory.mb
mapred.cluster.reduce.memory.mbmapreduce.cluster.reducememory.mb
mapred.committer.job.setup.cleanup.neededmapreduce.job.committer.setup.cleanup.needed
mapred.compress.map.outputmapreduce.map.output.compress
mapred.data.field.separatormapreduce.fieldsel.data.field.separator
mapred.debug.out.linesmapreduce.task.debugout.lines
mapred.healthChecker.intervalmapreduce.tasktracker.healthchecker.interval
mapred.healthChecker.script.argsmapreduce.tasktracker.healthchecker.script.args
mapred.healthChecker.script.pathmapreduce.tasktracker.healthchecker.script.path
mapred.healthChecker.script.timeoutmapreduce.tasktracker.healthchecker.script.timeout
mapred.heartbeats.in.secondmapreduce.jobtracker.heartbeats.in.second
mapred.hosts.excludemapreduce.jobtracker.hosts.exclude.filename
mapred.hostsmapreduce.jobtracker.hosts.filename
mapred.inmem.merge.thresholdmapreduce.reduce.merge.inmem.threshold
mapred.input.dir.formatsmapreduce.input.multipleinputs.dir.formats
mapred.input.dir.mappersmapreduce.input.multipleinputs.dir.mappers
mapred.input.dirmapreduce.input.fileinputformat.inputdir
mapred.input.pathFilter.classmapreduce.input.pathFilter.class
mapred.jarmapreduce.job.jar
mapred.job.classpath.archivesmapreduce.job.classpath.archives
mapred.job.classpath.filesmapreduce.job.classpath.files
mapred.job.idmapreduce.job.id
mapred.jobinit.threadsmapreduce.jobtracker.jobinit.threads
mapred.job.map.memory.mbmapreduce.map.memory.mb
mapred.job.namemapreduce.job.name
mapred.job.prioritymapreduce.job.priority
mapred.job.queue.namemapreduce.job.queuename
mapred.job.reduce.input.buffer.percentmapreduce.reduce.input.buffer.percent
mapred.job.reduce.markreset.buffer.percentmapreduce.reduce.markreset.buffer.percent
mapred.job.reduce.memory.mbmapreduce.reduce.memory.mb
mapred.job.reduce.total.mem.bytesmapreduce.reduce.memory.totalbytes
mapred.job.reuse.jvm.num.tasksmapreduce.job.jvm.numtasks
mapred.job.shuffle.input.buffer.percentmapreduce.reduce.shuffle.input.buffer.percent
mapred.job.shuffle.merge.percentmapreduce.reduce.shuffle.merge.percent
mapred.job.tracker.handler.countmapreduce.jobtracker.handler.count
mapred.job.tracker.history.completed.locationmapreduce.jobtracker.jobhistory.completed.location
mapred.job.tracker.http.addressmapreduce.jobtracker.http.address
mapred.jobtracker.instrumentationmapreduce.jobtracker.instrumentation
mapred.jobtracker.job.history.block.sizemapreduce.jobtracker.jobhistory.block.size
mapred.job.tracker.jobhistory.lru.cache.sizemapreduce.jobtracker.jobhistory.lru.cache.size
mapred.job.trackermapreduce.jobtracker.address
mapred.jobtracker.maxtasks.per.jobmapreduce.jobtracker.maxtasks.perjob
mapred.job.tracker.persist.jobstatus.activemapreduce.jobtracker.persist.jobstatus.active
mapred.job.tracker.persist.jobstatus.dirmapreduce.jobtracker.persist.jobstatus.dir
mapred.job.tracker.persist.jobstatus.hoursmapreduce.jobtracker.persist.jobstatus.hours
mapred.jobtracker.restart.recovermapreduce.jobtracker.restart.recover
mapred.job.tracker.retiredjobs.cache.sizemapreduce.jobtracker.retiredjobs.cache.size
mapred.job.tracker.retire.jobsmapreduce.jobtracker.retirejobs
mapred.jobtracker.taskalloc.capacitypadmapreduce.jobtracker.taskscheduler.taskalloc.capacitypad
mapred.jobtracker.taskSchedulermapreduce.jobtracker.taskscheduler
mapred.jobtracker.
taskScheduler.maxRunningTasksPerJob
mapreduce.jobtracker.taskscheduler.maxrunningtasks.perjob
mapred.join.exprmapreduce.join.expr
mapred.join.keycomparatormapreduce.join.keycomparator
mapred.lazy.output.formatmapreduce.output.lazyoutputformat.outputformat
mapred.line.input.format.linespermapmapreduce.input.lineinputformat.linespermap
mapred.linerecordreader.maxlengthmapreduce.input.linerecordreader.line.maxlength
mapred.local.dirmapreduce.cluster.local.dir
mapred.local.dir.minspacekillmapreduce.tasktracker.local.dir.minspacekill
mapred.local.dir.minspacestartmapreduce.tasktracker.local.dir.minspacestart
mapred.map.child.envmapreduce.map.env
mapred.map.child.java.optsmapreduce.map.java.opts
mapred.map.child.log.levelmapreduce.map.log.level
mapred.map.max.attemptsmapreduce.map.maxattempts
mapred.map.output.compression.codecmapreduce.map.output.compress.codec
mapred.mapoutput.key.classmapreduce.map.output.key.class
mapred.mapoutput.value.classmapreduce.map.output.value.class
mapred.mapper.regex.groupmapreduce.mapper.regexmapper..group
mapred.mapper.regexmapreduce.mapper.regex
mapred.map.task.debug.scriptmapreduce.map.debug.script
mapred.map.tasksmapreduce.job.maps
mapred.map.tasks.speculative.executionmapreduce.map.speculative
mapred.max.map.failures.percentmapreduce.map.failures.maxpercent
mapred.max.reduce.failures.percentmapreduce.reduce.failures.maxpercent
mapred.max.split.sizemapreduce.input.fileinputformat.split.maxsize
mapred.max.tracker.blacklistsmapreduce.jobtracker.tasktracker.maxblacklists
mapred.max.tracker.failuresmapreduce.job.maxtaskfailures.per.tracker
mapred.merge.recordsBeforeProgressmapreduce.task.merge.progress.records
mapred.min.split.sizemapreduce.input.fileinputformat.split.minsize
mapred.min.split.size.per.nodemapreduce.input.fileinputformat.split.minsize.per.node
mapred.min.split.size.per.rackmapreduce.input.fileinputformat.split.minsize.per.rack
mapred.output.compression.codecmapreduce.output.fileoutputformat.compress.codec
mapred.output.compression.typemapreduce.output.fileoutputformat.compress.type
mapred.output.compressmapreduce.output.fileoutputformat.compress
mapred.output.dirmapreduce.output.fileoutputformat.outputdir
mapred.output.key.classmapreduce.job.output.key.class
mapred.output.key.comparator.classmapreduce.job.output.key.comparator.class
mapred.output.value.classmapreduce.job.output.value.class
mapred.output.value.groupfn.classmapreduce.job.output.group.comparator.class
mapred.permissions.supergroupmapreduce.cluster.permissions.supergroup
mapred.pipes.user.inputformatmapreduce.pipes.inputformat
mapred.reduce.child.envmapreduce.reduce.env
mapred.reduce.child.java.optsmapreduce.reduce.java.opts
mapred.reduce.child.log.levelmapreduce.reduce.log.level
mapred.reduce.max.attemptsmapreduce.reduce.maxattempts
mapred.reduce.parallel.copiesmapreduce.reduce.shuffle.parallelcopies
mapred.reduce.slowstart.completed.mapsmapreduce.job.reduce.slowstart.completedmaps
mapred.reduce.task.debug.scriptmapreduce.reduce.debug.script
mapred.reduce.tasksmapreduce.job.reduces
mapred.reduce.tasks.speculative.executionmapreduce.reduce.speculative
mapred.seqbinary.output.key.classmapreduce.output.seqbinaryoutputformat.key.class
mapred.seqbinary.output.value.classmapreduce.output.seqbinaryoutputformat.value.class
mapred.shuffle.connect.timeoutmapreduce.reduce.shuffle.connect.timeout
mapred.shuffle.read.timeoutmapreduce.reduce.shuffle.read.timeout
mapred.skip.attempts.to.start.skippingmapreduce.task.skip.start.attempts
mapred.skip.map.auto.incr.proc.countmapreduce.map.skip.proc-count.auto-incr
mapred.skip.map.max.skip.recordsmapreduce.map.skip.maxrecords
mapred.skip.onmapreduce.job.skiprecords
mapred.skip.out.dirmapreduce.job.skip.outdir
mapred.skip.reduce.auto.incr.proc.countmapreduce.reduce.skip.proc-count.auto-incr
mapred.skip.reduce.max.skip.groupsmapreduce.reduce.skip.maxgroups
mapred.speculative.execution.slowNodeThresholdmapreduce.job.speculative.slownodethreshold
mapred.speculative.execution.slowTaskThresholdmapreduce.job.speculative.slowtaskthreshold
mapred.speculative.execution.speculativeCapmapreduce.job.speculative.speculativecap
mapred.submit.replicationmapreduce.client.submit.file.replication
mapred.system.dirmapreduce.jobtracker.system.dir
mapred.task.cache.levelsmapreduce.jobtracker.taskcache.levels
mapred.task.idmapreduce.task.attempt.id
mapred.task.is.mapmapreduce.task.ismap
mapred.task.partitionmapreduce.task.partition
mapred.task.profilemapreduce.task.profile
mapred.task.profile.mapsmapreduce.task.profile.maps
mapred.task.profile.paramsmapreduce.task.profile.params
mapred.task.profile.reducesmapreduce.task.profile.reduces
mapred.task.timeoutmapreduce.task.timeout
mapred.tasktracker.dns.interfacemapreduce.tasktracker.dns.interface
mapred.tasktracker.dns.nameservermapreduce.tasktracker.dns.nameserver
mapred.tasktracker.events.batchsizemapreduce.tasktracker.events.batchsize
mapred.tasktracker.expiry.intervalmapreduce.jobtracker.expire.trackers.interval
mapred.task.tracker.http.addressmapreduce.tasktracker.http.address
mapred.tasktracker.indexcache.mbmapreduce.tasktracker.indexcache.mb
mapred.tasktracker.instrumentationmapreduce.tasktracker.instrumentation
mapred.tasktracker.map.tasks.maximummapreduce.tasktracker.map.tasks.maximum
mapred.tasktracker.memory_calculator_pluginmapreduce.tasktracker.resourcecalculatorplugin
mapred.tasktracker.memorycalculatorpluginmapreduce.tasktracker.resourcecalculatorplugin
mapred.tasktracker.reduce.tasks.maximummapreduce.tasktracker.reduce.tasks.maximum
mapred.task.tracker.report.addressmapreduce.tasktracker.report.address
mapred.task.tracker.task-controllermapreduce.tasktracker.taskcontroller
mapred.tasktracker.taskmemorymanager.monitoring-intervalmapreduce.tasktracker.taskmemorymanager.monitoringinterval
mapred.tasktracker.tasks.sleeptime-before-sigkillmapreduce.tasktracker.tasks.sleeptimebeforesigkill
mapred.temp.dirmapreduce.cluster.temp.dir
mapred.text.key.comparator.optionsmapreduce.partition.keycomparator.options
mapred.text.key.partitioner.optionsmapreduce.partition.keypartitioner.options
mapred.textoutputformat.separatormapreduce.output.textoutputformat.separator
mapred.tip.idmapreduce.task.id
mapreduce.combine.classmapreduce.job.combine.class
mapreduce.inputformat.classmapreduce.job.inputformat.class
mapreduce.job.counters.limitmapreduce.job.counters.max
mapreduce.jobtracker.permissions.supergroupmapreduce.cluster.permissions.supergroup
mapreduce.map.classmapreduce.job.map.class
mapreduce.outputformat.classmapreduce.job.outputformat.class
mapreduce.partitioner.classmapreduce.job.partitioner.class
mapreduce.reduce.classmapreduce.job.reduce.class
mapred.used.genericoptionsparsermapreduce.client.genericoptionsparser.used
mapred.userlog.limit.kbmapreduce.task.userlog.limit.kb
mapred.userlog.retain.hoursmapreduce.job.userlog.retain.hours
mapred.working.dirmapreduce.job.working.dir
mapred.work.output.dirmapreduce.task.output.dir
min.num.spills.for.combinemapreduce.map.combine.minspills
reduce.output.key.value.fields.specmapreduce.fieldsel.reduce.output.key.value.fields.spec
security.job.submission.protocol.aclsecurity.job.client.protocol.acl
security.task.umbilical.protocol.aclsecurity.job.task.protocol.acl
sequencefile.filter.classmapreduce.input.sequencefileinputfilter.class
sequencefile.filter.frequencymapreduce.input.sequencefileinputfilter.frequency
sequencefile.filter.regexmapreduce.input.sequencefileinputfilter.regex
session.iddfs.metrics.session-id
slave.host.namedfs.datanode.hostname
slave.host.namemapreduce.tasktracker.host.name
tasktracker.contention.trackingmapreduce.tasktracker.contention.tracking
tasktracker.http.threadsmapreduce.tasktracker.http.threads
topology.node.switch.mapping.implnet.topology.node.switch.mapping.impl
topology.script.file.namenet.topology.script.file.name
topology.script.number.argsnet.topology.script.number.args
user.namemapreduce.job.user.name
webinterface.private.actionsmapreduce.jobtracker.webinterface.trusted
  下面表格列出一些配置属性的相关改变:
Deprecated property nameNew property name
mapred.create.symlinkNONE - symlinking一直都是开的状态
mapreduce.job.cache.symlink.createNONE - symlinking一直都是开的状态





回复

使用道具 举报

zeroone 发表于 2016-5-1 11:34:35
具体是那个文件呢
回复

使用道具 举报

bioger_hit 发表于 2016-5-1 17:49:05
zeroone 发表于 2016-5-1 11:34
具体是那个文件呢

这里官网也是没给的。
这些自己猜即可。
core-default.xml
hdfs-default.xml
mapred-default.xml
yarn-default.xml


就这些文件,关于dfs相信是在hdfs-default.xml
关于mapreduce,相信是在mapred-default.xml


这些遗弃的属性和替换的属性,官网也没一一给出说明
这是2.7.2的属性
http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条