用hive查询1g数据量时就报错,这个错误代码都不知道去哪里找:Stack trace: ExitCodeException exitCode=255:
但是此时的hive能使用,select count(*) from pagecount;都可以查询。
然后减小数据量到400m试了一下,发现能用,请问这是怎么回事,要怎么处理啊?
谢谢啦
hive报错如下:
hive> SELECT percentile(countall,array(0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9)) FROM pagecount;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = spark2_20170809190519_17a24e4c-a303-4d87-ae3a-879f3cfcf8b4
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Starting Job = job_1501586770689_0094, Tracking URL = http://hop51:8088/proxy/application_1501586770689_0094/
Kill Command = /home/user/spark2/apps/hadoop-2.7.3/bin/hadoop job -kill job_1501586770689_0094
Hadoop job information for Stage-1: number of mappers: 5; number of reducers: 1
2017-08-09 19:05:29,517 Stage-1 map = 0%, reduce = 0%
2017-08-09 19:05:37,832 Stage-1 map = 40%, reduce = 0%, Cumulative CPU 21.58 sec
2017-08-09 19:05:38,874 Stage-1 map = 60%, reduce = 0%, Cumulative CPU 34.32 sec
2017-08-09 19:05:39,907 Stage-1 map = 93%, reduce = 0%, Cumulative CPU 71.9 sec
2017-08-09 19:05:40,939 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 72.77 sec
2017-08-09 19:06:41,952 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 72.77 sec
2017-08-09 19:07:42,807 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 72.77 sec
2017-08-09 19:08:05,522 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 72.77 sec
MapReduce Total cumulative CPU time: 1 minutes 12 seconds 770 msec
Ended Job = job_1501586770689_0094 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1501586770689_0094_m_000003 (and more) from job job_1501586770689_0094
Task with the most failures(4):
-----
Task ID:
task_1501586770689_0094_r_000000
URL:
http://hop51:8088/taskdetails.jsp?jobid=job_1501586770689_0094&tipid=task_1501586770689_0094_r_000000
-----
Diagnostic Messages for this Task:
Exception from container-launch.
Container id: container_1501586770689_0094_01_000010
Exit code: 255
Stack trace: ExitCodeException exitCode=255:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
at org.apache.hadoop.util.Shell.run(Shell.java:479)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 255
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 5 Reduce: 1 Cumulative CPU: 72.77 sec HDFS Read: 875790735 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 1 minutes 12 seconds 770 msec
|
|