一曲离殇 发表于 2019-3-5 17:51:01

hbase put时内存溢出了,如何调整





2019-03-05 17:39:35,569 WARN - #39, table=TABLE_TEST, attempt=3/3 failed=203ops, last exception: java.io.IOException: java.lang.OutOfMemoryError: Direct buffer memory on datanode09,60020,1551777514745, tracking started Tue Mar 05 17:39:34 CST 2019; not retrying 203 - final failure
2019-03-05 17:39:35,570 ERROR - file load failed!
org.hbase.load.exception.ClientRuntimeException: pool-19-thread-1 TABLE_TEST commit failed !
        at org.hbase.load.TestLoad.commit(TestLoad.java:277)
        at org.hbase.load.TestLoad.hbasePut(TestLoad.java:172)
        at org.hbase.load.TestLoad.run(TestLoad.java:88)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 406 actions: IOException: 406 times,
        at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:258)
        at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$2000(AsyncProcess.java:238)
        at org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1817)
        at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:246)
        at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:152)
        at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1097)
        at com.asiainfo.cloudload.put.LoadActors.commit(LoadActors.java:266)
        ... 7 more

bioger_hit 发表于 2019-3-6 07:53:49

到datanode09上,看看详细日志。里面肯定有更细的说明。上面只是看出内存溢出,但是却看不出什么原因。需要更详细的日志。
页: [1]
查看完整版本: hbase put时内存溢出了,如何调整