GC overhead limit exceeded问题

 1.问题现象

程序包运行时候发生了java.lang.OutOfMemoryError: GC overhead limit exceeded异常， 详细信息如下

org.apache.ibatis.exceptions.PersistenceException:
### Error querying database. Cause: org.jboss.util.NestedSQLException: Error; - nested throwable: (java.lang.OutOfMemoryError: GC overhead limit exceeded)
### The error may exist in org/activiti/db/mapping/entity/Job.xml
### The error may involve org.activiti.engine.impl.persistence.entity.JobEntity.selectNextJobsToExecute-Inline
### The error occurred while setting parameters
### SQL: select * from ( select a.*, ROWNUM rnum from ( select JOB.* from ACT_RU_JOB JOB LEFT OUTER JOIN ACT_RU_EXECUTION PI ON PI.ID_ = JOB.PROCESS_INSTANCE_ID_ LEFT OUTER JOIN ACT_RE_PROCDEF PD ON PD.ID_ = PI.PROC_DEF_ID_ where (JOB.RETRIES_ > 0) and (JOB.DUEDATE_ is null or JOB.DUEDATE_ < ?) and (JOB.LOCK_OWNER_ is null or JOB.LOCK_EXP_TIME_ < ?) and ( (JOB.EXECUTION_ID_ is null) or ((PI.SUSPENSION_STATE_ = 1) and (PD.SUSPENSION_STATE_ = 1)) ) ) a where ROWNUM < ?) where rnum >= ?
### Cause: org.jboss.util.NestedSQLException: Error; - nested throwable: (java.lang.OutOfMemoryError: GC overhead limit exceeded)
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:23)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:104)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:95)
at org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:216)
at org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:200)
at org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:187)
at org.activiti.engine.impl.persistence.entity.JobManager.findNextJobsToExecute(JobManager.java:105)
at org.activiti.engine.impl.cmd.AcquireJobsCmd.execute(AcquireJobsCmd.java:51)
at org.activiti.engine.impl.cmd.AcquireJobsCmd.execute(AcquireJobsCmd.java:33)
at org.activiti.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:24)
at org.activiti.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:42)
at org.activiti.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:42)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
at org.activiti.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:40)
at org.activiti.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
at org.activiti.engine.impl.jobexecutor.AcquireJobsRunnable.run(AcquireJobsRunnable.java:57)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.jboss.util.NestedSQLException: Error; - nested throwable: (java.lang.OutOfMemoryError: GC overhead limit exceeded)
at org.jboss.resource.adapter.jdbc.WrappedConnection.checkException(WrappedConnection.java:873)
at org.jboss.resource.adapter.jdbc.WrappedStatement.checkException(WrappedStatement.java:852)

2.问题原因

从网上搜集了一些相关信息

Exception in thread thread_name: java.lang.OutOfMemoryError: GC Overhead limit exceeded Cause: The detail message "GC overhead limit exceeded" indicates that the garbage collector is running all the time and Java program is making very slow progress. After a garbage collection, if the Java process is spending more than approximately 98% of its time doing garbage collection and if it is recovering less than 2% of the heap and has been doing so far the last 5 (compile time constant) consecutive garbage collections, then a java.lang.OutOfMemoryError is thrown. This exception is typically thrown because the amount of live data barely fits into the Java heap having little free space for new allocations.

Action: Increase the heap size. The java.lang.OutOfMemoryError exception for GC Overhead limit exceeded can be turned off with the command line flag -XX:-UseGCOverheadLimit.

Java运行环境包含了一个内置的Garbage Collection (GC)垃圾回收进程，Jvm会根据程序的运行情况，执行GC垃圾回收操作，用于对不再使用的内存区域进行回收，释放被占用的内存。 当发生此问题时，JVM发现内存不够了,通过GC回收垃圾对象来获得内存,新的对象来临时,JVM发现内存空间不足,于是进行GC, GC在经过一番运算之后,可能发现回收的内存很小,不能满足新对象所需的内存,新对象所需的内存空间.而且进行过几次GC之后发现空间仍然存放不下或者不足以放下新的对象，那么JVM就会抛出该错误， 默认情况下,如果GC花费的时间超过98%,并且GC回收的内存少于2%,就会发生该错误.

3. 解决方法

1、增加heap堆内存。

2、增加对内存后错误依旧，获取heap内存快照，使用Eclipse MAT工具，找出内存泄露发生的原因并进行修复。

3、优化代码以使用更少的内存或重用对象，而不是创建新的对象，从而减少垃圾收集器运行的次数。如果代码中创建了许多临时对象(例如在循环中)，应该尝试重用它们。

4、升级JDK到1.8，最起码也是1.7，并使用G1GC垃圾回收算法。

5、除了使用命令-xms1g -xmx2g设置堆内存之外，尝试在启动脚本中加入配置:

-XX:+UseG1GC -XX:G1HeapRegionSize=n -XX:MaxGCPauseMillis=m -XX:ParallelGCThreads=n -XX:ConcGCThreads=n

还有一个非常不建议使用的解决方法：

在启动脚本中添加-XX:-UseGCOverheadLimit命令。这个方法只会把“java.lang.OutOfMemoryError: GC overhead limit exceeded”变成更常见的java.lang.OutOfMemoryError: Java heap space错误。