那些遇到过的问题

从java 6升级到java 7时本机堆上的OutOfMemoryError

mda*_*win 7 java garbage-collection jvm jaxb jboss5.x

我们最近将我们的大型webapp(在jboss 5上运行)从java 6升级到java 7.

几个小时之后,我们看到了一个OutOfMemory错误,它看起来就像是本机堆耗尽了.

我们运行32位JVM,因此限制为4GB,JVM分配为2GB.

在java 6下,整个进程占用了大约2.3GB,但是对于java 7,这大大增加了,我们达到了4GB的限制而没有触发完整的GC,因为Java堆还没有满.

堆栈跟踪显示XML解组代码在每个请求上创建新的SAXParserFactory,而用于解压缩jar文件的Inflater类在本机堆中存储大量数据(~200,000 Inflater实例).这对我来说效率相当低 - 无论是新的SAXParserFactory还是多个Inflaters.也许这是一个红鲱鱼,但我没有这个比较行为的java 6版本.

通过每隔几个小时手动运行gc,应用程序可以快乐地连续几天没有任何问题.但那不是解决方案.

问题是为什么内存分配增加了1.5GB,我该如何阻止这种情况发生呢？

我希望更新的java版本更高效,更快,等等.但相反,我们看起来像JVM中的内存泄漏.

无论如何,堆栈跟踪如下:

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 512128 bytes for Chunk::new
# An error report file with more information is saved as:
# /opt/SP/apps/er_core/current/hs_err_pid25455.log
00:49:30,072 ERROR [[DecouplingProcessServlet]] Servlet.service() for servlet DecouplingProcessServlet threw exception
java.lang.OutOfMemoryError
    at java.util.zip.Inflater.init(Native Method)
    at java.util.zip.Inflater.<init>(Inflater.java:101)
    at java.util.zip.ZipFile.getInflater(ZipFile.java:450)
    at java.util.zip.ZipFile.getInputStream(ZipFile.java:369)
    at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:214)
    at org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipEntryContext.java:1066)
    at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipEntryHandler.java:153)
    at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:230)
    at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:93)
    at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:233)
    at javax.xml.parsers.SecuritySupport$4.run(SecuritySupport.java:94)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.xml.parsers.SecuritySupport.getResourceAsStream(SecuritySupport.java:87)
    at javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:283)
    at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:255)
    at javax.xml.parsers.SAXParserFactory.newInstance(SAXParserFactory.java:126)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.getXMLReader(AbstractUnmarshallerImpl.java:100)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204)
    at com.mycompany.global.er.decoupling.util.xml.JAXBHelper.bind(JAXBHelper.java:88)
    at com.mycompany.global.er.decoupling.util.xml.JAXBHelper.bind(JAXBHelper.java:56)
    at com.mycompany.global.er.decoupling.DecouplingProcessServlet.doPost(DecouplingProcessServlet.java:163)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
    at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:269)
    at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:81)
    at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
    at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
    at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at pk.vodafone.valves.PKAccessLogValve.invoke(PKAccessLogValve.java:547)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
    at java.lang.Thread.run(Thread.java:724)
00:49:30,117 ERROR [CoyoteAdapter] An exception or error occurred in the container during the request processing
java.lang.NoClassDefFoundError: javax/servlet/ServletRequestAttributeEvent
    at org.apache.catalina.connector.Request.setAttribute(Request.java:1452)
    at org.apache.catalina.core.StandardWrapperValve.exception(StandardWrapperValve.java:523)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:280)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
    at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:269)
    at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:81)
    at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
    at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
    at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at pk.vodafone.valves.PKAccessLogValve.invoke(PKAccessLogValve.java:547)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
    at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.ClassNotFoundException: Unexpected error during load of: javax.servlet.ServletRequestAttributeEvent, msg=null
    at org.jboss.classloader.spi.base.ClassLoaderManager.process(ClassLoaderManager.java:173)
    at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:259)
    at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1102)
    at org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:772)
    at org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:415)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 19 more
Caused by: java.lang.OutOfMemoryError
    at java.util.zip.Inflater.init(Native Method)
    at java.util.zip.Inflater.<init>(Inflater.java:101)
    at java.util.zip.ZipFile.getInflater(ZipFile.java:450)
    at java.util.zip.ZipFile.getInputStream(ZipFile.java:369)
    at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:214)
    at org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipEntryContext.java:1066)
    at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipEntryHandler.java:153)
    at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:230)
    at org.jboss.classloading.spi.vfs.policy.VFSClassLoaderPolicy.getResourceAsStream(VFSClassLoaderPolicy.java:483)
    at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:508)
    at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:506)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.jboss.classloader.spi.base.BaseClassLoader.loadClassLocally(BaseClassLoader.java:504)
    at org.jboss.classloader.spi.base.BaseClassLoader.loadClassLocally(BaseClassLoader.java:481)
    at org.jboss.classloader.spi.base.BaseDelegateLoader.loadClass(BaseDelegateLoader.java:134)
    at org.jboss.classloader.spi.filter.FilteredDelegateLoader.loadClass(FilteredDelegateLoader.java:131)
    at org.jboss.classloader.spi.base.ClassLoadingTask$ThreadTask.run(ClassLoadingTask.java:452)
    at org.jboss.classloader.spi.base.ClassLoaderManager.nextTask(ClassLoaderManager.java:258)
    at org.jboss.classloader.spi.base.ClassLoaderManager.process(ClassLoaderManager.java:152)
    ... 24 more
[thread 123611 also had an error]
Run Code Online (Sandbox Code Playgroud)

我在这个主题上找到了几个有用的链接:

http://practicalcloudcomputing.com/post/444939181/outofmemoryjnigzip

http://www.javacodegeeks.com/2013/01/java-heap-space-native-heap-and-memory-problems.html

JVM选项如下:

-server -Xms2048m -Xmx2048m -XX:NewSize=384m 
-XX:MaxNewSize=384m 
-XX:SurvivorRatio=4 
-XX:MinHeapFreeRatio=11 
-XX:PermSize=80m 
-verbose:gc 
-XX:+PrintGCDateStamps 
-XX:+PrintGCDetails 
-XX:+DisableExplicitGC 
-Djava.awt.headless=TRUE -DUseSunHttpHandler=TRUE -Dsun.net.client.defaultConnectTimeout=25000 
-Dsun.net.client.defaultReadTimeout=50000 
-Dfile.encoding=UTF-8 
-Xloggc:gc.log 
Dcom.sun.management.jmxremote.port=9003 
-Dcom.sun.management.jmxremote.authenticate=FALSE 
-Dcom.sun.management.jmxremote.ssl=FALSE 
-Duser.language=de 
-Duser.region=DE 
-Duser.country=DE 
-Djboss.vfs.cache.TimedPolicyCaching.lifetime=1440000 
-DVFjavaWL=er.core.de
Run Code Online (Sandbox Code Playgroud)

更新

多一点分析.这是我对xml解析发生的事情的天真解释:

  1. 我们有一个单例JAXBHelper类,有一个实例级JAXBContext(这是一个线程安全对象).
  2. 每次请求进入时,我们都会尝试将传入的xml字符串绑定到Java对象(之前由JAXB从xsd生成).
  3. bind方法创建一个实例级unmarshaller(这些不是线程安全的,但创建起来很便宜).
  4. 然后bind方法调用unmarshaller的unmarshal(InputStream)方法.
  5. 现在,javax.xml.bind.helpers.AbstractUnmarshallerImpl然后调用SAXParserFactory.newInstance,进而调用一个类加载器,而这又需要的Inflater.

当然,实现(我们使用的是jaxb-ri 2.2.7)应该保留SAXParserFactory还是SAXParser并重新使用它们？我注意到它们都不是线程安全的,所以可以用a来完成ThreadLocal.或者可以Inflater重复使用实例？

那些不被重复使用的事实意味着我没有取消引用我的对象吗？在UnMarshaller似乎并不具有任何种类的close方法.

apa*_*gin 2

内存泄漏是由于 Inflaters 未关闭造成的。
看起来这些 Inflater 会落入 Old Generation，只要不触发 Full GC，它们的终结器就不会被调用。

为了防止 JRE 扫描所有 JAR 文件来查找 SAXParserFactory，您可以创建$JAVA_HOME/jre/lib/jaxp.properties包含以下行的文件：

javax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl
Run Code Online (Sandbox Code Playgroud)

删除选项也可能很有用-XX:+DisableExplicitGC，因为 JVM 运行时依赖于System.gc检测到本机内存不足时的调用。

归档时间：

查看次数：

1661 次

最近记录：

10 年，11 月 前
相关归档
将InputStream读取为UTF-8 94
什么时候我应该在JavaScript中使用delete vs setting元素为null？ 66
如何从一组数字计算平均值,中位数,模式和范围 48
将double转换为Int,向下舍入 44
如何选择jvm堆大小？ 24
如何使用JAXB2.0禁用DTD获取 21
哪里包含jaxb.properties文件？ 17
使用JaxB创建Java类 6
Java方法调用重载逻辑 3
JVM字节码浮动否定:不允许但可能吗？ 1
难疑归档
Java是"通过引用传递"还是"传递价值"？ 6270
使用jQuery为复选框设置"选中"？ 3988
@staticmethod和@classmethod有什么区别？ 3360
有一个CSS父选择器吗？ 2986
Python join:为什么是string.join(list)而不是list.join(string)？ 1669
外部"C"在C++中有什么影响？ 1511
我应该在哪里将<script>标记放在HTML标记中？ 1392
获取数据库中所有表的大小 1180
NP,NP-Complete和NP-Hard有什么区别？ 1064
Vim中的remap,noremap,nnoremap和vnoremap映射命令有什么区别？ 1045