JVM使用超过最大堆的LOT

Joh*_*ohn 7 java memory jvm

我们遇到的问题是JVM比max -Xmx使用的要多得多.

我们正在使用-Xmx2048,它目前正在使用17GB的OS内存.

我意识到JVM可以使用超过最大堆的使用,但我们使用15GB以上看起来很疯狂.

顶部转储如下所示:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                          
30477 root      20   0 22.683g 0.017t  18536 S   0.7 29.1  27:43.44 java -Xmx2048M -Xms1024M -XX:MaxPermSize=256M -XX:ReservedCodeCacheSize=128M  ....
Run Code Online (Sandbox Code Playgroud)

(注意它使用的是0.017TB或17GB).

我尝试使用jmap进行调试:

sudo jmap -J-d64 -heap 30477

Attaching to process ID 30477, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.75-b04

using thread-local object allocation.
Parallel GC with 8 thread(s)

Heap Configuration:
   MinHeapFreeRatio = 0
   MaxHeapFreeRatio = 100
   MaxHeapSize      = 2147483648 (2048.0MB)
   NewSize          = 1310720 (1.25MB)
   MaxNewSize       = 17592186044415 MB
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 268435456 (256.0MB)
   G1HeapRegionSize = 0 (0.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 316145664 (301.5MB)
   used     = 150828136 (143.8409194946289MB)
   free     = 165317528 (157.6590805053711MB)
   47.708431009827166% used
From Space:
   capacity = 20447232 (19.5MB)
   used     = 3406624 (3.248809814453125MB)
   free     = 17040608 (16.251190185546875MB)
   16.660563151041668% used
To Space:
   capacity = 21495808 (20.5MB)
   used     = 0 (0.0MB)
   free     = 21495808 (20.5MB)
   0.0% used
PS Old Generation
   capacity = 716177408 (683.0MB)
   used     = 516302952 (492.3848648071289MB)
   free     = 199874456 (190.6151351928711MB)
   72.09148825873044% used
PS Perm Generation
   capacity = 56098816 (53.5MB)
   used     = 55842840 (53.255882263183594MB)
   free     = 255976 (0.24411773681640625MB)
   99.54370516482915% used

15065 interned Strings occupying 1516808 bytes.
Run Code Online (Sandbox Code Playgroud)

并且...

sudo jmap -J-d64 -permstat 30477

Attaching to process ID 30477, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.75-b04
finding class loader instances ..done.
computing per loader stat ..done.
please wait.. computing liveness...liveness analysis may be inaccurate ...
class_loader    classes bytes   parent_loader   alive?  type

<bootstrap> 1908    11793728      null      live    <internal>
0x0000000780b97590  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b99b28  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b96cb0  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780113ca0  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b97c90  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780114458  1   3048      null      dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b892b0  1   3024    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b989a8  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780019cf0  7731    44085000    0x0000000780019d40  live    sun/misc/Launcher$AppClassLoader@0x0000000770239a20
0x0000000780113f10  1   3152    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780aa8918  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780114008  1   3040    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b7a8b8  1   3024    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780d236d0  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780019d40  207 1327296   null      live    sun/misc/Launcher$ExtClassLoader@0x00000007701eb8b8
0x0000000780113e28  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x00000007801140b0  1   3064    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x000000078dc814f8  1   3024      null      dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b8abf8  1   3208    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b990a8  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780114130  1   1904    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b98010  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x00000007801141b0  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780113620  27  86320   0x0000000780019cf0  live    org/apache/tapestry5/internal/plastic/PlasticClassLoader@0x000000077181b058
0x0000000780114418  1   3064    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b98d28  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b89270  1   3024    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780113ed0  1   3024    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b97910  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b997a8  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b96930  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780d36d38  1   3024    0x0000000780019d40  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b99ea8  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780bddff8  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780a63ca0  1   3024    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780114048  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780113fc8  1   3024      null      dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780113de8  1   3072    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b7afb8  1   3040      null      dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780113e68  1   1904    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b7ac38  1   1880      null      dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780113d58  1   3064    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x00000007801140f0  1   3280    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b98628  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x000000078088f900  0   0   0x0000000780019cf0  dead    java/util/ResourceBundle$RBClassLoader@0x00000007709cda40
0x0000000780b99428  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780d28ba0  1   3024    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b97030  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780114170  1   3064    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x00000007801143d8  1   3048    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780114240  1   1880    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770
0x0000000780b8af60  1   3104    0x0000000780019cf0  dead    sun/reflect/DelegatingClassLoader@0x0000000770059770

total = 53  9921    57418080        N/A     alive=4, dead=49        N/A    
Run Code Online (Sandbox Code Playgroud)

该应用程序是一个Play应用程序,我知道与类加载器有奇怪的事情,但我认为-permstat应该考虑到这一点,所以我很难过.

有任何想法吗?

约翰

Pio*_*cia 5

请注意,堆区域(由Xmx限制)只是JVM进程的一部分。

基本上,JVM占用的内存是以下各项的总和:

  • 堆(以Xmx为界)
  • PermGen(在Java 7及更低版本中,由MaxPermSize界定)
  • 线程堆栈:每个线程大约512KB-1024KB(取决于JVM和OS)
  • 本机缓冲区
  • JVM代码
  • 通过JNI访问的本机数据/代码

可能还有更多,您可以通过谷歌搜索找到它:)

因此,您必须检查所有这些区域,以确保正在消耗内存。

我敢打赌,您的线程泄漏很简单-数千个线程会吃掉很多内存(正如我所说,假设每个线程1MB,10000个线程对应10GB内存)。您可以简单地进行线程转储(jstack -l> dumpfile.txt)以查看是否存在问题。

当然也可以是使用本机缓冲区或JNI的外部库,但是可能性较小。


Joh*_*ohn 5

所以我们发现了这一点,这只是开发人员的错误。感谢所有建议的帮助。

无论如何,我都会回答这个问题,以防它对将来的任何人有帮助。

问题只是没有关闭使用 Class.getInputAsStream() 读取的资源上打开的流。

之所以会出现混乱,是因为 JVM 堆的使用量可以忽略不计,但每个未关闭的流不知何故留下了一些已分配但未释放的本机代码。

因为没有文件句柄泄漏(可能是因为我们每次都从主 JAR 文件中读取,所以它重复使用相同的句柄),并且因为 JVM 堆使用量很小,所以我们直到实际使用时才发现它。做了堆转储。

无论如何,这个故事的寓意是不要犯我所犯的错误,即假设如果 JVM 堆使用率较低,则没有必要查看堆转储,因为在这种情况下,看到仍然分配的数千个对象会导致我们一条线索。

谢谢大家!