基于Prometheus的jvm监控指标详解

使用Prometheus 监控Springboot应用参考 Prometheus Operator实战—— Prometheus、Alertmanager、Grafana 监控Springboot服务
下面来看看jvm的监控指标

# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds.
# TYPE jvm_gc_collection_seconds summary

#这是一个Summary指标，与Histogram类似，可以对指标数据进行采样

并发收集器 CMS(Concurrent Mark-Sweep)

以牺牲吞吐量为代价来获得最短回收停顿时间的垃圾回收器
对于要求服务器响应速度的应用上，这种垃圾回收器非常适合。
CMS是用于对tenured generation的回收，也就是年老代的回收，目标是尽量减少应用的暂停时间，减少full gc发生的几率，
利用和应用程序线程并发的垃圾回收线程来标记清除年老代。
在启动JVM参数加上-XX:+UseConcMarkSweepGC ，这个参数表示对于老年代的回收采用CMS。CMS采用的基础算法是：标记—清除。

CMS不对堆空间整理压缩节约了垃圾回收的停顿时间，但也带来的堆空间的浪费。为了解决堆空间浪费问题，
CMS回收器不再采用简单的指针指向一块可用堆空 间来为下次对象分配使用。
而是把一些未分配的空间汇总成一个列表，当JVM分配对象空间的时候，会搜索这个列表找到足够大的空间来hold住这个对象。

CMS的另一个缺点是它需要更大的堆空间。因为CMS标记阶段应用程序的线程还是在执行的，那么就会有堆空间继续分配的情况，
为了保证在CMS回 收完堆之前还有空间分配给正在运行的应用程序，必须预留一部分空间。

在回收完成之前，堆没有足够空间分配！默认当老年代使用68%的时候，CMS就开始行动了。 – XX:CMSInitiatingOccupancyFraction =n 来设置这个阀值。

总得来说，CMS回收器减少了回收的停顿时间，但是降低了堆空间的利用率。

如果你的应用程序对停顿比较敏感，并且在应用程序运行的时候可以提供更大的内存和更多的CPU(也就是硬件牛逼)，那么使用CMS来收集会给你带来好处。
还有，如果在JVM中，有相对较多存活时间较长的对象(老年代比较大)会更适合使用CMS。

ParNew垃圾收集器
Par是Parallel的缩写,多线程的意思,
但是这里的多线程仅仅指垃圾收集多线程并行,并不是垃圾收集和程序并行运行.ParNew也需要暂停一切工作,然后多线程并行垃圾收集.

参数
“-XX:+UseConcMarkSweepGC”：指定使用CMS后，会默认使用ParNew作为新生代收集器；
“-XX:+UseParNewGC”：强制指定使用ParNew；
“-XX:ParallelGCThreads”：指定垃圾收集的线程数量，ParNew默认开启的收集线程尽量与CPU的数量相当；

# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds.
# TYPE jvm_gc_collection_seconds summary
jvm_gc_collection_seconds_count{gc="ParNew",} 87.0 ==>YGC
jvm_gc_collection_seconds_sum{gc="ParNew",} 15.487 ==>YGCT
jvm_gc_collection_seconds_count{gc="ConcurrentMarkSweep",} 0.0 ==>FGC
jvm_gc_collection_seconds_sum{gc="ConcurrentMarkSweep",} 0.0 ==>FGCT

# HELP jvm_memory_bytes_used Used bytes of a given JVM memory area.
# TYPE jvm_memory_bytes_used gauge
#jvm已用内存区域
jvm_memory_bytes_used{area="heap",} 2.334615096E9 ==>堆内存使用
jvm_memory_bytes_used{area="nonheap",} 1.8031396E8 ==>非堆内存使用

备注:

 nonheap =  "Code Cache" + "Metaspace" + "Compressed Class Space"heap = "Par Eden Space" +  "Par Survivor Space" +  "CMS Old Gen" 

结论：init约等于xms的值，max约等于xmx的值。

used是已经被使用的内存大小，committed是当前可使用的内存大小（包括已使用的），### committed >= used。

committed不足时jvm向系统申请，若超过max则发生OutOfMemoryError错误。

# HELP jvm_memory_bytes_committed Committed (bytes) of a given JVM memory area.
# TYPE jvm_memory_bytes_committed gauge
jvm_memory_bytes_committed{area="heap",} 8.267825152E9
jvm_memory_bytes_committed{area="nonheap",} 1.85270272E8# HELP jvm_memory_bytes_max Max (bytes) of a given JVM memory area.
# TYPE jvm_memory_bytes_max gauge
#jvm内存区域的最大字节数
jvm_memory_bytes_max{area="heap",} 8.267825152E9
jvm_memory_bytes_max{area="nonheap",} 1.59383552E9# HELP jvm_memory_bytes_init Initial bytes of a given JVM memory area.
# TYPE jvm_memory_bytes_init gauge
#jvm内存区域的初始化字节数
jvm_memory_bytes_init{area="heap",} 8.589934592E9
jvm_memory_bytes_init{area="nonheap",} 2555904.0# HELP jvm_memory_pool_bytes_used Used bytes of a given JVM memory pool.
# TYPE jvm_memory_pool_bytes_used gauge
#jvm内存池使用情况
jvm_memory_pool_bytes_used{pool="Code Cache",} 7.2995456E7
jvm_memory_pool_bytes_used{pool="Metaspace",} 9.6191616E7
jvm_memory_pool_bytes_used{pool="Compressed Class Space",} 1.1126888E7
jvm_memory_pool_bytes_used{pool="Par Eden Space",} 2.184666584E9
jvm_memory_pool_bytes_used{pool="Par Survivor Space",} 3342224.0
jvm_memory_pool_bytes_used{pool="CMS Old Gen",} 1.46606288E8# HELP jvm_memory_pool_bytes_committed Committed bytes of a given JVM memory pool.
# TYPE jvm_memory_pool_bytes_committed gauge
jvm_memory_pool_bytes_committed{pool="Code Cache",} 7.3596928E7
jvm_memory_pool_bytes_committed{pool="Metaspace",} 9.9876864E7
jvm_memory_pool_bytes_committed{pool="Compressed Class Space",} 1.179648E7
jvm_memory_pool_bytes_committed{pool="Par Eden Space",} 2.577006592E9
jvm_memory_pool_bytes_committed{pool="Par Survivor Space",} 3.2210944E8
jvm_memory_pool_bytes_committed{pool="CMS Old Gen",} 5.36870912E9# HELP jvm_memory_pool_bytes_max Max bytes of a given JVM memory pool.
# TYPE jvm_memory_pool_bytes_max gauge
#jvm内存池最大数
jvm_memory_pool_bytes_max{pool="Code Cache",} 2.5165824E8
jvm_memory_pool_bytes_max{pool="Metaspace",} 2.68435456E8
jvm_memory_pool_bytes_max{pool="Compressed Class Space",} 1.073741824E9
jvm_memory_pool_bytes_max{pool="Par Eden Space",} 2.577006592E9
jvm_memory_pool_bytes_max{pool="Par Survivor Space",} 3.2210944E8
jvm_memory_pool_bytes_max{pool="CMS Old Gen",} 5.36870912E9# HELP jvm_memory_pool_bytes_init Initial bytes of a given JVM memory pool.
# TYPE jvm_memory_pool_bytes_init gauge
#jvm内存池初始化数
jvm_memory_pool_bytes_init{pool="Code Cache",} 2555904.0
jvm_memory_pool_bytes_init{pool="Metaspace",} 0.0
jvm_memory_pool_bytes_init{pool="Compressed Class Space",} 0.0
jvm_memory_pool_bytes_init{pool="Par Eden Space",} 2.577006592E9
jvm_memory_pool_bytes_init{pool="Par Survivor Space",} 3.2210944E8
jvm_memory_pool_bytes_init{pool="CMS Old Gen",} 5.36870912E9# HELP jmx_config_reload_success_total Number of times configuration have successfully been reloaded.
# TYPE jmx_config_reload_success_total counter
jmx_config_reload_success_total 0.0# HELP jvm_classes_loaded The number of classes that are currently loaded in the JVM
# TYPE jvm_classes_loaded gauge
#当前jvm已加载类数量
jvm_classes_loaded 16377.0# HELP jvm_classes_loaded_total The total number of classes that have been loaded since the JVM has started execution
# TYPE jvm_classes_loaded_total counter
#从jvm运行开始加载的类的数量，这是一个Counter指标，递增
jvm_classes_loaded_total 16377.0# HELP jvm_classes_unloaded_total The total number of classes that have been unloaded since the JVM has started execution
# TYPE jvm_classes_unloaded_total counter
#jvm运行后卸载的类数量，这是一个Counter指标。生产环境一直是0
jvm_classes_unloaded_total 0.0# HELP jvm_info JVM version info
# TYPE jvm_info gauge
jvm_info{version="1.8.0_151-b12",vendor="Oracle Corporation",runtime="Java(TM) SE Runtime Environment",} 1.0# HELP os_free_physical_memory_bytes FreePhysicalMemorySize (java.lang<type=OperatingSystem><>FreePhysicalMemorySize)
# TYPE os_free_physical_memory_bytes gauge
os_free_physical_memory_bytes 1.9491221504E10# HELP os_committed_virtual_memory_bytes CommittedVirtualMemorySize (java.lang<type=OperatingSystem><>CommittedVirtualMemorySize)
# TYPE os_committed_virtual_memory_bytes gauge
os_committed_virtual_memory_bytes 3.4423967744E10# HELP os_total_swap_space_bytes TotalSwapSpaceSize (java.lang<type=OperatingSystem><>TotalSwapSpaceSize)
# TYPE os_total_swap_space_bytes gauge
os_total_swap_space_bytes 0.0# HELP os_max_file_descriptor_count MaxFileDescriptorCount (java.lang<type=OperatingSystem><>MaxFileDescriptorCount)
# TYPE os_max_file_descriptor_count gauge
os_max_file_descriptor_count 1048576.0# HELP os_system_load_average SystemLoadAverage (java.lang<type=OperatingSystem><>SystemLoadAverage)
# TYPE os_system_load_average gauge
os_system_load_average 3.84# HELP os_total_physical_memory_bytes TotalPhysicalMemorySize (java.lang<type=OperatingSystem><>TotalPhysicalMemorySize)
# TYPE os_total_physical_memory_bytes gauge
os_total_physical_memory_bytes 2.02692759552E11
# HELP os_system_cpu_load SystemCpuLoad (java.lang<type=OperatingSystem><>SystemCpuLoad)
# TYPE os_system_cpu_load gauge
os_system_cpu_load 0.04950495049504951# HELP os_free_swap_space_bytes FreeSwapSpaceSize (java.lang<type=OperatingSystem><>FreeSwapSpaceSize)
# TYPE os_free_swap_space_bytes gauge
os_free_swap_space_bytes 0.0# HELP os_available_processors AvailableProcessors (java.lang<type=OperatingSystem><>AvailableProcessors)
# TYPE os_available_processors gauge
os_available_processors 48.0# HELP os_process_cpu_load ProcessCpuLoad (java.lang<type=OperatingSystem><>ProcessCpuLoad)
# TYPE os_process_cpu_load gauge
os_process_cpu_load 0.0# HELP os_open_file_descriptor_count OpenFileDescriptorCount (java.lang<type=OperatingSystem><>OpenFileDescriptorCount)
# TYPE os_open_file_descriptor_count gauge
os_open_file_descriptor_count 163.0
# HELP jmx_scrape_duration_seconds Time this JMX scrape took, in seconds.
# TYPE jmx_scrape_duration_seconds gauge
jmx_scrape_duration_seconds 0.00121965# HELP jmx_scrape_error Non-zero if this scrape failed.
# TYPE jmx_scrape_error gauge
jmx_scrape_error 0.0# HELP jmx_config_reload_failure_total Number of times configuration have failed to be reloaded.
# TYPE jmx_config_reload_failure_total counter
jmx_config_reload_failure_total 0.0# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
#用户和系统的总cpu使用时间
process_cpu_seconds_total 2293.82# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.618365917041E9# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 163.0# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1048576.0# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 3.4423963648E10# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 3.942313984E9# HELP jvm_buffer_pool_used_bytes Used bytes of a given JVM buffer pool.
# TYPE jvm_buffer_pool_used_bytes gauge
#jvm缓冲区使用情况，包括Code Cache(编译后的代码缓存，
不同版本的jvm默认大小不同)、PS Old Gen（老年代）、PS Eden Space（伊甸园）、PS Survivor Space（幸存者）、PS Perm Gen（永久代）jvm_buffer_pool_used_bytes{pool="direct",} 1729229.0jvm_buffer_pool_used_bytes{pool="mapped",} 0.0
# HELP jvm_buffer_pool_capacity_bytes Bytes capacity of a given JVM buffer pool.
# TYPE jvm_buffer_pool_capacity_bytes gauge
#给定jvm的估算缓冲区大小
jvm_buffer_pool_capacity_bytes{pool="direct",} 1729229.0
jvm_buffer_pool_capacity_bytes{pool="mapped",} 0.0# HELP jvm_buffer_pool_used_buffers Used buffers of a given JVM buffer pool.
# TYPE jvm_buffer_pool_used_buffers gauge
#给定jvm的已使用缓冲区大小
jvm_buffer_pool_used_buffers{pool="direct",} 243.0
jvm_buffer_pool_used_buffers{pool="mapped",} 0.0# HELP jvm_threads_current Current thread count of a JVM
# TYPE jvm_threads_current gauge
#jvm当前线程数
jvm_threads_current 284.0# HELP jvm_threads_daemon Daemon thread count of a JVM
# TYPE jvm_threads_daemon gauge
#jvm后台线程数
jvm_threads_daemon 241.0# HELP jvm_threads_peak Peak thread count of a JVM
# TYPE jvm_threads_peak gauge
#jvm线程峰值
jvm_threads_peak 286.0# HELP jvm_threads_started_total Started thread count of a JVM
# TYPE jvm_threads_started_total counter
#jvm总启动线程数量，Counter指标
jvm_threads_started_total 1260.0# HELP jvm_threads_deadlocked Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers
# TYPE jvm_threads_deadlocked gauge
#死锁线程数量
jvm_threads_deadlocked 0.0# HELP jvm_threads_deadlocked_monitor Cycles of JVM-threads that are in deadlock waiting to acquire object monitors
# TYPE jvm_threads_deadlocked_monitor gauge
jvm_threads_deadlocked_monitor 0.0