在生成报告时通过jenkins内存不足错误运行Jmeter脚本

Question

[通过Jenkins运行Jmeter脚本时，我面临内存不足的问题。过程是在框架中编写了代码，该代码会将生成的XML文件结果转换为CSV，然后将HTML转换为在仪表板中发布报告。

我已经尝试过将Jenkins服务器的堆空间从32 GB增加到25 GB。似乎最初需要1 Gb，然后经过一段时间后会抛出错误，即使堆中仍有24 GB pf内存，我也跑了为此，可以使用-h。

还尝试通过以下方式增加Jmeter内存：设置HEAP = -Xms1g -Xmx8g -XX：MaxMetaspaceSize = 512m

[该脚本在Windows上的Jnekins服务器中可以很好地执行。但是，当jenkin服务器处于Linux模式时，它将克服错误。

听到是我的错误日志。

/var/lib/jenkins/workspace/ITT2_Execution/Resources/csvReportPath/ITT2_Unicast_Broker_Download_count.xml/var/lib/jenkins/workspace/ITT2_Execution/Resources//Configuration/transformGaurav.xsl/var/lib/jenkins/workspace/ITT2_Execution/jmeter_reports/ITT2_Unicast_Broker_Download_2_Oct_2019_19_3_52_Count.html{titleReport=ITT2_Unicast_Broker_DownloadCountReport, dateReport=2-Oct-2019 22:02:38}
Finished Parsing
/var/lib/jenkins/workspace/ITT2_Execution/Resources/csvReportPath/AutomationReport_5.2.4.2018.20_2_Oct_2019_19_3_52_count.xml/var/lib/jenkins/workspace/ITT2_Execution/Resources//Configuration/transformGaurav.xsl/var/lib/jenkins/workspace/ITT2_Execution/jmeter_reports/AutomationReport_5.2.4.2018.20_2_Oct_2019_19_3_52.html{titleReport=nullCountReport, dateReport=2-Oct-2019 22:02:39}
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2019/10/02 18:33:10 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2019/10/02 18:33:10 - please wait.
JVMDUMP032I JVM requested Heap dump using '/var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0001.phd' in response to an event
JVMDUMP010I Heap dump written to /var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0001.phd
JVMDUMP032I JVM requested System dump using '/var/lib/jenkins/workspace/ITT2_Execution/core.20191002.183310.40181.0002.dmp' in response to an event
JVMDUMP010I System dump written to /var/lib/jenkins/workspace/ITT2_Execution/core.20191002.183310.40181.0002.dmp
JVMDUMP032I JVM requested Java dump using '/var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0003.txt' in response to an event
JVMDUMP010I Java dump written to /var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0003.txt
JVMDUMP032I JVM requested Snap dump using '/var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0005.trc' in response to an event
JVMDUMP010I Snap dump written to /var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0005.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Heap dump using '/var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0004.phd' in response to an event
JVMDUMP010I Heap dump written to /var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0004.phd
JVMDUMP032I JVM requested Java dump using '/var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0006.txt' in response to an event
JVMDUMP010I Java dump written to /var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0006.txt
JVMDUMP032I JVM requested Snap dump using '/var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0007.trc' in response to an event
JVMDUMP010I Snap dump written to /var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0007.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:508)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    at java.lang.Thread.run(Thread.java:811)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.xerces.xni.XMLString.toString(Unknown Source)
    at org.apache.xerces.parsers.AbstractDOMParser.characters(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanCharReference(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
    at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
    at utils.APIReportProcessing.fetchAPIReportDetailModuleWise(APIReportProcessing.java:110)
    at jmeterRun.RunProcess.prepareFinalResultsMerged(RunProcess.java:228)
    at jmeterRun.ControllerJMeter.main(ControllerJMeter.java:139)
    ... 6 more
[WARNING] Attempt to (de-)serialize anonymous class hudson.maven.reporters.MavenArtifactArchiver$2; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
[WARNING] Attempt to (de-)serialize anonymous class hudson.maven.reporters.MavenFingerprinter$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/

感谢Bibek

Answer 1

JMeter进程在Java虚拟机（JVM）中的各种设置和参数下运行。 Java堆空间（如错误消息中所述）是JVM从底层操作系统获取的用于分配空间以创建必要对象的内存。

JMeter的默认配置（对于Windows请参见jmeter.bat或对于非Windows系统脚本请参见jmeter）假定堆空间仅为512 MB。考虑到许多现代智能手机的容量要高出四倍，这实际上是很低的！如果您的测试运行的对象超过512Mb，则将出现OOM错误，并且测试将失败。

幸运的是，有一个简单的解决方案。只需将最大堆大小增加到总可用物理RAM的〜80％。为此，请在您的JMeter启动脚本中找到以下行：

HEAP =“-Xms1g -Xmx25g”

现在相应地更改-Xmx值。例如：如果要将最大堆大小设置为25 GB，则需要将行更改为：

HEAP =“-Xms1g -Xmx25g”

要应用更改，您需要重新启动JMeter。

Answer 2

根据Understand the OutOfMemoryError Exception条：

线程线程名中的异常：java.lang.OutOfMemoryError：Java堆空间

原因：详细消息Java堆空间指示无法在Java堆中分配对象。此错误不一定表示内存泄漏。该问题可以像配置问题一样简单，在该问题中，指定的堆大小（或默认大小，如果未指定）对于应用程序来说是不够的。在其他情况下，尤其是对于寿命长的应用程序，该消息可能表明该应用程序无意间持有了对对象的引用，这防止了对象被垃圾回收。这是等效于Java语言的内存泄漏。注意：应用程序调用的API也可能无意中包含对象引用。

此错误的另一个潜在来源是过度使用终结器的应用程序。如果类具有finalize方法，则该类型的对象在垃圾回收时不会回收其空间。取而代之的是，在进行垃圾回收之后，将对象排队等待定稿，这将在以后发生。在Oracle Sun实施中，终结器由为终结队列提供服务的守护程序线程执行。如果终结器线程无法跟上终结器队列的速度，则Java堆可能会填满，并且会抛出此类OutOfMemoryError异常。一种可能导致这种情况的情况是，应用程序创建了高优先级线程，这些线程导致终结处理队列以比终结器线程为该队列提供服务的速率更快的速率增加。

操作：您可以在Monitor the Objects Pending Finalization.中找到有关如何监视最终确定的对象的更多信息>

因此，请使用Java Memory Map工具调查最大的对象及其所属的过程。

[而且似乎您正在Linux或其他类似Unix的操作系统上运行Jenkins，而SET命令是Windows特定的。此外，Jenkins不尊重HEAP环境变量，而是使用JAVA_ARGS和/或JENKINS_JAVA_OPTIONS。

对于JMeter：

HEAP="-Xms1g -Xmx8g" && export HEAP
对于詹金斯：
JENKINS_JAVA_OPTIONS="-Xms1g -Xmx8g" && export JENKINS_JAVA_OPTIONS
更多信息：

• How to add Java arguments to Jenkins?
• 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure