Solr和JVM内存管理问题

问题描述 投票:1回答:2

我们在具有以下参数的4.6.0版上运行的Solr部署面临问题:

java -Xss256k -Xms512m -Xmx30g -jar start.jar

Solr在具有64GB RAM和12核的Windows Server 2008 R2上运行。 30 GB分配给Solr实例,数据导入后大约为索引大小。 1.2 GB。启动Solr实例后,Solr仪表板上的JVM内存使用情况将显示:

Used Memory: 8.61 GB
Peak Memory: 15.19 GB
Total Memory: 29.02 GB

[几天后,我们注意到内存的读像是

Used Memory: 22.32 GB
Peak Memory: 29.02 GB
Total Memory: 29.02 GB

每小时每小时刷新一次索引(凌晨2点至凌晨4点之间除外),其大小几乎从未改变。

但是,再等一会儿,Solr似乎变得毫无反应。 Solr仪表板无法加载,查询失败,并显示以下错误。我们进行了大量搜索,但似乎找不到任何解决方法。唯一的临时解决方法是,一旦Java进程超过30 GB的内存使用量,就重新启动Solr实例。此后,它可以正常运行约10-12天,但再次出现相同的问题。

并且以下给出的错误每天发生,通常发生在上午12点至早上6点之间。但是,这段时间内的流量非常少,并且在该时段没有数据更改的情况下,凌晨2点至凌晨4点之间也没有对Solr核心进行重新索引。

我们已经更改了jetty.xml中的以下几个配置,但这似乎并没有太大帮助。

<Set name="maxIdleTime">1200000</Set>
<Set name="lowResourceMaxIdleTime">10000</Set>

ERROR - 2015-12-04 02:10:56.821; org.apache.solr.common.SolrException; null:org.eclipse.jetty.io.EofException 
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:914) 
at org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:507) 
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:147) 
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107) 
at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) 
at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) 
at sun.nio.cs.StreamEncoder.write(Unknown Source) 
at java.io.OutputStreamWriter.write(Unknown Source) 
at org.apache.solr.util.FastWriter.flush(FastWriter.java:141) 
at org.apache.solr.util.FastWriter.write(FastWriter.java:55) 
at org.apache.solr.response.JSONWriter.writeStr(JSONResponseWriter.java:449) 
at org.apache.solr.response.JSONWriter.writeKey(JSONResponseWriter.java:103) 
at org.apache.solr.response.JSONWriter.writeSolrDocument(JSONResponseWriter.java:346) 
at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:275) 
at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:172) 
at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:183) 
at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:299) 
at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:95) 
at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:60) 
at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:698) 
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426) 
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) 
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) 
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) 
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) 
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) 
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) 
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) 
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) 
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) 
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) 
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) 
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) 
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) 
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) 
at org.eclipse.jetty.server.Server.handle(Server.java:368) 
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) 
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) 
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) 
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) 
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) 
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) 
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) 
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) 
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) 
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
at java.lang.Thread.run(Unknown Source) 

Caused by: java.net.SocketException: Software caused connection abort: socket write error 
at java.net.SocketOutputStream.socketWrite0(Native Method) 
at java.net.SocketOutputStream.socketWrite(Unknown Source) 
at java.net.SocketOutputStream.write(Unknown Source) 
at org.eclipse.jetty.io.ByteArrayBuffer.writeTo(ByteArrayBuffer.java:375) 
at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:164) 
at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:182) 
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:838) 
java memory solr jvm
2个回答
0
投票

根据我的经验,逐渐的无响应可能是JVM调整不良的症状。我建议使用带有VisualGC插件的VisualVM之类的工具监视JVM,或将GC登录到文件中,然后使用许多可用工具之一对其进行分析。

来自Wiki:


0
投票

如果solr和java版本分别为> 5.5和> 1.8.0_5,我建议在ConcurrentMarkSweep上为solr配置G1GC。


0
投票

感谢@Rajesh,这解决了我的问题。

© www.soinside.com 2019 - 2024. All rights reserved.