我最近创建了一个Nifi docker镜像,可以使用我的自定义配置和状态运行nifi docker。
一切都很好,但过了一段时间我收到以下错误:
2019-04-08 10:42:26,107 ERROR [Timer-Driven Process Thread-10] o.a.n.c.r.StandardProcessSession Failed to commit session java.lang.IllegalStateException: Cannot update journal file ./flowfile_repository/journals/449486.journal because no header has been written yet.. Will roll back.
2019-04-08 10:42:26,108 ERROR [Timer-Driven Process Thread-10] d.f.fit.processors.opcua.GetOPCData GetOPCData[id=e8affa98-0169-1000-1717-d9e8f54293ea] GetOPCData[id=e8affa98-0169-1000-1717-d9e8f54293ea] failed to process session due to java.lang.IllegalStateException: Cannot update journal file ./flowfile_repository/journals/449486.journal because no header has been written yet.; Processor Administratively Yielded for 1 sec:
java.lang.IllegalStateException: Cannot update journal file ./flowfile_repository/journals/449486.journal because no header has been written yet.
at org.apache.nifi.wali.LengthDelimitedJournal.update(LengthDelimitedJournal.java:230)
at org.apache.nifi.wali.SequentialAccessWriteAheadLog.update(SequentialAccessWriteAheadLog.java:123)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:309)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:259)
at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:368)
at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:343)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:205)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.lang.IllegalStateException: Cannot update journal file ./flowfile_repository/journals/449486.journal because no header has been written yet.
at org.apache.nifi.wali.LengthDelimitedJournal.update(LengthDelimitedJournal.java:230)
at org.apache.nifi.wali.SequentialAccessWriteAheadLog.update(SequentialAccessWriteAheadLog.java:123)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:309)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:259)
at org.apache.nifi.controller.repository.StandardProcessSession.rollback(StandardProcessSession.java:1060)
at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:501)
... 12 common frames omitted
它与java堆或内存有关吗?我该如何更正此错误?
如果您不关心流中的任何数据,可以关闭并删除所有“存储库”目录,默认情况下它们是在NiFi的主目录中创建的。
它没有解释是什么原因造成的,但它应该让你回到工作状态。
在我的情况下,我用完了打开文件句柄,不幸的是错误没有给出任何指示。我怎么想的是:
除非查询诊断。来源:curl http://localhost:8080/nifi-api/system-diagnostics
检查所有“免费”字段(例如“provenanceRepositoryStorageUsage.freeSpace”)并确保您有足够的资源。
对我来说这返回的值如下:
flowFileRepositoryStorageUsage.totalSpaceBytes=-1
当NiFi无法确定这些值时,会报告-1。作为诊断的一部分,将记录正确的错误消息。您可能需要快速检查日志。在我的情况下,我设置了滚动日志,因此日志消息很快消失(许多处理器记录失败并填满日志)。错误看起来像:
2019-06-03 18:48:59,619 ERROR [pool-10-thread-1] org.apache.nifi.controller.repository.WriteAheadFlowFileRepository Unable to checkpoint FlowFile Repository due to java.io.FileNotFoundException: ./flowfile_repository/journals/8243.journal (Too many open files)
java.io.FileNotFoundException: ./flowfile_repository/journals/8243.journal (Too many open files)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.\u003cinit\u003e(FileOutputStream.java:213)
at java.io.FileOutputStream.\u003cinit\u003e(FileOtutputStream.java:162)\n\tat org.apache.nifi.wali.LengthDelimitedJournal.getOutputStream(LengthDelimitedJournal.java:136)
at org.apache.nifi.wali.LengthDelimitedJournal.writeHeader(LengthDelimitedJournal.java:151)
at org.apache.nifi.wali.SequentialAccessWriteAheadLog.checkpoint(SequentialAccessWriteAheadLog.java:306)
at org.apache.nifi.wali.SequentialAccessWriteAheadLog.checkpoint(SequentialAccessWriteAheadLog.java:251)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.checkpoint(WriteAheadFlowFileRepository.java:735)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository$1.run(WriteAheadFlowFileRepository.java:693)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
这里的重要部分是“打开文件过多”。
使用linux获取当前限制:ulimit -x -n
要使用linux获取打开文件的数量:sudo ls /proc/$NIFI_PID/fd | wc -l
我正在使用docker所以我使用docker run命令增加了限制:docker run --ulimit nofile=1048576:1048576 ...
其他解决方案是:增加主机的限制,确定哪些处理器占用了手柄并重新处理它们(不那么容易)。
使用日志限制运行Nifi docker:
docker run --name nifi -p 8080:8080 --log-driver json-file --log-opt max-size=5m --log-opt max-file=4 --restart always -d nifi