OverlappingFileLockException from SerializerCheckCompilerPluginComponent$$anon$1.interactWithTheCacheFile

[gervaisb]

Hi there,

We have enabled the plugin on many modules of our multi-module project. However, we are facing strange compilation errors:

[error] ## Exception when compiling 3 sources to /Users/me/acme/a-module/target/scala-2.13/test-classes
[error] java.nio.channels.OverlappingFileLockException
[error] java.base/sun.nio.ch.FileLockTable.checkList(FileLockTable.java:229)
[error] java.base/sun.nio.ch.FileLockTable.add(FileLockTable.java:123)
[error] java.base/sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1109)
[error] java.base/java.nio.channels.FileChannel.lock(FileChannel.java:1063)
[error] org.virtuslab.ash.SerializerCheckCompilerPluginComponent$$anon$1.interactWithTheCacheFile(SerializerCheckCompilerPluginComponent.scala:212)
[error] org.virtuslab.ash.SerializerCheckCompilerPluginComponent$$anon$1.apply(SerializerCheckCompilerPluginComponent.scala:57)
[error] scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:459)
[error] scala.tools.nsc.Global$GlobalPhase.run(Global.scala:406)
[error] scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1524)
[error] scala.tools.nsc.Global$Run.compileUnits(Global.scala:1508)
[error] scala.tools.nsc.Global$Run.compileSources(Global.scala:1500)
...

Sometimes, we also get another error:

[error] ## Exception when compiling 10 sources to /Users/blaise/Projects/masana-backend/services/masana-webapi/beneficiary-profile/target/scala-2.13/classes
[error] spray.json.JsonParser$ParsingException: Unexpected end-of-input at input index 0 (line 1, position 1), expected JSON Value:
[error]
[error] ^
[error]
[error] spray.json.JsonParser.fail(JsonParser.scala:237)
[error] spray.json.JsonParser.value(JsonParser.scala:79)
[error] spray.json.JsonParser.parseJsValue(JsonParser.scala:51)
[error] spray.json.JsonParser.parseJsValue(JsonParser.scala:47)
[error] spray.json.JsonParser$.apply(JsonParser.scala:30)
[error] spray.json.RichString.parseJson(package.scala:50)
[error] org.virtuslab.ash.writer.PersistenceSchemaWriter.$anonfun$lastDump$2(PersistenceSchemaWriter.scala:21)
[error] scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
[error] scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
[error] scala.collection.mutable.Growable.addAll(Growable.scala:62)
...

Both errors seem to be random and does not appear on the same module. Do you think it may be caused by your plugin ?

• SBT : 1.8.2
• sbt-akka-serialization-helper: 0.7.0

Plugin is enabled on each module:

lazy val `a-module` = (project in file("a-module"))
  .settings(name := "a-module")
  .enablePlugins(AkkaSerializationHelperPlugin)
  .settings(
    Compile / scalacOptions += "-P:serializability-checker-plugin:--types-explicitly-marked-as-serializable=scala.util.Either",
  )

[PawelLipski]

Huh they might be caused by ASH indeed :/// such issues are notoriously difficult to debug... anyway, as a first step: would it be possible for you to share the repo with me? or is it a NDA-ed/proprietary project?

[gervaisb]

Indeed that will be very difficult to debug. Unfortunately it is proprietary project.

This project as more than 20 modules and the error started when the plugin has been enabled on around 15 (I guess).

At this time. I had to disable the plugin on all modules and everything goes back to normal. Which seems to prove that the plugin is in cause, sadly.

[PawelLipski]

Ha! with respect to java.nio.channels.OverlappingFileLockException, we've already got a similar problem fixed at #218... stay tuned, I'll get that fixed here as well soon.

If it comes to spray.json.JsonParser$ParsingException... that's a new one for me, but still I'll try to get that sorted out. Actually, do you rely on the functionality of dump-persistence-schema-compiler-plugin, i.e. do you run ashDumpPersistenceSchema sbt task? If not, then you can just use ashDumpPersistenceSchemaCompilerPlugin / ashCompilerPluginEnable := false (as documented in the GUIDE).

[gervaisb]

No, I do not run ’ashDumpPersistenceSchema´. I have tried once but that’s not part of our build.
I was a bit surprised to see that name in the stack trace since we do not run it.

I’ll try to disable it. Thanks for the suggestion. 👌

[gervaisb]

About the issue fixed in #218 and the comment in the code:

Below retry-loop is needed because of possible OverlappingFileLockException that might occur if codec-registration-checker-plugin is enabled in multiple projects (modules) and these projects are compiled in parallel by sbt compile. ...

This looks like our actual problem. The issue started once I generalized the plugin to all our modules. 🤞

[PawelLipski]

Okay... I've pushed a potential fix, I'll give you the snapshot version soon once it builds...

[PawelLipski]

@gervaisb pls try 0.7.0+3-53846ab8-SNAPSHOT, set up as in #268 (comment) 🧐

[gervaisb]

At first sight, this seems to be solved.

I did not get the issue anymore on my machine and the CI is passing. I have launched a few more CI jobs and told my colleagues to test locally to be sure. I'll keep you informed.

[PawelLipski]

Gr8! 🎉 I'll release 0.7.1 soon then

[PawelLipski]

v0.7.1 is out 📦

[gervaisb]

🎉 I confirm it is solved on our side. Will get the 0.7.1 asap.
Thanks a lot.

[gervaisb]

Hello there. Me again.

We got another similar error:

java.lang.RuntimeException: OverlappingFileLockException thrown, message: null
[error] 	at org.virtuslab.ash.SerializerCheckCompilerPluginComponent$$anon$1.interactWithTheCacheFile(SerializerCheckCompilerPluginComponent.scala:254)
[error] 	at org.virtuslab.ash.SerializerCheckCompilerPluginComponent$$anon$1.apply(SerializerCheckCompilerPluginComponent.scala:58)
[error] 	at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:459)

This one is much less frequent but still appearing from time to time.

[PawelLipski]

Huh looks like even more retries are needed ☹️ I'll get that increased soon then...

[PawelLipski]

Let's check a simple timeout increase in #316... if it doesn't help, we'll think of a more proper solution (rel: #312)

[PawelLipski]

...and pls check v0.7.2