[Bug]: Dead Code Elimination Fails for ballerina/random module

[aashikam]

Description

The dead code elimination fails with the following error for ballerina/random module.

Command used:

BAL_DISABLE_HARDCODED_OPTIMIZATIONS=true ../target/ballerina-runtime/bin/bal test --eliminate-dead-code --dead-code-elimination-report

Compiling source
        ballerina/random:1.5.1
Duration for unused BIR node analysis : 4ms
    ballerina/random:1.5.1 : 3ms
    ballerina/time:2.6.0 : 1ms

ballerina: Oh no, something really went wrong. Bad. Sad.

We appreciate it if you can report the code that broke Ballerina in
https://github.com/ballerina-platform/ballerina-lang/issues with the
log you get below and your sample code.

We thank you for helping make us better.

[2024-11-13 15:22:29,840] SEVERE {b7a.log.crash} - Cannot find the generated jar library for module: time 
java.lang.IllegalStateException: Cannot find the generated jar library for module: time
        at io.ballerina.projects.JBallerinaBackend.lambda$codeGeneratedLibrary$17(JBallerinaBackend.java:1031)
        at java.base/java.util.Optional.orElseThrow(Optional.java:403)
        at io.ballerina.projects.JBallerinaBackend.codeGeneratedLibrary(JBallerinaBackend.java:1030)
        at io.ballerina.projects.JBallerinaBackend.codeGeneratedLibrary(JBallerinaBackend.java:520)
        at io.ballerina.projects.JarResolver.addCodeGeneratedLibraryPaths(JarResolver.java:172)
        at io.ballerina.projects.JarResolver.lambda$getJarFilePathsRequiredForExecution$4(JarResolver.java:121)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
        at io.ballerina.projects.JarResolver.getJarFilePathsRequiredForExecution(JarResolver.java:119)
        at io.ballerina.projects.JarResolver.getJarFilePathsRequiredForExecution(JarResolver.java:100)
        at io.ballerina.projects.JarResolver.getJarFilePathsRequiredForTestExecution(JarResolver.java:306)
        at org.ballerinalang.testerina.core.TestProcessor.addTestExecutionDependencies(TestProcessor.java:172)
        at org.ballerinalang.testerina.core.TestProcessor.generateTestSuite(TestProcessor.java:158)
        at org.ballerinalang.testerina.core.TestProcessor.testSuite(TestProcessor.java:118)
        at io.ballerina.cli.utils.TestUtils.createTestSuitesForProject(TestUtils.java:306)
        at io.ballerina.cli.task.RunTestsTask.runTestsUsingSuiteJSON(RunTestsTask.java:201)
        at io.ballerina.cli.task.RunTestsTask.execute(RunTestsTask.java:185)
        at io.ballerina.cli.TaskExecutor.executeTasks(TaskExecutor.java:40)
        at io.ballerina.cli.cmd.TestCommand.execute(TestCommand.java:415)
        at java.base/java.util.Optional.ifPresent(Optional.java:178)
        at io.ballerina.cli.launcher.Main.main(Main.java:63)

Versions
Ballerina lang: 2201.11.0-20241111-172200-095db58f
time: 2.6.0-20241113-103200-92c5939

Note that BAL_DISABLE_HARDCODED_OPTIMIZATIONS=true ../target/ballerina-runtime/bin/bal test works without an issue.

Steps to Reproduce

Steps:
ballerina-platform/ballerina-library#7339

Affected Version(s)

Ballerina lang: 2201.11.0-20241111-172200-095db58f

[ayeshLK]

Same thing happened with Ballerina OS module. Following is the stacktrace;

[2024-11-13 16:25:22,565] SEVERE {b7a.log.crash} - Cannot find the generated jar library for module: io 
java.lang.IllegalStateException: Cannot find the generated jar library for module: io
	at io.ballerina.projects.JBallerinaBackend.lambda$codeGeneratedLibrary$17(JBallerinaBackend.java:1031)
	at java.base/java.util.Optional.orElseThrow(Optional.java:403)
	at io.ballerina.projects.JBallerinaBackend.codeGeneratedLibrary(JBallerinaBackend.java:1030)
	at io.ballerina.projects.JBallerinaBackend.codeGeneratedOptimizedLibrary(JBallerinaBackend.java:525)
	at io.ballerina.projects.JarResolver.addOptimizedLibraryPaths(JarResolver.java:181)
	at io.ballerina.projects.JarResolver.addCodeGeneratedLibraryPaths(JarResolver.java:169)
	at io.ballerina.projects.JarResolver.lambda$getJarFilePathsRequiredForExecution$4(JarResolver.java:121)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at io.ballerina.projects.JarResolver.getJarFilePathsRequiredForExecution(JarResolver.java:119)
	at io.ballerina.projects.JarResolver.getJarFilePathsRequiredForExecution(JarResolver.java:100)
	at io.ballerina.projects.JarResolver.getJarFilePathsRequiredForTestExecution(JarResolver.java:306)
	at org.ballerinalang.testerina.core.TestProcessor.addTestExecutionDependencies(TestProcessor.java:172)
	at org.ballerinalang.testerina.core.TestProcessor.generateTestSuite(TestProcessor.java:158)
	at org.ballerinalang.testerina.core.TestProcessor.testSuite(TestProcessor.java:118)
	at io.ballerina.cli.utils.TestUtils.createTestSuitesForProject(TestUtils.java:306)
	at io.ballerina.cli.task.RunTestsTask.runTestsUsingSuiteJSON(RunTestsTask.java:201)
	at io.ballerina.cli.task.RunTestsTask.execute(RunTestsTask.java:185)
	at io.ballerina.cli.TaskExecutor.executeTasks(TaskExecutor.java:40)
	at io.ballerina.cli.cmd.TestCommand.execute(TestCommand.java:415)
	at java.base/java.util.Optional.ifPresent(Optional.java:178)
	at io.ballerina.cli.launcher.Main.main(Main.java:63)

@MaryamZi are we looking into this ?

[MaryamZi]

This is most probably not related to dead code elimination. cc @azinneera

We can also probably compare the dead_code_elimination_report and native_dependency_optimization_report files when it succeeds vs fails to verify that this isn't due to dead code elimination.

[ayeshLK]

Following is the dead_code_elimination_report generated when this error occur;

dead_code_elimination_report.json

[MaryamZi]

Looping in @Thushara-Piyasekara also.

[Thushara-Piyasekara]

If the analyzer determines a module is 100% unused, we dont generate thin JARs or BIRs at all for that module. Even if its used we don't cache the optimized Thin JARs or BIRs in the usual .ballerina cache. As per the offline discussion with @MaryamZi, she mentioned this is happening intermittently. The reason might be because we don't have the ballerina/time module in .ballerina cache at all and it is failing some kind of check in the compiler. I will check and update here.

[ayeshLK]

@ayeshLK are you sure this is the correct optimization report?

I tried and could not reproduce the usage of ballerina/os. There doesnt seems to any bal tests in ballerina/random repository that imports ballerina/os. Please correct me if im wrong. I will keep debugging the available tests in the meantime.

@Thushara-Piyasekara OS module does not use Random module. What I mentioned here [1] is that I am getting a similar issue (which Random module has) in the OS module

[1] - #43579 (comment)

[Thushara-Piyasekara]

@ayeshLK are you sure this is the correct optimization report?
I tried and could not reproduce the usage of ballerina/os. There doesnt seems to any bal tests in ballerina/random repository that imports ballerina/os. Please correct me if im wrong. I will keep debugging the available tests in the meantime.

@Thushara-Piyasekara OS module does not use Random module. What I mentioned here [1] is that I am getting a similar issue (which Random module has) in the OS module

[1] - #43579 (comment)

It is my mistake. I didn't read your comment properly. Will make sure to read before replying in the future.

[MaryamZi]

Fixed in #43582.