[Bug]: Dead code elimination mark the annotation type as unused #43580

TharmiganK · 2024-11-13T11:57:48Z

Description

$Subject

Steps to Reproduce

Testing protobuf package with the dead code elimination and when I run the tests without any external dependency annotation. I am getting the following error:

        protobuf.types.any
[2024-11-13 17:16:37,531] SEVERE {b7a.log.crash} - $typedesce$MessageDescriptor 
java.lang.NoSuchFieldError: $typedesce$MessageDescriptor
        at ballerina.protobuf&0046types&0046any$test.1.tests.any_msg_pb.$annot_func$_911(tests/any_msg_pb.bal:25)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$gen$&0046&0060init&0062(protobuf.types.any:2)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$moduleInit(protobuf.types.any)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$moduleExecute(protobuf.types.any)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$lambda$$moduleExecute$(protobuf.types.any)
        at io.ballerina.runtime.internal.scheduling.SchedulerItem.execute(SchedulerItem.java:54)
        at io.ballerina.runtime.internal.scheduling.Scheduler.run(Scheduler.java:324)
        at io.ballerina.runtime.internal.scheduling.Scheduler.runSafely(Scheduler.java:291)
        at java.base/java.lang.Thread.run(Thread.java:840)

In the report, I see this type as a unused type:

    "unusedTypeDefNames": {
      "sourceTypeDefinitions": [
        "MessageDescriptor",
        "Error"
      ],
      "virtualTypeDefinitions": []
    },

This MessageDescriptor is the type of the annotation and the annotation is present in the test source code

Affected Version(s)

Ballerina SwanLake Update 11

OS, DB, other environment details and versions

Mac OS

Related area

-> Compilation

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

The text was updated successfully, but these errors were encountered:

MaryamZi · 2024-11-14T07:58:39Z

Seems like this happens when the annotation is defined in the default module and a non-default module's tests use the annotation.

test_proj

public type Config record {|
    int value;
|};

public annotation Config annot on type;

public function main() {
    
}

test_proj.mod's tests

import ballerina/test;
import test_proj;

@test_proj:annot {
    value: 100
}
type TestType record {|

|};

@test:Config
function testFn() {
}

$ bal test --eliminate-dead-code --dead-code-elimination-report
Compiling source
        maryamm/test_proj:0.1.0
Duration for unused BIR node analysis : 1ms
    maryamm/test_proj:0.1.0 : 1ms


Running Tests

        test_proj.mod
[2024-11-14 13:26:57,174] SEVERE {b7a.log.crash} - $typedesce$Config 
java.lang.NoSuchFieldError: $typedesce$Config
        at maryamm.test_proj&0046mod$test.0.tests.mod_test.$annot_func$_816(tests/mod_test.bal:4)
        at maryamm.test_proj&0046mod$test.0.$_init.$gen$&0046&0060init&0062(test_proj.mod:2)
        at maryamm.test_proj&0046mod$test.0.$_init.$moduleInit(test_proj.mod)
        at maryamm.test_proj&0046mod$test.0.$_init.$moduleExecute(test_proj.mod)
        at maryamm.test_proj&0046mod$test.0.$_init.$lambda$$moduleExecute$(test_proj.mod)
        at io.ballerina.runtime.internal.scheduling.SchedulerItem.execute(SchedulerItem.java:54)
        at io.ballerina.runtime.internal.scheduling.Scheduler.run(Scheduler.java:324)
        at io.ballerina.runtime.internal.scheduling.Scheduler.runSafely(Scheduler.java:291)
        at java.base/java.lang.Thread.run(Thread.java:840)

MaryamZi · 2024-11-14T09:26:05Z

In io.ballerina.projects.JBallerinaBackend#registerUnusedBIRNodes, the module gets sorted to the end, and then shouldOptimize(moduleContext) && (isRootModule(moduleContext) || moduleContext.isUsed()) evaluates to false for the module.

https://github.com/ballerina-platform/ballerina-lang/blob/feature-codegen-optimizer/compiler/ballerina-lang/src/main/java/io/ballerina/projects/JBallerinaBackend.java#L253

Thushara-Piyasekara · 2024-11-16T21:04:52Z

The reason for this behavior is because we added a constraint to not analyze non parent packages of testable packages during DCE. This was done to avoid false positive test results (see https://docs.google.com/document/d/1WNBN9hI7rbCL6wpozDAwImagUrM-sn7F/ for more details).

In the implementation, the logic for handling those scenarios was to stop the analysis if we refer to a non parent package node from a testable package.

Even though we stop the analysis, since the dependencies of testable packages are not optimized, the code will work. But in this case the logic used to determine testable pkg to build pkg usages fails.

We will have to update the logic to handle these cases or not support such cases.

I vaguely remember a having a conversation with @hasithaa about whether we should allow running tests from non build pkg modules with DCE flags. If I remember correctly the conclusion was not to allow such cases. But I cant remember the exact reasoning. I will look into this more and update.

Thushara-Piyasekara · 2024-11-16T21:23:01Z

In io.ballerina.projects.JBallerinaBackend#registerUnusedBIRNodes, the module gets sorted to the end, and then shouldOptimize(moduleContext) && (isRootModule(moduleContext) || moduleContext.isUsed()) evaluates to false for the module.

https://github.com/ballerina-platform/ballerina-lang/blob/feature-codegen-optimizer/compiler/ballerina-lang/src/main/java/io/ballerina/projects/JBallerinaBackend.java#L253

We can fix this by allowing the analyzer to enter the non default modules with testable pkgs by checking for moduleContext.bLangPackage().hasTestablePackage() condtion.

But this will not fully fix the issue because even though the analyzer goes inside the pkg it will stop analyzing the type reference because the following condition fails.

ballerina-lang/compiler/ballerina-lang/src/main/java/io/ballerina/projects/UsedTypeDefAnalyzer.java

Line 185 in 092cc69

!bType.tsymbol.pkgID.equals(usedBIRNodeAnalyzer.currentPkgID);

This condition only works for testable pkg's parent pkg references. We will have to introduce a new mechanism to determine build pkg references from non default testable pkgs.

Also we will have to update the logic in a similar manner for function invocations as well.

ballerina-lang/compiler/ballerina-lang/src/main/java/io/ballerina/projects/UsedBIRNodeAnalyzer.java

Line 471 in 092cc69

if (childPkgId.equals(currentPkgID)) {

I will update here if i find a proper way to do this.

chiranSachintha · 2024-11-26T08:17:39Z

Seems like this happens when the annotation is defined in the default module and a non-default module's tests use the annotation.

test_proj

public type Config record {|
    int value;
|};

public annotation Config annot on type;

public function main() {
    
}

test_proj.mod's tests

import ballerina/test;
import test_proj;

@test_proj:annot {
    value: 100
}
type TestType record {|

|};

@test:Config
function testFn() {
}

$ bal test --eliminate-dead-code --dead-code-elimination-report
Compiling source
        maryamm/test_proj:0.1.0
Duration for unused BIR node analysis : 1ms
    maryamm/test_proj:0.1.0 : 1ms


Running Tests

        test_proj.mod
[2024-11-14 13:26:57,174] SEVERE {b7a.log.crash} - $typedesce$Config 
java.lang.NoSuchFieldError: $typedesce$Config
        at maryamm.test_proj&0046mod$test.0.tests.mod_test.$annot_func$_816(tests/mod_test.bal:4)
        at maryamm.test_proj&0046mod$test.0.$_init.$gen$&0046&0060init&0062(test_proj.mod:2)
        at maryamm.test_proj&0046mod$test.0.$_init.$moduleInit(test_proj.mod)
        at maryamm.test_proj&0046mod$test.0.$_init.$moduleExecute(test_proj.mod)
        at maryamm.test_proj&0046mod$test.0.$_init.$lambda$$moduleExecute$(test_proj.mod)
        at io.ballerina.runtime.internal.scheduling.SchedulerItem.execute(SchedulerItem.java:54)
        at io.ballerina.runtime.internal.scheduling.Scheduler.run(Scheduler.java:324)
        at io.ballerina.runtime.internal.scheduling.Scheduler.runSafely(Scheduler.java:291)
        at java.base/java.lang.Thread.run(Thread.java:840)

Encountering the following error with latest changes with the feature-optimizer-codegen branch.

[2024-11-26 11:32:12,484] SEVERE {b7a.log.crash} - Compile the module first! 
java.lang.IllegalStateException: Compile the module first!
        at io.ballerina.projects.ModuleContext.getBLangPackageOrThrow(ModuleContext.java:227)
        at io.ballerina.projects.ModuleContext.bLangPackage(ModuleContext.java:214)
        at io.ballerina.projects.JBallerinaBackend.markTestDependenciesForDuplicateBIRGen(JBallerinaBackend.java:294)
        at io.ballerina.projects.JBallerinaBackend.performCodeGen(JBallerinaBackend.java:193)
        at io.ballerina.projects.JBallerinaBackend.<init>(JBallerinaBackend.java:180)
        at io.ballerina.projects.JBallerinaBackend.lambda$from$0(JBallerinaBackend.java:150)
        at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1228)
        at io.ballerina.projects.PackageCompilation.getCompilerBackend(PackageCompilation.java:179)
        at io.ballerina.projects.JBallerinaBackend.from(JBallerinaBackend.java:149)
        at io.ballerina.projects.JBallerinaBackend.from(JBallerinaBackend.java:137)
        at io.ballerina.cli.task.CompileTask.execute(CompileTask.java:242)
        at io.ballerina.cli.TaskExecutor.executeTasks(TaskExecutor.java:40)
        at io.ballerina.cli.cmd.TestCommand.execute(TestCommand.java:415)
        at java.base/java.util.Optional.ifPresent(Optional.java:178)
        at io.ballerina.cli.launcher.Main.main(Main.java:63)

Create an issue for this #43652

TharmiganK added the Type/Bug label Nov 13, 2024

ballerina-bot added needTriage The issue has to be inspected and labeled manually userCategory/Compilation labels Nov 13, 2024

TharmiganK mentioned this issue Nov 13, 2024

Fix issues with Dead Code Elimination feature ballerina-platform/module-ballerina-protobuf#245

Merged

5 tasks

MaryamZi added Team/CompilerFE All issues related to Language implementation and Compiler, this exclude run times. and removed needTriage The issue has to be inspected and labeled manually labels Nov 13, 2024

gimantha self-assigned this Nov 18, 2024

MaryamZi mentioned this issue Nov 19, 2024

[Phase 1][Ballerina standard libraries] Dead code elimination report for Ballerina library modules ballerina-platform/ballerina-library#7339

Open

51 tasks

chiranSachintha assigned chiranSachintha and unassigned gimantha Nov 25, 2024

chiranSachintha added this to Ballerina Team Main Board Nov 25, 2024

chiranSachintha moved this to Planned for Sprint in Ballerina Team Main Board Nov 25, 2024

chiranSachintha moved this from Planned for Sprint to In Progress in Ballerina Team Main Board Nov 26, 2024

chiranSachintha moved this from In Progress to On Hold in Ballerina Team Main Board Nov 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Dead code elimination mark the annotation type as unused #43580

[Bug]: Dead code elimination mark the annotation type as unused #43580

TharmiganK commented Nov 13, 2024 •

edited

Loading

MaryamZi commented Nov 14, 2024

MaryamZi commented Nov 14, 2024

Thushara-Piyasekara commented Nov 16, 2024

Thushara-Piyasekara commented Nov 16, 2024 •

edited

Loading

chiranSachintha commented Nov 26, 2024

[Bug]: Dead code elimination mark the annotation type as unused #43580

[Bug]: Dead code elimination mark the annotation type as unused #43580

Comments

TharmiganK commented Nov 13, 2024 • edited Loading

Description

Steps to Reproduce

Affected Version(s)

OS, DB, other environment details and versions

Related area

Related issue(s) (optional)

Suggested label(s) (optional)

Suggested assignee(s) (optional)

MaryamZi commented Nov 14, 2024

MaryamZi commented Nov 14, 2024

Thushara-Piyasekara commented Nov 16, 2024

Thushara-Piyasekara commented Nov 16, 2024 • edited Loading

chiranSachintha commented Nov 26, 2024

TharmiganK commented Nov 13, 2024 •

edited

Loading

Thushara-Piyasekara commented Nov 16, 2024 •

edited

Loading