Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AL-Pipeline: Table "XXXX" for the extension object is not found, despite symbols being installed correctly #3717

Closed
EsatSayinkaplanAccelent opened this issue Oct 10, 2024 · 17 comments
Assignees

Comments

@EsatSayinkaplanAccelent
Copy link

EsatSayinkaplanAccelent commented Oct 10, 2024

Describe the issue
Pipeline is set up in DevOps with 3rd Party Apps and Dependencies located in InstallApps Folder.
The sync and publish step of the 3rd parties works without any errors.
The problem is later on when the app folders are gettings compiled with external dependencies.
Some Apps do compile correctly with packages but one of the big ones throws the error:
The target Table 'XXXX' for the extension object is not found, despite the apps definitely being already installed.
The symbols used are either directly provided by the partners or downloaded from the production env. via "download symbols".
If compiled via VS Code, it works.

Dont mind the override to input the DLLs.

If you need additional Info, feel free to ask.

Scripts used to create container and cause the issue

Run-AlPipeline @params `
    -pipelinename $pipelineName `
    -containerName $containerName `
    -imageName $imageName `
    -bcAuthContext $authContext `
    -environment $environmentName `
    -artifact $artifact.replace('{INSIDERSASTOKEN}',$insiderSasToken) `
    -memoryLimit $memoryLimit `
    -baseFolder $baseFolder `
    -licenseFile $LicenseFile `
    -installApps $installApps `
    -installTestApps $installTestApps `
    -previousApps $previousApps `
    -appFolders $appFolders `
    -testFolders $testFolders `
    -doNotRunTests:$doNotRunTests `
    -testResultsFile $testResultsFile `
    -testResultsFormat 'JUnit' `
    -installTestRunner:$installTestRunner `
    -installTestFramework:$installTestFramework `
    -installTestLibraries:$installTestLibraries `
    -installPerformanceToolkit:$installPerformanceToolkit `
    -enableCodeCop:$enableCodeCop `
    -enableAppSourceCop:$enableAppSourceCop `
    -enablePerTenantExtensionCop:$enablePerTenantExtensionCop `
    -enableUICop:$enableUICop `
    -azureDevOps:($environment -eq 'AzureDevOps') `
    -gitLab:($environment -eq 'GitLab') `
    -gitHubActions:($environment -eq 'GitHubActions') `
    -failOn 'error' `
    -AppSourceCopMandatoryAffixes $appSourceCopMandatoryAffixes `
    -AppSourceCopSupportedCountries $appSourceCopSupportedCountries `
    -additionalCountries $additionalCountries `
    -buildArtifactFolder $buildArtifactFolder `
    -CreateRuntimePackages:$CreateRuntimePackages `
    -appBuild $appBuild -appRevision $appRevision `
    -credential $credential `
    -NewBcContainer {
        Param([Hashtable]$parameters)
        New-BcContainer @parameters

        $dllFiles = Get-ChildItem -Path '.\.netpackages\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($dll in $dllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($dll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\dotnet\shared\$($dll.Name)"
        }

        $serverdllFiles = Get-ChildItem -Path '.\.resources\ServerAddIns\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($serverdll in $serverdllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($serverdll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\dotnet\shared\$($serverdll.Name)"
        }

        
        Invoke-ScriptInBcContainer $parameters.ContainerName -scriptblock {
            $progressPreference = 'SilentlyContinue'
            Set-NavServerInstance -ServerInstance 'BC' -Restart
        }
    } `
    -PublishBcContainerApp {
        Param([HashTable] $parameters)
        $parameters.sync = $true 
        $parameters.syncMode = 'ForceSync'
        Publish-BcContainerApp @parameters
    }

Full output of scripts

Processing dependency CKL Software GmbH_Kostenrechnung 365 / Cost Accounting 365_6.1.13.0 (32a8e514-216f-487a-82ea-23709b370af9)
Downloading symbols: CKL Software GmbH_Kostenrechnung 365  Cost Accounting 365_6.1.17.0.app
Url : .../packages?appId=32a8e514-216f-487a-82ea-23709b370af9&versionText=6.1.13.0&tenant=default

Compiling...
.\alc.exe /project:"c:\sources\XXXX" /packagecachepath:"c:\sources\.packages" /out:"c:\sources\.output\XXXXX.Base_1.0.11639.0.app" /BuildBy:"BcContainerHelper,6.0.26-preview1247" /assemblyprobingpaths:"C:\Program Files\dotnet\shared","C:\ProgramData\BcContainerHelper\Extensions\XXXXX\.netPackages\Service"
Microsoft (R) AL Compiler version 12.7.14.31432
Copyright (C) Microsoft Corporation. All rights reserved

Compilation started for project 'XXX.Base' containing '1604' files at '12:37:42.793'.

##[error]./c:/sources/XXXX/src/tableextension/BaseAllocationAssignmentCOA.TableExt.al(1,63): error AL0247: The target Table 'Allocation Assignment COA' for the extension object is not found
##[error]./c:/sources/XXXX/src/tableextension/BaseCostAccJnlLine.TableExt.al(1,57): error AL0247: The target Table 'Cost Acc. Journal Line COA' for the extension object is not found
##[error]./c:/sources/XXXX/src/tableextension/BaseCostAccLedgEntry.TableExt.al(1,59): error AL0247: The target Table 'Cost Acc. Ledger Entry COA' for the extension object is not found

@freddydk
Copy link
Contributor

You would need to include the full script.

@EsatSayinkaplanAccelent
Copy link
Author

Ist mostly based on the old ms repo for devops.

Param(
    [Parameter(Mandatory=$false)]
    [ValidateSet('AzureDevOps','GithubActions','GitLab')]
    [string] $environment = 'AzureDevOps',
    [Parameter(Mandatory=$true)]
    [string] $version,
    [Parameter(Mandatory=$false)]
    [int] $appBuild = 0,
    [Parameter(Mandatory=$false)]
    [int] $appRevision = 0
)

if ($environment -eq "AzureDevOps") {
    $buildArtifactFolder = $ENV:BUILD_ARTIFACTSTAGINGDIRECTORY
}
elseif ($environment -eq "GitHubActions") {
    $buildArtifactFolder = Join-Path $ENV:GITHUB_WORKSPACE "output"
    New-Item $buildArtifactFolder -ItemType Directory | Out-Null
}

# XXX-DOC: Determine base Folder from PSScriptRoot
$scriptFullPath = (Get-Item (Join-Path $PSScriptRoot "...")).FullName
$baseFolder = (Get-Item (Join-Path $scriptFullPath "..\..")).FullName
. (Join-Path $PSScriptRoot "Read-Settings.ps1") -environment $environment -version $version
. (Join-Path $PSScriptRoot "Install-BcContainerHelper.ps1") -bcContainerHelperVersion $bcContainerHelperVersion -genericImageName $genericImageName

Write-Host "Using BaseFolder: $baseFolder"

$authContext = $null
$refreshToken = "$($ENV:BcSaasRefreshToken)"
$environmentName = "$($ENV:EnvironmentName)"
if ($refreshToken -and $environmentName) {
    $authContext = New-BcAuthContext -refreshToken $refreshToken
    if (Get-BcEnvironments -bcAuthContext $authContext | Where-Object { $_.Name -eq $environmentName -and  $_.type -eq "Sandbox" }) {
        Remove-BcEnvironment -bcAuthContext $authContext -environment $environmentName
    }
    $countryCode = $artifact.Split('/')[3]
    New-BcEnvironment -bcAuthContext $authContext -environment $environmentName -countryCode $countrycode -environmentType "Sandbox" | Out-Null
    do {
        Start-Sleep -Seconds 10
        $baseApp = Get-BcPublishedApps -bcAuthContext $authContext -environment $environmentName | Where-Object { $_.Name -eq "Base Application" }
    } while (!($baseApp))
    $baseapp | Out-Host

    $artifact = Get-BCArtifactUrl `
        -country $countryCode `
        -version $baseApp.Version `
        -select Closest
    
    if ($artifact) {
        Write-Host "Using Artifacts: $artifact"
    }
    else {
        throw "No artifacts available"
    }
}

$params = @{}
$insiderSasToken = "$ENV:insiderSasToken"
$licenseFile = "$ENV:licenseFile"
$codeSigncertPfxFile = "$ENV:CodeSignCertPfxFile"
if (!$doNotSignApps -and $codeSigncertPfxFile) {
    if ("$ENV:CodeSignCertPfxPassword" -ne "") {
        $codeSignCertPfxPassword = try { "$ENV:CodeSignCertPfxPassword" | ConvertTo-SecureString } catch { ConvertTo-SecureString -String "$ENV:CodeSignCertPfxPassword" -AsPlainText -Force }
        $params = @{
            "codeSignCertPfxFile" = $codeSignCertPfxFile
            "codeSignCertPfxPassword" = $codeSignCertPfxPassword
        }
    }
    else {
        $codeSignCertPfxPassword = $null
    }
}

$allTestResults = "testresults*.xml"
$testResultsFile = Join-Path $baseFolder "TestResults.xml"
$testResultsFiles = Join-Path $baseFolder $allTestResults
if (Test-Path $testResultsFiles) {
    Remove-Item $testResultsFiles -Force
}

# XXX-DOC: Set credentials for persistent use, adviced to be used with -keepContainers and reUseContainer
$credential = New-Object pscredential 'admin', (ConvertTo-SecureString -String 'admin' -AsPlainText -Force)

# XXX-DOC: Sets the AppFolders to compile
$appFolders = @(Get-ChildItem -Path ".\XXX*" -Directory)

# XXX-DOC: AL Language to use, gets loaded from Artifact
$vsixFile = Get-AlLanguageExtensionFromArtifacts -artifactUrl (Get-BCArtifactUrl -type OnPrem -country "de" -version "23.5." -select Latest) -Verbose
Write-Host "Using vsix-File located in $vsixFile"

Run-AlPipeline @params `
    -pipelinename $pipelineName `
    -containerName $containerName `
    -imageName $imageName `
    -bcAuthContext $authContext `
    -environment $environmentName `
    -artifact $artifact.replace('{INSIDERSASTOKEN}',$insiderSasToken) `
    -memoryLimit $memoryLimit `
    -baseFolder $baseFolder `
    -licenseFile $LicenseFile `
    -installApps $installApps `
    -installTestApps $installTestApps `
    -previousApps $previousApps `
    -appFolders $appFolders `
    -testFolders $testFolders `
    -doNotRunTests:$doNotRunTests `
    -testResultsFile $testResultsFile `
    -testResultsFormat 'JUnit' `
    -installTestRunner:$installTestRunner `
    -installTestFramework:$installTestFramework `
    -installTestLibraries:$installTestLibraries `
    -installPerformanceToolkit:$installPerformanceToolkit `
    -enableCodeCop:$enableCodeCop `
    -enableAppSourceCop:$enableAppSourceCop `
    -enablePerTenantExtensionCop:$enablePerTenantExtensionCop `
    -enableUICop:$enableUICop `
    -azureDevOps:($environment -eq 'AzureDevOps') `
    -gitLab:($environment -eq 'GitLab') `
    -gitHubActions:($environment -eq 'GitHubActions') `
    -failOn 'error' `
    -AppSourceCopMandatoryAffixes $appSourceCopMandatoryAffixes `
    -AppSourceCopSupportedCountries $appSourceCopSupportedCountries `
    -additionalCountries $additionalCountries `
    -buildArtifactFolder $buildArtifactFolder `
    -CreateRuntimePackages:$CreateRuntimePackages `
    -appBuild $appBuild -appRevision $appRevision `
    -credential $credential `
    -NewBcContainer {
        Param([Hashtable]$parameters)
        New-BcContainer @parameters

        $dllFiles = Get-ChildItem -Path '.\.netpackages\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($dll in $dllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($dll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\dotnet\shared\$($dll.Name)"
        }

        $serverdllFiles = Get-ChildItem -Path '.\.resources\ServerAddIns\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($serverdll in $serverdllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($serverdll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\dotnet\shared\$($serverdll.Name)"
        }

        
        Invoke-ScriptInBcContainer $parameters.ContainerName -scriptblock {
            $progressPreference = 'SilentlyContinue'
            Set-NavServerInstance -ServerInstance 'BC' -Restart
        }
    } `
    -PublishBcContainerApp {
        Param([HashTable] $parameters)
        $parameters.sync = $true 
        $parameters.syncMode = 'ForceSync'
        Publish-BcContainerApp @parameters
    }
    
if ($environment -eq 'AzureDevOps') {
    Write-Host "##vso[task.setvariable variable=TestResults]$allTestResults"
}

@EsatSayinkaplanAccelent
Copy link
Author

@freddydk inserted full script

@freddydk
Copy link
Contributor

and the full output?

@EsatSayinkaplanAccelent
Copy link
Author

EsatSayinkaplanAccelent commented Oct 10, 2024

Too long for GitHub so its in the txt file Error Full Output.txt
@freddydk

@EsatSayinkaplanAccelent
Copy link
Author

@freddydk quick additional info from not directly related issue #3604
To confirm that the used package is a valid one I ran Extract-AppFileToFolder with it, to check its integrity.
The extraction worked and in the extracted SymbolReference.json there are definitely the hints to the missing table "Allocation Assignment COA". Hope this helps.

@freddydk
Copy link
Contributor

All the external apps you have - are they runtime packages, full apps or symbol apps?

@EsatSayinkaplanAccelent
Copy link
Author

If possible I used the runtime versions. The first error throwing app "CKL Kostenrechnung 365 / Cost Accounting 365" is an symbol app now after the full app did not work. The error further down with "ECM" in it is from EASY SOFTWARE AG and as runtime version.

@freddydk
Copy link
Contributor

The way it works (by default) is to publish all apps from installedApps to the container and then when building apps - downloading the symbols from the container for the compiler.
This process doesn't work if you use symbol apps.
If you add -doNotPublishApps to your Run-AlPipeline, then it might build - but you won't be able to run tests (but you cannot do that anyway if some of your apps are symbol apps)

@EsatSayinkaplanAccelent
Copy link
Author

EsatSayinkaplanAccelent commented Oct 11, 2024

Alright, as far as I can see the customer does not use tests. I will the try the -doNotPublishApps switch. Is there a way to check if an app is a symbol, full or runtime app?

And ofc thx for your input.

@EsatSayinkaplanAccelent
Copy link
Author

After looking into https://github.com/microsoft/navcontainerhelper/blob/main/AppHandling/Run-AlPipeline.ps1 for the -doNotPublishApps, does this mean I would need to use $useCompilerFolder? And do I need a database for the pipeline to compile against, or does that work with just the compiler?

@freddydk
Copy link
Contributor

You can use compilerfolder if you want - else it will use a filesonly container.

@EsatSayinkaplanAccelent
Copy link
Author

I think I have to use it, else the .NET-DLLs are not there for compilation? Or do I have a thinking error rn?

@EsatSayinkaplanAccelent
Copy link
Author

@freddydk quick update, thanks to your help it worked with a filesonly container. But somehow the script does not get to seem the $buildArtifactFolder = $ENV:BUILD_ARTIFACTSTAGINGDIRECTORY right.

I get the following output after all extensions get compiled:

Compilation ended at '13:05:36.328'.

C:\azure_pipeline\bcnas01_agent1\_work\1\s\.output\XXX.Corp.Support_1.0.11666.0.app successfully created in 88 seconds
XXX.Corp.Support_1.0.11666.0.app copied to C:\azure_pipeline\bcnas01_agent1\_work\1\s\.packages

Compiling apps took 1720 seconds
Copy to build artifacts
  _____                     _          _           _ _     _              _   _  __           _
 / ____|                   | |        | |         (_) |   | |            | | (_)/ _|         | |
 | |     ___  _ __  _   _  | |_ ___   | |__  _   _ _| | __| |   __ _ _ __| |_ _| |_ __ _  ___| |_ ___
 | |    / _ \| '_ \| | | | | __/ _ \  | '_ \| | | | | |/ _` |  / _` | '__| __| |  _/ _` |/ __| __/ __|
 | |___| (_) | |_) | |_| | | || (_) | | |_) | |_| | | | (_| | | (_| | |  | |_| | || (_| | (__| |_\__ \
 \______\___/| .__/ \__, |  \__\___/  |_.__/ \__,_|_|_|\__,_|  \__,_|_|   \__|_|_| \__,_|\___|\__|___/
             | |     __/ |
             |_|    |___/
Files in build artifacts folder:

Copying to Build Artifacts took 0 seconds

Any Idea?

@freddydk
Copy link
Contributor

In your log, the buildartifactsFolder seems to be the same as the basefolder:

2024-10-10T10:52:09.8288772Z BuildArtifactFolder C:\azure_pipeline\bcnas01_agent1_work\1\a

(or maybe it is empty - don't know)

In AL-Go I use this parameter to specify where I want the artifacts.

@freddydk
Copy link
Contributor

I think I have to use it, else the .NET-DLLs are not there for compilation? Or do I have a thinking error rn?

A Files-Only container will have the DLLs as well

@freddydk
Copy link
Contributor

Looks like the build now succeeds, will close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants