[BUG]: VM extension TeamServicesAgentLinux fails with "AttributeError: module 'importlib' has no attribute 'load_source'"

[amasover]

What happened?

I've created a VMSS in Azure using the latest Ubuntu 24.04 LTS image ("server" sku).

ADO Agents fail to start with AttributeError: module 'importlib' has no attribute 'load_source'

I tried SSH'ing into a VM instance, and managed to obtain the full error by running handler.sh manually, and I have attached the output below.

Agents seem to work fine rolling back to 22.04.

22.04 python version: 3.10.12
24.04 python version: 3.12.3

Versions

Agent version 1.25.2.0
OS version Ubuntu 24.04 LTS

Environment type (Please select at least one enviroment where you face this issue)

• Self-Hosted
• Microsoft Hosted
• VMSS Pool
• Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu 24.04 LTS

Version controll system

No response

Relevant log output

/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0# ./Handler.sh
Traceback (most recent call last):
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/WAAgentUtil.py", line 57, in <module>
    waagent = imp.load_source('waagent', agentPath)
              ^^^^^^^^^^^^^^^
AttributeError: module 'importlib' has no attribute 'load_source'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/./AzureRM.py", line 9, in <module>
    import Utils.HandlerUtil as Util
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/HandlerUtil.py", line 71, in <module>
    from .WAAgentUtil import waagent
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/WAAgentUtil.py", line 64, in <module>
    waagent = imp.load_source('waagent', agentPath)
              ^^^^^^^^^^^^^^^
AttributeError: module 'importlib' has no attribute 'load_source'

[tkasparek]

We've rolled back extension to previous version (1.23), I am looking into the root cause.

[Paddy420]

Is the rollback still applied? I am still getting the same issue with my Ubuntu 24.04 LTS VMSS created 3 days ago which apparently uses the agent version 1.25.2.0.

The handler for VM extension type 'Microsoft.VisualStudio.Services.TeamServicesAgentLinux' has reported terminal failure for VM extension 'Microsoft.Azure.DevOps.Pipelines.Agent' with error message: '[ExtensionOperationError] Non-zero exit code: 1, /var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Handler.sh [stdout] [stderr] Traceback (most recent call last): File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/WAAgentUtil.py", line 57, in <module> waagent = imp.load_source('waagent', agentPath) ^^^^^^^^^^^^^^^ AttributeError: module 'importlib' has no attribute 'load_source' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/./AzureRM.py", line 9, in <module> import Utils.HandlerUtil as Util File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/HandlerUtil.py", line 71, in <module> from .WAAgentUtil import waagent File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/WAAgentUtil.py", line 64, in <module> waagent = imp.load_source('waagent', agentPath) ^^^^^^^^^^^^^^^ AttributeError: module 'importlib' has no attribute 'load_source' '. 'Install handler failed for the extension. More information on troubleshooting is available at https://aka.ms/vmextensionlinuxtroubleshoot'

[andrea-migliaccio]

Same here. A VMSS based on an Image I've just created from the same Ubuntu 24.04 template.

There were errors in your deployment. Error code: DeploymentFailed.
##[error]At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.
##[error]Details:
##[error]VMExtensionHandlerNonTransientError: The handler for VM extension type 'Microsoft.VisualStudio.Services.TeamServicesAgentLinux' has reported terminal failure for VM extension 'Microsoft.Azure.DevOps.Pipelines.Agent' with error message: '[ExtensionOperationError] Non-zero exit code: 1, /var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Handler.sh
[stdout]


[stderr]
Traceback (most recent call last):
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/WAAgentUtil.py", line 57, in <module>
    waagent = imp.load_source('waagent', agentPath)
              ^^^^^^^^^^^^^^^
AttributeError: module 'importlib' has no attribute 'load_source'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/./AzureRM.py", line 9, in <module>
    import Utils.HandlerUtil as Util
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/HandlerUtil.py", line 71, in <module>
    from .WAAgentUtil import waagent
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.25.2.0/Utils/WAAgentUtil.py", line 64, in <module>
    waagent = imp.load_source('waagent', agentPath)
              ^^^^^^^^^^^^^^^
AttributeError: module 'importlib' has no attribute 'load_source'
'.
    
'Install handler failed for the extension. More information on troubleshooting is available at https://aka.ms/vmextensionlinuxtroubleshoot'

[Luiz-Monad]

I'm trying to create a new VMSS using Ubuntu 24.04 and had exactly the same problem.

[Paddy420]

A workaround until this is fixed can be using a VMSS with Ubuntu 22.04. It's important to delete the agent pool and 24.04 VMSS and set up everything from scratch. With 22.04 it's working fine for me.

[Luiz-Monad]

There's something wrong that's still depending on Python 2 in the AzureRM.py. It seems to be some regression from this fix #3156

I added a script to reinstall python2 and then it worked

      extensionProfile: {
        extensions: [
          {
            name: 'FixPython2'
            properties: {
              autoUpgradeMinorVersion: true
              publisher: 'Microsoft.Azure.Extensions'
              type: 'CustomScript'
              typeHandlerVersion: '2.1'
              settings: {
                skipDos2Unix: false
                commandToExecute: 'apt-get remove -y python-is-python3 && rm /usr/bin/python3 && apt install -y build-essential checkinstall libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev && wget https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tgz && tar -xvf Python-2.7.18.tgz && cd Python-2.7.18 && ./configure --enable-optimizations && make build_all && make install && ln -s /usr/local/bin/python2.7 /usr/bin/python && ln -s /usr/local/bin/python2.7 /usr/bin/python2 && ln -s /usr/local/bin/python2.7 /usr/bin/python2.7'
              }
            }
          }
          {
            name: 'Microsoft.Azure.DevOps.Pipelines.Agent'
            properties: {
              autoUpgradeMinorVersion: true
              publisher: 'Microsoft.VisualStudio.Services'
              type: 'TeamServicesAgentLinux'
              typeHandlerVersion: '1.25'
              settings: {
                isPipelinesAgent: true
                agentFolder: '/agent'
                agentDownloadUrl: agentDownloadUrl
                enableScriptDownloadUrl: enableScriptDownloadUrl
              }
              provisionAfterExtensions: [
                'FixPython2'
              ]
            }
          }
        ]
      }