Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Triggering builds randomly fails with NullPointerException #261

Open
MichalHaluza opened this issue Jun 15, 2023 · 0 comments
Open

Triggering builds randomly fails with NullPointerException #261

MichalHaluza opened this issue Jun 15, 2023 · 0 comments
Labels

Comments

@MichalHaluza
Copy link

Jenkins and plugins versions report

Environment
Jenkins: 2.387.1
OS: Linux - 4.18.0-372.53.1.el8_6.x86_64
Java: 11.0.18 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
---
ace-editor:1.1
apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5
authentication-tokens:1.4
authorize-project:1.4.0
bootstrap4-api:4.6.0-5
bootstrap5-api:5.2.1-3
bouncycastle-api:2.27
branch-api:2.1071.v1a_188a_562481
caffeine-api:3.1.6-115.vb_8b_b_328e59d8
checks-api:1.8.1
cloud-stats:267.v577e3742c282
cloudbees-disk-usage-simple:178.v1a_4d2f6359a_8
cloudbees-folder:6.815.v0dd5a_cb_40e0e
command-launcher:90.v669d7ccb_7c31
commons-lang3-api:3.12.0-36.vd97de6465d5b_
commons-text-api:1.10.0-36.vc008c8fcda_7b_
config-file-provider:3.11.1
configuration-as-code:1625.v27444588cc3d
credentials:1224.vc23ca_a_9a_2cb_0
credentials-binding:604.vb_64480b_c56ca_
data-tables-api:1.12.1-4
dependency-check-jenkins-plugin:5.4.0
display-url-api:2.3.7
durable-task:504.vb10d1ae5ba2f
echarts-api:5.4.0-1
font-awesome-api:6.2.1-1
git:5.0.1
git-client:4.2.0
git-server:99.va_0826a_b_cdfa_d
gitlab-plugin:1.7.12
handlebars:3.0.8
htmlpublisher:1.31
instance-identity:142.v04572ca_5b_265
ionicons-api:45.vf54fca_5d2154
jackson2-api:2.15.0-334.v317a_165f9b_7c
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:226.v71211feb_e7e9
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-8
jaxb:2.3.8-1
jdk-tool:63.v62d2fd4b_4793
jersey2-api:2.39.1-1
jms-messaging:1.1.27
job-dsl:1.82
jquery3-api:3.6.1-2
jsch:0.1.55.61.va_e9ee26616e7
junit:1198.ve38db_d1b_c975
kerberos-sso:1.11
kubernetes:3896.v19b_160fd9589
kubernetes-client-api:6.4.1-215.v2ed17097a_8e9
kubernetes-credentials:0.10.0
ldap:671.v2a_9192a_7419d
lockable-resources:1131.vb_7c3d377e723
mailer:448.v5b_97805e3767
matrix-auth:3.1.6
matrix-project:789.v57a_725b_63c79
metrics:4.2.13-420.vea_2f17932dd6
mina-sshd-api-common:2.9.2-62.v199162f0a_2f8
mina-sshd-api-core:2.9.2-62.v199162f0a_2f8
momentjs:1.1.1
okhttp-api:4.10.0-132.v7a_7b_91cef39c
openstack-cloud:2.63
parameterized-scheduler:1.2
parameterized-trigger:2.45
pipeline-build-step:487.va_823138eee8b_
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:629.vb_5627b_ee2104
pipeline-input-step:466.v6d0a_5df34f81
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2121.vd87fb_6536d1e
pipeline-model-definition:2.2121.vd87fb_6536d1e
pipeline-model-extensions:2.2121.vd87fb_6536d1e
pipeline-rest-api:2.31
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2121.vd87fb_6536d1e
pipeline-stage-view:2.31
pipeline-utility-steps:2.15.1
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:2.20.0
popper-api:1.16.1-3
popper2-api:2.11.6-2
prometheus:2.1.1
rebuild:1.34
redhat-ci-plugin:1.5.16
resource-disposer:0.21
role-strategy:587.588.v850a_20a_30162
saml:4.385.v4dea_91565e9d
scm-api:667.v8b_6e07cdc7f2
script-security:1244.ve463715a_f89c
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
sonar:2.15
splunk-devops:1.10.1
splunk-devops-extend:1.10.1
ssh-credentials:305.v8f4381501156
ssh-slaves:2.877.v365f5eb_a_b_eec
sshd:3.275.v9e17c10f2571
structs:324.va_f5d6774f3a_d
swarm:3.39
token-macro:359.vb_cde11682e0c
trilead-api:2.84.v72119de229b_7
update-sites-manager:2.0.0
variant:59.vf075fe829ccb
workflow-aggregator:596.v8c21c963d92d
workflow-api:1208.v0cc7c6e0da_9e
workflow-basic-steps:1010.vf7a_b_98e847c1
workflow-cps:3641.vf58904a_b_b_5d8
workflow-durable-task-step:1234.v019404b_3832a
workflow-job:1292.v27d8cc3e2602
workflow-multibranch:733.v109046189126
workflow-scm-step:408.v7d5b_135a_b_d49
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:839.v35e2736cfd5c
ws-cleanup:0.44

What Operating System are you using (both controller, and any agents involved in the problem)?

RHEL 8 (CSB supported instance of Jenkins)

Reproduction steps

Relevant JCasC configuration:

  jmsProviders:
    configs:
    - activeMq:
        name: "Red Hat UMB"
        broker: "ssl://umb.api.redhat.com:61616"
        authenticationMethod:
          sslCertificate:
            keystore: ***
            keypwd: ***
            truststore: ***
            trustpwd: ***
        topicProvider: "umb"
        useQueues: true

Relevant pipeline trigger:

    triggers {
        ciBuildTrigger {
            providers {
                providerDataEnvelope {
                    providerData {
                        activeMQSubscriber {
                            name("Red Hat UMB")
                            overrides {
                                topic("Consumer.rheldst-jenkins-csb.jenkins-errata-dispatcher.VirtualTopic.eng.errata.activity.status")
                            }
                            selector("errata_status='SHIPPED_LIVE' AND product='RHEL'")
                            variable("CI_MESSAGE")
                            timeout(60)
                        }
                    }
                }
            }
        }
    }

Expected Results

Each consumed message results in a successfully triggered / scheduled job.

Actual Results

Triggering jobs occasionally fails with

java.lang.NullPointerException
	at com.redhat.jenkins.plugins.ci.CIShouldScheduleQueueAction.shouldSchedule(CIShouldScheduleQueueAction.java:33)
	at hudson.model.Queue.scheduleInternal(Queue.java:606)
	at hudson.model.Queue.schedule2(Queue.java:576)
	at jenkins.model.ParameterizedJobMixIn.scheduleBuild2(ParameterizedJobMixIn.java:158)
	at jenkins.model.ParameterizedJobMixIn.scheduleBuild2(ParameterizedJobMixIn.java:123)
	at com.redhat.jenkins.plugins.ci.CIBuildTrigger.scheduleBuild(CIBuildTrigger.java:544)
	at com.redhat.jenkins.plugins.ci.messaging.JMSMessagingWorker.trigger(JMSMessagingWorker.java:76)
	at com.redhat.jenkins.plugins.ci.messaging.ActiveMqMessagingWorker.process(ActiveMqMessagingWorker.java:280)
	at com.redhat.jenkins.plugins.ci.messaging.ActiveMqMessagingWorker.receive(ActiveMqMessagingWorker.java:311)
	at com.redhat.jenkins.plugins.ci.threads.CITriggerThread.run(CITriggerThread.java:90) 

e.g. on 2023-06-13, our instance received 9 messages, but only 7 jobs were triggered. The other 2 failed with the above stack trace.

Looking into company-wide Splunk logs between 2023-05-01 and 2023-06-15, this has happened

  • 117x in instance1
  • 222x on instance2
  • 1415x on instance3
  • 112x on instance4

The total number of jobs which have been triggered properly is currently unknown to me, but it is high enough so the instances appear to work almost normally. Each instance is a separate Jenkins master owned and configured by a different team.

Anything else?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant