Credentials seem to "rot" after a few days

[funkypenguin]

Hi all!

I deploy my Harbor instance using an Ansible playbook, which interacts with the API to setup ldap auth via /configurations and a registry (Docker Hub) via /registries

These all work well after running the playbook - I can login with LDAP auth, replicate to Docker Hub, etc.

After a few days, however, without any apparent changes, the LDAP logins no longer work, and the registry reports as "unhealthy", with a "too many failed logins" error.

When I examine the logs of the ldap server, the authentication request fails, with the correct username used. Presumably therefore, the ldap server, username, etc are unaffected, but the password seems to somehow have been "lost" over time. When I re-run the playbook to apply my config via the API, LDAP login works again.

A similar issue plagues me with the healthy/unhealthy registry. I reapply the registry credentials via the UI, and the registry is healthy again (for a few days).

I've been unable to reproduce this issue on demand - even if I delete all my Harbor pods (I'm using the latest in Kubernetes), I can't make it happen while I'm watching!

Any ideas?

Thanks :)
D

[funkypenguin]

FWIW, I disabled all TLS on my ldap server so that I could capture the failing auth request.. here's what it looks like:

After I re-apply the same ldap config, the authentication works, and the credentials in the tcpdump output (per above) is correct.

I'm wondering whether some automatic process is affecting stored credentials?