[Bug] eksctl since 1.181.0 does not respect SDK configuration using global endpoint configuration

[dfangl]

What were you trying to accomplish?

When trying to use eksctl with the environment variable AWS_ENDPOINT_URL, or an endpoint specified in the shared aws config (using endpoint_url), as described here, eksctl ignores the configuration and still contacts the default endpoint.

This worked until including version 1.180.0, and broke with version 1.181.0.
I am aware that there are the environment variables AWS_CLOUDFORMATION_ENDPOINT and so on, but using the official endpoint configuration is more convenient in many cases (like overriding all endpoint with the same address).

The commit breaking this behavior is e64db43 .

It seems the change in endpoint strategy, using the options override to provide the baseEndpoint disables this behavior in the go sdk v2 somehow.
Perhaps, we should avoid setting the options at all, if no endpoint is overridden on the eksctl side.

The AWS documentation also marks the go sdk v2 as compatible: https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html#ss-endpoints-sdk-compat

What happened?

eksctl ignores the endpoint override using AWS_ENDPOINT_URL and contacts AWS regularly.
For illustration, I did not provide valid AWS credentials, so we can see the issue immediately.

AWS_ENDPOINT_URL=http://localhost:4566 ./eksctl create cluster -v 4
2024-09-12 17:15:42 [▶]  Setting credentials expiry window to 30 minutes
Error: checking AWS STS access – cannot get role ARN for current session: operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: d3f6eacb-f8b5-45cc-abd4-0d37449d99f8, api error InvalidClientTokenId: The security token included in the request is invalid.

How to reproduce it?

1. Set AWS_ENDPOINT_URL in your environment, or the endpoint_url parameter in your active profile in your shared aws config.
2. Try to create a cluster using ./eksctl create cluster -v 4
3. Check what endpoint is hit (ideally avoid using actual AWS credentials, if you do not want to accidentally create a cluster)

Logs

Commit e64db43

AWS_ENDPOINT_URL=http://localhost:4566 ./eksctl create cluster -v 4
2024-09-12 17:15:42 [▶]  Setting credentials expiry window to 30 minutes
Error: checking AWS STS access – cannot get role ARN for current session: operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: d3f6eacb-f8b5-45cc-abd4-0d37449d99f8, api error InvalidClientTokenId: The security token included in the request is invalid.

Commit bb87f30 (the one before the breaking commit)

AWS_ENDPOINT_URL=http://localhost:4566 ./eksctl create cluster -v 4
2024-09-12 17:17:06 [▶]  Setting credentials expiry window to 30 minutes
2024-09-12 17:17:06 [▶]  role ARN for the current session is "arn:aws:iam::000000000000:root"
2024-09-12 17:17:06 [ℹ]  eksctl version 0.181.0-dev+bb87f300b.2024-09-12T17:16:49Z
2024-09-12 17:17:06 [ℹ]  using region us-east-1
2024-09-12 17:17:06 [▶]  determining availability zones
...

Anything else we need to know?

OS: Arch Linux
eksctl version: Self compiled from breaking commits, used downloaded ones as well for the initial check.
Credentials: named profile

I am happy to give this a try myself, if this is something to be fixed.

Versions

Newest version tested:

$ ./eksctl info
eksctl version: 0.191.0-dev+ceae16253.2024-09-12T17:21:15Z
kubectl version: v1.30.3
OS: linux

First breaking version:

$ ./eksctl info
eksctl version: 0.181.0-dev+e64db43bd.2024-09-12T17:23:49Z
kubectl version: v1.30.3
OS: linux

Last working version:

$ ./eksctl info
eksctl version: 0.181.0-dev+bb87f300b.2024-09-12T17:24:55Z
kubectl version: v1.30.3
OS: linux

[github-actions]

Hello dfangl 👋 Thank you for opening an issue in eksctl project. The team will review the issue and aim to respond within 1-5 business days. Meanwhile, please read about the Contribution and Code of Conduct guidelines here. You can find out more information about eksctl on our website