Fix CI to resolve current fastlane dependencies (bump bundler to 2.4.22, downgrade Ruby to 2.7.8)

[janbrasna]

Fixes #1234

Newly released v2.219.0 in #1229 introduces some new dependency woes that break CI here.

This is a PoC to fix the env to enable #1229 and newer to land…

❌ Bundler 2.3.21:

OK overlooked that Netlify bootstraps Ruby 3.1.1 and is happy to build:

Using Ruby version 3.1.1

9:19:49 PM: Required ruby-3.1.1 is not installed.
9:19:49 PM: Attempting Ruby version 3.1.1, read from .ruby-version file
9:19:50 PM: Required ruby-3.1.1 is not installed - installing.
9:19:50 PM: Requirements installation successful.
9:19:59 PM: ruby-3.1.1 - #gemset created /opt/buildhome/.rvm/gems/ruby-3.1.1@global
9:19:59 PM: ruby-3.1.1 - #importing gemset /opt/buildhome/.rvm/gemsets/
9:19:59 PM: ruby-3.1.1 - #gemset created /opt/buildhome/.rvm/gems/ruby-3.1.1
9:19:59 PM: ruby-3.1.1 - #importing gemsetfile /opt/buildhome/.rvm/gemsets/default.gems
9:20:00 PM: ruby-3.1.1 - #generating default wrappers........
9:20:00 PM: Using /opt/buildhome/.rvm/gems/ruby-3.1.1
9:20:00 PM: Using Ruby version 3.1.1
9:20:00 PM: Using Bundler version 2.3.21 from Gemfile.lock
9:20:01 PM: Successfully installed bundler-2.3.21

while the failed CircleCI gets stuck on conflicting requirements:

fastlane 2.219.0 which depends on http-cookie (~> 1.0.5), which depends on domain_name (~> 0.5), which depends on Ruby (>= 2.7) — Current Ruby version: Ruby (= 2.6.6)

Fetching https://github.com/fastlane/fastlane
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies.................
Bundler found conflicting requirements for the Ruby version:
  In Gemfile:
    fastlane was resolved to 2.219.0, which depends on
      http-cookie (~> 1.0.5) was resolved to 1.0.5, which depends on
        domain_name (~> 0.5) was resolved to 0.6.20240107, which depends on
          Ruby (>= 2.7)

  Current Ruby version:
    Ruby (= 2.6.6)

❌ Bundler 2.3.22:

CircleCI is happy!

fastlane.tools finished successfully 🎉 All fastlane code samples (from 245 files) work as expected ✅

Fetching https://github.com/fastlane/fastlane
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies............
Fetching rake 13.1.0
Installing rake 13.1.0
Using bundler 2.3.22
Fetching colored 1.2
Fetching highline 2.0.3
Fetching faraday-patron 1.0.0
Fetching faraday-net_http 1.0.1
Fetching faraday-net_http_persistent 1.2.0
Fetching faraday-em_http 1.0.0
Fetching faraday-retry 1.0.3
Fetching ruby2_keywords 0.0.5
Fetching base64 0.2.0
Fetching rexml 3.2.6
Fetching public_suffix 5.0.4
Fetching ast 2.4.2
Fetching atomos 0.1.3
Fetching aws-eventstream 1.3.0
Fetching aws-partitions 1.894.0 (was 1.877.0)
Fetching jmespath 1.6.2
Fetching babosa 1.0.4
Fetching faraday-excon 1.1.0
Installing colored 1.2
Fetching colored2 3.1.2
Fetching open4 1.3.4
Installing faraday-patron 1.0.0
Fetching faraday-rack 1.0.0
Installing faraday-retry 1.0.3
Installing faraday-net_http 1.0.1
Fetching unicode-display_width 1.8.0
Installing faraday-net_http_persistent 1.2.0
Fetching declarative 0.0.20
Fetching dotenv 2.8.1
Fetching nkf 0.2.0
Installing faraday-em_http 1.0.0
Fetching emoji_regex 3.2.3
Fetching nap 1.1.0
Installing ast 2.4.2
Fetching digest-crc 0.6.5
Fetching multipart-post 2.4.0 (was 2.3.0)
Fetching claide 1.1.0
Installing aws-eventstream 1.3.0
Fetching faraday-em_synchrony 1.0.0
Fetching no_proxy_fix 0.1.2
Fetching unf_ext 0.0.9.1
Installing base64 0.2.0
Installing ruby2_keywords 0.0.5
Installing babosa 1.0.4
Installing atomos 0.1.3
Installing faraday-rack 1.0.0
Fetching rchardet 1.8.0
Installing unicode-display_width 1.8.0
Installing colored2 3.1.2
Installing open4 1.3.4
Fetching faraday-httpclient 1.0.1
Installing faraday-excon 1.1.0
Installing no_proxy_fix 0.1.2
Installing jmespath 1.6.2
Installing emoji_regex 3.2.3
Fetching artifactory 3.0.15
Installing aws-partitions 1.894.0 (was 1.877.0)
Installing dotenv 2.8.1
Installing faraday-em_synchrony 1.0.0
Installing declarative 0.0.20
Installing claide 1.1.0
Installing multipart-post 2.4.0 (was 2.3.0)
Installing public_suffix 5.0.4
Installing rexml 3.2.6
Installing faraday-httpclient 1.0.1
Installing digest-crc 0.6.5 with native extensions
Installing artifactory 3.0.15
Installing nap 1.1.0
Fetching excon 0.109.0
Installing highline 2.0.3
Fetching fastimage 2.3.0
Installing nkf 0.2.0 with native extensions
Fetching gh_inspector 1.1.3
Installing rchardet 1.8.0
Fetching multi_json 1.15.0
Installing gh_inspector 1.1.3
Fetching os 1.1.4
Installing fastimage 2.3.0
Fetching httpclient 2.8.3
Fetching mini_mime 1.1.5
Fetching trailblazer-option 0.1.2
Installing multi_json 1.15.0
Installing os 1.1.4
Installing trailblazer-option 0.1.2
Installing mini_mime 1.1.5
Fetching uber 0.1.0
Fetching retriable 3.1.2
Installing uber 0.1.0
Fetching google-cloud-errors 1.3.1
Installing excon 0.109.0
Fetching json 2.7.1
Fetching mini_magick 4.12.0
Fetching naturally 2.2.1
Installing unf_ext 0.0.9.1 with native extensions
Installing retriable 3.1.2
Installing google-cloud-errors 1.3.1
Fetching optparse 0.4.0
Fetching plist 3.7.1
Installing naturally 2.2.1
Installing mini_magick 4.12.0
Fetching rubyzip 2.3.2
Installing json 2.7.1 with native extensions
Installing plist 3.7.1
Installing optparse 0.4.0
Installing rubyzip 2.3.2
Fetching security 0.1.5 (was 0.1.3)
Fetching terminal-notifier 2.0.0
Installing httpclient 2.8.3
Installing security 0.1.5 (was 0.1.3)
Fetching tty-screen 0.8.2
Fetching tty-cursor 0.7.1
Installing tty-screen 0.8.2
Fetching word_wrap 1.0.0
Fetching nanaimo 0.3.0
Installing tty-cursor 0.7.1
Fetching rouge 2.0.7
Installing word_wrap 1.0.0
Fetching parallel 1.24.0
Fetching powerpack 0.1.3
Fetching rainbow 2.2.2
Installing nanaimo 0.3.0
Fetching ruby-progressbar 1.13.0
Installing parallel 1.24.0
Installing powerpack 0.1.3
Installing rainbow 2.2.2 with native extensions
Installing ruby-progressbar 1.13.0
Fetching slack-notifier 2.4.0
Fetching xcresult 0.2.1
Fetching jwt 2.8.0 (was 2.7.1)
Installing slack-notifier 2.4.0
Fetching parser 2.7.2.0
Installing xcresult 0.2.1
Fetching aws-sigv4 1.8.0
Fetching terminal-table 3.0.2
Installing jwt 2.8.0 (was 2.7.1)
Fetching cork 0.3.0
Fetching faraday-multipart 1.0.4
Installing aws-sigv4 1.8.0
Installing terminal-table 3.0.2
Installing cork 0.3.0
Installing rouge 2.0.7
Installing faraday-multipart 1.0.4
Fetching addressable 2.8.6
Fetching tty-spinner 0.9.3
Installing addressable 2.8.6
Installing tty-spinner 0.9.3
Fetching claide-plugins 0.9.2
Fetching representable 3.2.0
Installing claide-plugins 0.9.2
Installing terminal-notifier 2.0.0
Fetching faraday 1.10.3
Installing parser 2.7.2.0
Installing representable 3.2.0
Installing faraday 1.10.3
Fetching aws-sdk-core 3.191.3 (was 3.190.1)
Installing aws-sdk-core 3.191.3 (was 3.190.1)
Fetching git 1.19.1 (was 1.19.0)
Installing git 1.19.1 (was 1.19.0)
Fetching commander 4.6.0
Installing commander 4.6.0
Fetching kramdown 2.4.0
Installing kramdown 2.4.0
Fetching signet 0.18.0
Fetching google-cloud-env 1.6.0
Fetching faraday-http-cache 2.5.1 (was 2.5.0)
Fetching faraday_middleware 1.2.0
Fetching sawyer 0.9.2
Installing google-cloud-env 1.6.0
Installing faraday-http-cache 2.5.1 (was 2.5.0)
Installing sawyer 0.9.2
Installing signet 0.18.0
Installing faraday_middleware 1.2.0
Fetching google-cloud-core 1.6.1
Installing google-cloud-core 1.6.1
Fetching octokit 4.25.1
Fetching googleauth 1.8.1
Installing octokit 4.25.1
Installing googleauth 1.8.1
Fetching google-apis-core 0.11.3 (was 0.11.2)
Installing google-apis-core 0.11.3 (was 0.11.2)
Fetching google-apis-androidpublisher_v3 0.54.0
Fetching google-apis-playcustomapp_v1 0.13.0
Fetching google-apis-storage_v1 0.29.0
Fetching google-apis-iamcredentials_v1 0.17.0
Installing google-apis-playcustomapp_v1 0.13.0
Installing google-apis-iamcredentials_v1 0.17.0
Installing google-apis-storage_v1 0.29.0
Installing google-apis-androidpublisher_v3 0.54.0
Fetching aws-sdk-kms 1.77.0 (was 1.75.0)
Fetching xcpretty 0.3.0
Installing aws-sdk-kms 1.77.0 (was 1.75.0)
Installing xcpretty 0.3.0
Fetching aws-sdk-s3 1.143.0 (was 1.142.0)
Installing aws-sdk-s3 1.143.0 (was 1.142.0)
Fetching xcpretty-travis-formatter 1.0.1
Installing xcpretty-travis-formatter 1.0.1
Fetching kramdown-parser-gfm 1.1.0
Installing kramdown-parser-gfm 1.1.0
Fetching danger 8.6.1
Installing danger 8.6.1
Fetching unf 0.1.4
Installing unf 0.1.4
Fetching domain_name 0.5.20190701 (was 0.6.20231109)
Installing domain_name 0.5.20190701 (was 0.6.20231109)
Fetching http-cookie 1.0.5
Installing http-cookie 1.0.5
Fetching faraday-cookie_jar 0.0.7
Installing faraday-cookie_jar 0.0.7
Fetching CFPropertyList 3.0.7 (was 3.0.6)
Installing CFPropertyList 3.0.7 (was 3.0.6)
Fetching simctl 1.6.10
Fetching xcodeproj 1.24.0 (was 1.23.0)
Installing xcodeproj 1.24.0 (was 1.23.0)
Installing simctl 1.6.10
Fetching rubocop 0.49.1
Installing rubocop 0.49.1
Fetching google-cloud-storage 1.45.0
Installing google-cloud-storage 1.45.0
Using fastlane 2.219.0 from https://github.com/fastlane/fastlane (at master@cdc2471)
Fetching xcov 1.8.1
Installing xcov 1.8.1
Bundle updated!

But Netlify fails now;D

Now I get what #1234 means. Sorry;)

✅ Ruby 2.7.8 from Docker/CircleCI to Netlify too + Bundler 2.4.22:

Finally success 🚀

CircleCI:

Netlify:

[janbrasna]

Ha! Got it working, bringing the envs as close as possible:

• fastlane/fastlane uses fastlanetools/ci:0.3.0 but 0.4.0 should be the same~ish so kept that
• tried its bundler 2.3.22 instead of 2.3.21 that passes for fastlane/fastlane ubuntu tests that helped get the docker/circle ruby+deps working for v2.219.0 here! ✅
• but that broke netlify; tried several versions to no avail… ❌
• following the max version for ruby 2.6.6 bumped bundler to 2.4.22 (no impact here, but realized maybe the netlify's ruby-3.1.1 env has some issues installing that for whatever reason, so why run ruby 3 really?…)
• but only downgrading .ruby-version to the same 2.6.6 as used in CircleCI runs fixed the Netlify builds for me ✅ ✅ 🏁

[rogerluan]

Hey @janbrasna 👋

Thanks for this PR! The 3 aforementioned PRs have been closed in favor of the one that released docs changes for v.220.0 released earlier this week (#1243), which has already been merged, actually 😅

Could you update this PR with the latest changes from master? 😃 I believe this will make it have the smallest change set and then we can see it passing CI 😊

I really appreciate your contributions!

[rogerluan]

Isn't there any combination of dependencies that keep the Ruby version at or above 3.1.1? This was last bumped 2 years ago and it sounds like a regression to revert back to a unsupported Ruby version 😬😬

[janbrasna]

I know, but that's what works fastlane/fastlane — the CircleCI and images use that already. It seems that Netlify can't work with non-default bundler otherwise so downgrading Netlify to the same env as CircleCI, not introducing nothing "new", see my elaboration: #1234 (comment) +here #1237 (comment) for the steps and reasoning. (Not that I particularly like it myself, but w/e 🤷…)

[rogerluan]

Got it 😅

Hey can you DM me on https://twitter.com/rogerluan_ or email me? (Email is in my GH profile) — I wanna reach out to you about something but couldn't find a contact :)!

[janbrasna]

Yea I can imagine better circumstances too;) but that is probably just tentatively until upstream tooling in fastlane/fastlane runs Ruby 3.3 and all the images consumed here are updated accordingly — but here we can only consume what't published upstream so…;) 🤷

[janbrasna]

@rogerluan That's intentional;) Look around your filtered folders for my username, or just gmail me at that…

[janbrasna]

BTW the .ruby-version is not really needed for CircleCI or local work, quite the contrary, so if someone feels fancy enough to just print the file during a CI run (specifically, a PR targeted preview CI only, not elsewhere), after checkout, before configuring the runtime env, this could be better fixed somewhere in the tooling to shim it only for Netlify previews. But I'm not a hotshot like that so it's just an idea for someone more comfortable with breaking and fixing things properly;D

This sucks because it basically ask for a zombie version for local work (pretty much a macOS default Ruby version FWIW;D) even thou the docs project builds happily at more current 3.x (and that should be used in local dev anyways…), I just don't have the imagination to fix the two disjoining pipelines without messing up the local workspace TBH:/

[janbrasna]

Just for the laughs — if you omit the ruby version completely, Netlify running the same image (note the sha) will pick different ruby versions for fastlane/docs ❌ and janbrasna/docs ✅: #1244 (comment) 🤷

#givinup

[janbrasna]

So to match what's now running in the test docker, I've tried the highest security update version for Ruby 2.x ever — the 2.7.8 — and that seems to work in both workflows. Hopefully that's the least lame we can do now.

[janbrasna]

@rogerluan Yup now that 2.220.0 is merged @KrauseFx the CI finally gave in and this is needed;D I'll remove the content changes and keep only the env fixes, and rebase that to check if the 2.220.0 brings more incompatible deps or satisfies the same as here.

[janbrasna]

@rogerluan CI passes. The PR is now cleaned up only to the pipeline changes, and rebased on fresh 2.220.0 — you can see the docs built https://deploy-preview-1237--fastlane-docs-preview.netlify.app/actions/git_add/#git_add (2.220.0 says "8 examples" while previous versions would show "7 examples" e.g.)

[rogerluan]

Despite being a significant downgrade, given the circumstances I think this is a compromise we have to take 👍

Thanks for the in-depth investigation @janbrasna 💪

Wanna take a second look @lacostej ? 😊

[janbrasna]

FYI this is all somewhat blocked on fastlane/fastlane#21685 (comment) — i. e. what version of the fastlane/docker container we can pull, and what's in it. So if you have any power over that, 💪 … ;)

[revolter]

@rogerluan, so, should we wait for the upstream removal of Ruby 2.6, or merge this as it is?

If we merge it, then we may create an issue as a reminder to bump it back to 3.* in the future.

[janbrasna]

@revolter @rogerluan This might not be needed right now, as the google-apis|cloud-* gem downgrades in #1213 are enough to keep the bundler happy, and it no longer trips over it in Netlify — I've opened #1248 to quickly check it's all green now in and against master.

I'll keep this draft around in case it's needed again, and will also elaborate a bit more about the details "why" in the first place, but it's not a priority ATM until landing some newer implicit fastlane dependencies from upstream break the elderly env again.