NEW: setup_python.py

.github/workflows/setup_python.yml

@@ -0,0 +1,38 @@
+name: setup-python
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.12", "3.11", "3.10", "3.9"]
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: 'Set up Python ${{ matrix.python-version }}'
+      uses: actions/setup-python@v5
+      # https://github.com/marketplace/actions/setup-python
+      with:
+        python-version: '${{ matrix.python-version }}'
+
+    - name: 'Just call --help with Python v${{ matrix.python-version }}'
+      run: |
+        python3 manageprojects/setup_python.py --help
+
+    - name: 'Call setup python script with Python v${{ matrix.python-version }}'
+      env:
+        PYTHONUNBUFFERED: 1
+        PYTHONWARNINGS: always
+      run: |
+        sudo python3 manageprojects/setup_python.py -vv
+
+    - name: 'Test the installed interpreter'
+      run: |
+        $(python3 manageprojects/setup_python.py) -VV

README.md

@@ -15,7 +15,8 @@ Besides this, `manageprojects` also includes other generic helper for Python pac
 
  * `publish_package()` - Build and upload a new release to PyPi, but with many pre-checks.
  * `format-file` - Format/Check a Python source file with Darker & Co., useful as IDE action.
- * `install_python.py` - Install Python interpreter, if needed, from official Python FTP server, verified.
+ * `install_python.py` - [Install Python interpreter, if needed, from official Python FTP server, verified.](https://github.com/jedie/manageprojects/blob/main/docs/install_python.md)
+ * `setup_python.py` - [Download and setup redistributable Python Interpreter, if needed.](https://github.com/jedie/manageprojects/blob/main/docs/setup_python.md)
 
 Read below the `Helper` section.
 
@@ -346,7 +347,8 @@ See also git tags: https://github.com/jedie/manageprojects/tags
 
 [comment]: <> (✂✂✂ auto generated history start ✂✂✂)
 
-* [**dev**](https://github.com/jedie/manageprojects/compare/v0.18.0...main)
+* [v0.19.0](https://github.com/jedie/manageprojects/compare/v0.18.0...v0.19.0)
+  * 2024-09-15 - NEW: setup_python.py
   * 2024-09-15 - Update requirements
 * [v0.18.0](https://github.com/jedie/manageprojects/compare/v0.17.1...v0.18.0)
   * 2024-08-29 - Fix wrong "module" in publish call :(

docs/install_python.md

@@ -54,7 +54,7 @@ class IncludeInstallPythonTestCase(IncludeInstallPythonBaseTestCase):
     # Set the path where the `install_python.py` should be copied to:
     DESTINATION_PATH = Path(your_package.__file__).parent) / 'install_python.py'
 
-    # Just call the method in a test, it will pass, if the file is up2date:
+    # The test will pass, if the file is up2date, if it's update the script!
     def test_install_python_is_up2date(self):
         self.auto_update_install_python()
 ```

docs/setup_python.md

@@ -0,0 +1,135 @@
+# Boot Redistributable Python
+
+This is a standalone script (no dependencies) to download and setup
+https://github.com/indygreg/python-build-standalone/ redistributable Python interpreter.
+But only if it's needed!
+
+Minimal version to used this script is Python v3.9.
+
+The downloaded archive will be verified with the hash checksum.
+
+The download will be only done, if the system Python is not the same major version as requested
+and if the local Python is not up-to-date.
+
+## CLI
+
+The CLI interface looks like e.g.:
+
+```shell
+$ python3 setup_python.py --help
+
+usage: setup_python.py [-h] [-v] [--skip-temp-deletion] [--force-update] [major_version]
+
+Download and setup redistributable Python Interpreter from https://github.com/indygreg/python-build-standalone/ if
+needed ;)
+
+positional arguments:
+  major_version         Specify the Python version like: 3.10, 3.11, 3.12, ... (default: 3.12)
+
+options:
+  -h, --help            show this help message and exit
+  -v, --verbose         Increase verbosity level (can be used multiple times, e.g.: -vv) (default: 0)
+  --skip-temp-deletion  Skip deletion of temporary files (default: False)
+  --force-update        Update local Python interpreter, even if it is up-to-date (default: False)
+
+```
+
+## Include in own projects
+
+There is a unittest base class to include `setup_python.py` script in your project.
+If will check if the file is up2date and if not, it will update it.
+
+Just include `manageprojects` as a dev dependency in your project.
+And add a test like this:
+
+```python
+class IncludeSetupPythonTestCase(IncludeSetupPythonBaseTestCase):
+
+    # Set the path where the `setup_python.py` should be copied to:
+    DESTINATION_PATH = Path(your_package.__file__).parent) / 'setup_python.py'
+
+    # The test will pass, if the file is up2date, if it's update the script!
+    def test_setup_python_is_up2date(self):
+        self.auto_update_setup_python()
+```
+
+Feel free to do it in a completely different way, this is just a suggestion ;)
+
+## Workflow - 1. Check system Python
+
+If the system Python is the same major version as the required Python, we skip the download.
+
+The script just returns the path to the system Python interpreter.
+
+A local installed interpreter (e.g. in "~/.local") will be auto updated.
+
+## Workflow - 2. Collect latest release data
+
+We fetch the latest release data from the GitHub API:
+https://raw.githubusercontent.com/indygreg/python-build-standalone/latest-release/latest-release.json
+
+## Workflow - 3. Obtaining optimized Python distribution
+
+See: https://gregoryszorc.com/docs/python-build-standalone/main/running.html
+
+We choose the optimized variant based on the priority list:
+
+1. `pgo+lto`
+2. `pgo`
+3. `lto`
+
+For `x86-64` Linux we check the CPU flags from `/proc/cpuinfo` to determine the best variant.
+
+The "debug" build are ignored.
+
+## Workflow - 4. Check existing Python
+
+If the latest Python version is already installed, we skip the download.
+
+## Workflow - 4. Download and verify Archive
+
+All downloads will be done with a secure connection (SSL) and server authentication.
+
+If the latest Python version is already installed, we skip the download.
+
+Download will be done in a temporary directory.
+
+We download the archive file and the hash file for verification:
+
+* Archive extension: `.tar.zst`
+* Hash extension: `.tar.zst.sha256`
+
+We check the file hash after downloading the archive.
+
+## Workflow - 5. Add info JSON
+
+We add the file `info.json` with all relevant information.
+
+## Workflow - 6. Setup Python
+
+We add a shell script to `~/.local/bin/pythonX.XX` to start the Python interpreter.
+
+We display version information from Python and pip on `stderr`.
+
+The extracted Python will be moved to the final destination in `~/.local/pythonX.XX/`.
+
+The script set's the correct `PYTHONHOME` environment variable.
+
+## Workflow - 7. print the path
+
+If no errors occurred, the path to the Python interpreter will be printed to `stdout`.
+So it's usable in shell scripts, like:
+
+```shell
+#!/usr/bin/env sh
+
+set -e
+
+PY_313_BIN=$(python3 setup_python.py -v 3.13)
+echo "Python 3.13 used from: '${PY_313_BIN}'"
+
+set -x
+
+${PY_313_BIN} -VV
+
+```

manageprojects/__init__.py

@@ -3,5 +3,5 @@
     Manage Python / Django projects
 """
 
-__version__ = '0.18.0'
+__version__ = '0.19.0'
 __author__ = 'Jens Diemer <[email protected]>'

manageprojects/install_python.py

@@ -38,12 +38,12 @@
 
 """DocWrite: install_python.md # Install Python Interpreter
 Download Python source code from official Python FTP server:
-DocWriteMacro: manageprojects.tests.docwrite_macros.ftp_url"""
+DocWriteMacro: manageprojects.tests.docwrite_macros_install_python.ftp_url"""
 PY_FTP_INDEX_URL = 'https://www.python.org/ftp/python/'
 
 """DocWrite: install_python.md ## Supported Python Versions
 The following major Python versions are supported and verified with GPG keys:
-DocWriteMacro: manageprojects.tests.docwrite_macros.supported_python_versions
+DocWriteMacro: manageprojects.tests.docwrite_macros_install_python.supported_python_versions
 The GPG keys taken from the official Python download page: https://www.python.org/downloads/"""
 GPG_KEY_IDS = {
     # Thomas Wouters (3.12.x and 3.13.x source files and tags):
@@ -58,7 +58,7 @@
 
 """DocWrite: install_python.md ## Workflow - 3. Check local installed Python
 We assume that the `make altinstall` will install local Python interpreter into:
-DocWriteMacro: manageprojects.tests.docwrite_macros.default_install_prefix
+DocWriteMacro: manageprojects.tests.docwrite_macros_install_python.default_install_prefix
 See: https://docs.python.org/3/using/configure.html#cmdoption-prefix"""
 DEFAULT_INSTALL_PREFIX = '/usr/local'
 
@@ -198,7 +198,7 @@ def install_python(
 
     """DocWrite: install_python.md ## Workflow - 2. Get latest Python release
     We fetch the latest Python release from the Python FTP server, from:
-    DocWriteMacro: manageprojects.tests.docwrite_macros.ftp_url"""
+    DocWriteMacro: manageprojects.tests.docwrite_macros_install_python.ftp_url"""
     # Get latest full version number of Python from Python FTP:
     py_required_version = get_latest_versions(
         html=get_html_page(PY_FTP_INDEX_URL),
@@ -225,7 +225,7 @@ def install_python(
     """DocWrite: install_python.md ## Workflow - 4. Download Python sources
     The download will be done in a temporary directory. The directory will be deleted after the installation.
     This can be skipped via CLI argument. The directory will be prefixed with:
-    DocWriteMacro: manageprojects.tests.docwrite_macros.temp_prefix"""
+    DocWriteMacro: manageprojects.tests.docwrite_macros_install_python.temp_prefix"""
     with TemporaryDirectory(prefix=TEMP_PREFIX, delete=delete_temp) as temp_path:
         base_url = f'{PY_FTP_INDEX_URL}{py_required_version}'
 
@@ -291,7 +291,7 @@ def get_parser() -> argparse.ArgumentParser:
     ```shell
     $ python3 install_python.py --help
 
-    DocWriteMacro: manageprojects.tests.docwrite_macros.help
+    DocWriteMacro: manageprojects.tests.docwrite_macros_install_python.help
     ```
     """
     parser = argparse.ArgumentParser(
@@ -349,6 +349,6 @@ def main() -> Path:
     """DocWrite: install_python.md ## Workflow - 7. print the path
     If no errors occurred, the path to the Python interpreter will be printed to `stdout`.
     So it's usable in shell scripts, like:
-    DocWriteMacro: manageprojects.tests.docwrite_macros.example_shell_script
+    DocWriteMacro: manageprojects.tests.docwrite_macros_install_python.example_shell_script
     """
     print(python_path)