Docker error: res.pkl not found.

[shijie2016]

🐛 Bug Description

rdagent fin_factor在运行docker回测时发生错误，错误来源于运行qrun conf_combined.yaml时候docker出现问题，导致python read_exp_res.py找不到ret.pkl.

To Reproduce

Steps to reproduce the behavior:

1. rdagent fin_factor
2. wait until " qrun conf_combined.yaml " docker logs begin
3. nothing was done after "Init data done."

Expected Behavior

用qlib非docker版本, build from source, python 3.8. 可以运行，qrun conf_combined.yaml （和"combined_factors_df.pkl"在同一路径下）。docker版本和非docker版本运行结果应该一致。

Screenshot

Environment

Note: Users can run rdagent collect_info to get system information and paste it directly here.
2024-10-23 14:20:03.588 | INFO | rdagent.app.utils.info:sys_info:22 - Name of current operating system: Linux
2024-10-23 14:20:03.590 | INFO | rdagent.app.utils.info:sys_info:22 - Processor architecture: x86_64
2024-10-23 14:20:03.592 | INFO | rdagent.app.utils.info:sys_info:22 - System, version, and hardware information: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
2024-10-23 14:20:03.595 | INFO | rdagent.app.utils.info:sys_info:22 - Version number of the system: #1 SMP Fri Mar 29 23:14:13 UTC 2024
2024-10-23 14:20:03.597 | INFO | rdagent.app.utils.info:python_info:29 - Python version: 3.10.15 | packaged by conda-forge | (main, Sep 30 2024, 17:51:04) [GCC 13.3.0]
2024-10-23 14:20:03.835 | INFO | rdagent.app.utils.info:docker_info:39 - Container ID: de01ca3cb5d127f06f62768eb0c8ace1d2f03aba738a993fc9cbc12cdc577542
2024-10-23 14:20:03.837 | INFO | rdagent.app.utils.info:docker_info:40 - Container Name: dazzling_lehmann
2024-10-23 14:20:03.840 | INFO | rdagent.app.utils.info:docker_info:41 - Container Status: exited
2024-10-23 14:20:03.855 | INFO | rdagent.app.utils.info:docker_info:42 - Image ID used by the container: sha256:6e7b71e839b4c7c1199d040a6e7ae11faa910da59138d839079dcc1b96eeb1d7
2024-10-23 14:20:03.868 | INFO | rdagent.app.utils.info:docker_info:43 - Image tag used by the container: ['local_qlib:latest']
2024-10-23 14:20:03.870 | INFO | rdagent.app.utils.info:docker_info:44 - Container port mapping: {}
2024-10-23 14:20:03.873 | INFO | rdagent.app.utils.info:docker_info:45 - Container Label: {'com.nvidia.volumes.needed': 'nvidia_driver', 'desktop.docker.io/wsl-distro': 'Ubuntu', 'org.opencontainers.image.ref.name': 'ubuntu', 'org.opencontainers.image.version': '22.04'}
2024-10-23 14:20:03.883 | INFO | rdagent.app.utils.info:docker_info:46 - Startup Commands: nvidia-smi
2024-10-23 14:20:03.887 | INFO | rdagent.app.utils.info:rdagent_info:54 - RD-Agent version: 0.2.1
2024-10-23 14:20:05.155 | INFO | rdagent.app.utils.info:rdagent_info:76 - Package version: ['pydantic-settings==2.5.2', 'typer==0.12.5', 'cython==3.0.11', 'python-Levenshtein==0.26.0', 'scikit-learn==1.5.2', 'filelock==3.16.1', 'loguru-mypy==0.0.4', 'loguru==0.7.2', 'fire==0.7.0', 'fuzzywuzzy==0.18.0', 'openai==1.51.2', 'ruamel-yaml==0.18.6', 'tabulate==0.9.0', 'numpy==1.26.4', 'pandas==2.2.3', 'pandarallel==1.6.5', 'feedparser==6.0.11', 'matplotlib==3.9.2', 'langchain==0.3.3', 'langchain-community==0.3.1', 'tiktoken==0.8.0', 'pymupdf==1.24.11', 'azure-identity==1.19.0', 'pypdf==5.0.1', 'azure-core==1.31.0', 'azure-ai-formrecognizer==3.3.3', 'tables==3.10.1', 'tree-sitter-python==0.23.2', 'tree-sitter==0.23.1', 'jupyter==1.1.1', 'python-dotenv==1.0.1', 'docker==7.1.0', 'streamlit==1.39.0', 'plotly==5.24.1', 'st-theme==1.2.3', 'selenium==4.25.0', 'kaggle==1.6.17', 'nbformat==5.10.4', 'seaborn==0.13.2', 'setuptools-scm==8.1.0']

• Name of current operating system:
• Processor architecture:
• System, version, and hardware information:
• Version number of the system:
• Python version:
• Container ID:
• Container Name:
• Container Status:
• Image ID used by the container:
• Image tag used by the container:
• Container port mapping:
• Container Label:
• Startup Commands:
• RD-Agent version:
• Package version:

Additional Notes

[TPLin22]

Hi,

We recommend using Python 3.10 or 3.11, and it's best to run rdagent in Docker. Please try using the suggested environment and see if the error persists.

[SunsetWolf]

Hi, @shijie2016 ,
I tested with the latest rdagent, and it didn't reproduce the FileNotFoundError, so I suggest you to re-create the environment and install the latest version of rdagent (0.3.0), it will solve this issue.
Here is the information of my test environment, I hope my reply can help you.

Name of current operating system: Linux
Processor architecture: x86_64
System, version, and hardware information: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Version number of the system: #1 SMP Fri Mar 29 23:14:13 UTC 2024
Python version: 3.10.15 (main, Oct  3 2024, 07:27:34) [GCC 11.2.0]
Container ID: 35a1ca2e1531a80a5354e4435e7beb5d24b9ec14cc42b18856776e5f4d0c0249
Container Name: jolly_kalam
Container Status: running
Image ID used by the container: sha256:3cd86cd25c1609c67c5731aaa0a521bd8e0b7c378cf4e2918099a8516f1e505f
Image tag used by the container: ['local_qlib:latest']
Container port mapping: {}
Container Label: {'com.nvidia.volumes.needed': 'nvidia_driver', 'org.opencontainers.image.ref.name': 'ubuntu', 'org.opencontainers.image.version': '22.04'}
Startup Commands: qrun conf_combined.yaml
RD-Agent version: 0.3.0
Package version: ['pydantic-settings==2.6.0', 'python-Levenshtein==0.26.1', 'scikit-learn==1.5.2', 'filelock==3.16.1', 'loguru==0.7.2', 'fire==0.7.0', 'fuzzywuzzy==0.18.0', 'openai==1.52.2', 'numpy==1.26.4', 'pandas==2.2.3', 'pandarallel==1.6.5', 'matplotlib==3.9.2', 'langchain==0.3.4', 'langchain-community==0.3.3', 'tiktoken==0.8.0', 'pymupdf==1.24.12', 'pypdf==5.1.0', 'azure-ai-formrecognizer==3.3.3', 'tables==3.10.1', 'tree-sitter-python==0.23.2', 'tree-sitter==0.23.2', 'python-dotenv==1.0.1', 'docker==7.1.0', 'streamlit==1.39.0', 'plotly==5.24.1', 'st-theme==1.2.3', 'selenium==4.25.0', 'kaggle==1.6.17', 'nbformat==5.10.4', 'seaborn==0.13.2', 'setuptools-scm==8.1.0']