Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prevent concurrent access to repodata files with lock #955

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tl-hbk
Copy link

@tl-hbk tl-hbk commented Nov 21, 2024

Description

I've been trying to run multiple rattler-build processes concurrently and ran into a few issues mentioned here.

prefix-dev/rattler-build#1149

Whenever rattler_index::index was called it would error either because another process was trying to also call rattler_index::index or a SparseRepoData had one of the files memory mapped. This PR adds locks to both locations.

The name of the lock files used match the one already used by rattler_repodata_gateway::fetch::fetch_repo_data

@tl-hbk
Copy link
Author

tl-hbk commented Nov 22, 2024

After testing this with rattler-build for a bit this ocassionally deadlocks if multiple processes want the locks for the same files but acquire them in different order

EDIT: Switched to open_ro on the lock which I think shouldn't block other processes from using the SparseRepoData but should prevent index from getting an exclusive lock. I think I'll need to add another lock before SparseRepoData are created or when fetch_repo_data is called and hold onto that first so that the order of the rest of the locks doesn't cause issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant