Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a checker to verify that shoots have been fully transferred #38

Open
paul-butcher opened this issue Nov 20, 2024 · 1 comment
Open

Comments

@paul-butcher
Copy link
Contributor

Currently, the checks for successful transfers are designed to facilitate retrying failed transfers.

There is an assumption that the presence of at least part of a shoot in born-digital-accessions implies that the shoot has been transferred. It also checks the storage_ingests index to see if the files have been recorded as successfully ingested.

It does not check the files on the target to see if the whole shoot has been successfully transferred. It is possible that a large shoot would be reported as having been successfully ingested, even if it has only been partly ingested.

e.g. if a 7 part shoot has only transferred 5 parts to the target system, and all of them have been successfully ingested, then that shoot would erroneously be declared successful.

@paul-butcher
Copy link
Contributor Author

This checker would need to iterate over all the object summaries in a shoot, and look for the corresponding files in the storage service. It would probably have to share the code for cracking apart large shoots by size, so that it can add the appropriate 00x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant