Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin updates and resource optimization #21

Open
wants to merge 39 commits into
base: main
Choose a base branch
from

Conversation

nakib103
Copy link
Contributor

@nakib103 nakib103 commented Jun 6, 2024

Related to ENSVAR-6218

  • Updates to VEP config ini file
    • Added following plugins -
      • REVEL
      • ClinPred
      • Downstream
      • Add pCADD
    • Allow some plugin files to be absent (just skip) , e.g - MaveDB and AlphaMissense do not have file for GRCh37 assembly.
    • Update gnomAD v4.1 file locations and add gnomAD frequency for all HPRC assemblies
  • Resource optimisation -
    • Memory -
      • Added low, medium, and high type resource. And removed separate human profile.
      • Added RunVEP memory that depends on bin size.
      • Separate wigToBigWig process as it takes more memory and added memory based on wig file size.
      • Added remove_variants and summary_stats memory based in vcf file size.
      • Added split_vcf memory as fixed 32GB (probably can be more refined later)
    • Storage -
      • Summary stats was taking a lot of storage as it work on uncompress file. Made it so that it can work on compress file and take less storage.
  • Accustom nextflow-vep changes -
  • Allow force creation of cache and FASTA for human too. Updated to download FASTA and cache for human 37 assembly.
  • Bugfix: skip plugin if file does not exist. Adding skip option to check_plugin_files function was adding plugin even if file not present instead of skipping. Fixed that bug. This bug was causing several folds slowness due to Conservation plugin with wrong file.
  • Bugfix: pipeline failed if source had / in its name. Replacing it with ASCII value.
  • Print pipeline summaries and generate reports by default.

@nakib103 nakib103 changed the title Do not error out on MaveDB GRCh37 Skip MaveDB/AlphaMissense file check for GRCh37 Jun 6, 2024
@nakib103 nakib103 changed the title Skip MaveDB/AlphaMissense file check for GRCh37 Plugin updates and resource optimization Aug 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants