Variant calling with OQFE pipeline on cluster systems

public 7mo ago 0 bookmarks

View Workflow

Help improve this workflow!

This workflow has been published but could be further improved with some additional meta data:

Keyword(s) in categories input, output, operation, topic

You can help improve this workflow by suggesting the addition or removal of keywords, suggest changes and report issues, or request to become a maintainer of the Workflow .

Description

This workflow modifies the docker version of the OQFE workflow so that it can be executed on a cluster system using singularity. It addresses a few problems with the original pipeline, which is designed to be executed on a single server or virtual machine.

Running environment

This workflow is designed to be executed on a cluster system (SLURM, PBS etc) using singularity.

Code Snippets

[ -f ${bin_dir}/oqfe_${oqfe_version}.sif ] || \
  singularity pull docker://dnanexus/oqfe:${oqfe_version}

SoS Notebook Singularity Hub From line 20 of oqfe_pipeline_cluster.ipynb

[ -f ${bin_dir}/deepvariant_${deep_variant_version}.sif ] || \
  singularity pull docker://google/deepvariant:${deep_variant_version}

SoS Notebook Singularity Hub From line 23 of oqfe_pipeline_cluster.ipynb

[ -f ${bin_dir}/deepvariant_${deep_variant_version}-gpu.sif ] || \
  singularity pull docker://google/deepvariant:${deep_variant_version}-gpu

SoS Notebook Singularity Hub From line 26 of oqfe_pipeline_cluster.ipynb

set -e

mkdir -p ${work_dir}
#
singularity exec --home ${work_dir} -W ${work_dir} \
  --bind "${source_dir}":/source_dir \
  --bind "${reference_dir}":/reference_dir \
  --bind "${bin_dir}":/oqfe_bin \
  ${bin_dir}/oqfe_${oqfe_version}.sif python3.6 /oqfe_bin/oqfe_mem \
      --forward-reads /source_dir/${_input:b} \
      --num-cores 32 --memory 60000 \
      --sample ${sample_id} --reuse-existing-outputs \
      --cram-reference-fasta  /reference_dir/GRCh38_full_analysis_set_plus_decoy_hla.fa
# remove intermediate files, move output
[ -f ${work_dir}/output/${sample_id}.oqfe.cram ] && mv ${work_dir}/output/* ${oqfe_output_dir} && rm -rf ${work_dir}

SoS Notebook Singularity Hub From line 55 of oqfe_pipeline_cluster.ipynb

mkdir -p ${deep_variant_output_dir}
# https://github.com/google/deepvariant/blob/r1.6/docs/deepvariant-quick-start.md
singularity run \
  --bind "${source_dir}":/input \
  --bind "${deep_variant_output_dir}":/output \
  --bind "${reference_dir}":/reference \
  --nv \
  ${bin_dir}/deepvariant_${deep_variant_version}-gpu.sif \
  /opt/deepvariant/bin/run_deepvariant \
      --model_type WGS \
      --ref /reference/GRCh38_full_analysis_set_plus_decoy_hla.fa \
      --reads /input/${sample_id}.oqfe.cram \
      --output_vcf /output/${sample_id}.output.vcf.gz \
      --output_gvcf /output/${sample_id}.output.g.vcf.gz \
      --num_shards 32 \
      --intermediate_results_dir /output/intermediate_results_dir