miRNA Analysis Pipeline: NGS Data Analysis for miRNA Using Snakemake

public 1yr ago Version: 2 0 bookmarks

View Workflow

Help improve this workflow!

This workflow has been published but could be further improved with some additional meta data:

Keyword(s) in categories input, output, operation

You can help improve this workflow by suggesting the addition or removal of keywords, suggest changes and report issues, or request to become a maintainer of the Workflow .

miRNA analysis pipeline

This Snakemake-based workflow was designed with the aim of enable reproducible analysis of miRNA NGS data.

As the QIAseq miRNA Library Kit (QIAGEN) was used in our lab, the pipeline is configured for managing UMIs present in the read as suggested by the manufacturer. Anyway, few changes would allow data anlysis from different kits.

The pipeline can be considered as a hybrid solution between common state-of-the-art literature, as reported in Potla et al. and Qiagen analysis guidelines .

Requirements

To run miRNA pipeline, Conda must be present in your computer.
To install Conda, see https://conda.io/miniconda.html

Installation

You can directly clone the repository in your working directory

git clone https://github.com/solida-core/miRNA.git

Then you need to create the base environment.

cd miRNA
conda create -q -n MYENV_NAME python=3.7
conda env update -q -n MYENV_NAME --file environment.yaml

Once the virtual environment is ready, activate it and check the correct installation of Snakmake

conda activate MYENV_NAME
snakemake --version

Run the Pipeline

To run the pipeline you need to edit manually the config.yaml file, providing paths for your references.

snakemake --configfile config.yaml --snakefile Snakefile --use-conda -d ANALYSIS_DIR

Generate Snakemake Report

When the analysis is completed, you can generate a Snakemake report with analysis overview, QC report and mmiRNA counts.

snakemake --configfile config.yaml --snakefile Snakefile --use-conda --report [--report-stylesheet path_to/custom.css] -d ANALYSIS_DIR

This produces the report.html file inside the ANALYSIS_DIR .

Pipeline Description

The workflow consists of 6 main steps:

Get UMI: Qiagen UMIs are integrated in the read sequence, near the 3' adapter. To identify the 12 bases sequence of the UMI we use UMI_tools which allow to search the adapter sequence, discard it and keep the UMI sequence, includeing it in the header of the read.
Trimming: TrimGalore is used for quality trimming and read length selection (default min read length is set to 16, max to 30). These values can be edited in the config.yaml file.
QC: a QC report is generated with MultiQC, including information from FastQC, TrimGalore and Mirtrace.
Mapping: reads are aligned against multiple databases. Only reads that do not map to a db undergo alignment against the succesive database.
Deduplication and Count: UMIs are used for the deduplication of mapped reads and then a table with counts for each miRNA is generated.
Discovery: WORK IN PROGRESS

Code Snippets

shell:
    "umi_tools dedup "
    "-I {input.bam} "
    "-S {output} "
    "--method=unique "
    "--log={log} "

SnakeMake umi_tools From line 12 of rules/count_mirna.smk

shell:
    "samtools idxstats {input.bam} | cut -f 1,3 "
    "> {output.counts} "

SnakeMake SAMtools From line 35 of rules/count_mirna.smk

shell:
    "htseq-count "
    "params"
    "-q {input.bam} "
    "{input.gff} "
    ">{output.counts}"

SnakeMake htseqcount From line 52 of rules/count_mirna.smk

shell:
    "cp {input} {output}"

SnakeMake From line 11 of rules/delivery.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log}"

SnakeMake Bowtie 2 From line 17 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log}"

SnakeMake Bowtie 2 From line 42 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log} "

SnakeMake Bowtie 2 From line 67 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log}"

SnakeMake Bowtie 2 From line 92 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log} "

SnakeMake Bowtie 2 From line 117 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log} "

SnakeMake Bowtie 2 From line 142 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log} "

SnakeMake Bowtie 2 From line 167 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log} "

SnakeMake Bowtie 2 From line 192 of rules/mapping.smk

shell:
    "bowtie "
    "{params.params} "
    "--threads {threads} "
    "{params.basename} "
    "{input} "
    "--un {output.fastq} "
    "-S {output.sam} "
    ">& {log} "

SnakeMake Bowtie 2 From line 217 of rules/mapping.smk

shell:
    "fastqc "
    "{input} "
    "--outdir {params.outdir} "
    "--quiet "
    ">& {log}"

SnakeMake FastQC From line 14 of rules/qc.smk

shell:
    "fastqc "
    "{input} "
    "--outdir {params.outdir} "
    "--quiet "
    ">& {log}"

SnakeMake FastQC From line 34 of rules/qc.smk

shell:
    "fastqc "
    "{input} "
    "--outdir {params.outdir} "
    "--quiet "
    ">& {log}"

SnakeMake FastQC From line 54 of rules/qc.smk

shell:
    "mirtrace "
    "qc "
    "{params.species} "
    "-o {params.outdir} "
    "{input} "
    "{params.params} "
    ">& {log} "

SnakeMake From line 78 of rules/qc.smk

shell:
    "multiqc "
    "{input} "
    "{params.fastqc} "
    "{params.trimming} "
    "{params.params} "
    "-o {params.outdir} "
    "-n {params.outname} "
    "--sample-names {params.reheader} "
    ">& {log}"

SnakeMake MultiQC From line 113 of rules/qc.smk

shell:
    "samtools view -b "
    "--threads {threads} "
    "-T {params.genome} "
    "-o {output.first} "
    "-O {params.output_fmt} "
    "{input.first} "

SnakeMake SAMtools From line 14 of rules/samtools.smk

shell:
    "samtools view -b "
    "--threads {threads} "
    "-T {params.genome} "
    "-o {output.second} "
    "-O {params.output_fmt} "
    "{input.second} "

SnakeMake SAMtools From line 35 of rules/samtools.smk

shell:
    "samtools sort "
    "--threads {threads} "
    "-T {params.tmp_dir} "
    "-O {params.output_fmt} "
    "--reference {params.genome} "
    "-o {output.first} "
    "{input.first} "

SnakeMake SAMtools From line 58 of rules/samtools.smk

shell:
    "samtools sort "
    "--threads {threads} "
    "-T {params.tmp_dir} "
    "-O {params.output_fmt} "
    "--reference {params.genome} "
    "-o {output.second} "
    "{input.second} "

SnakeMake SAMtools From line 83 of rules/samtools.smk

shell:
    "samtools merge "
    "--threads {threads} "
    "-O {params.output_fmt} "
    "--reference {params.genome} "
    "{output} "
    "{input.first} {input.second} "

SnakeMake SAMtools From line 108 of rules/samtools.smk

shell:
    "samtools index "
    "{input}"

SnakeMake SAMtools From line 124 of rules/samtools.smk

shell:
    "samtools index "
    "{input}"

SnakeMake SAMtools From line 136 of rules/samtools.smk

shell:
    "umi_tools extract "
    "--stdin={input} "
    "--stdout={output} "
    "--log={log} "
    "--extract-method=regex "
    "--bc-pattern='.+(?P<discard_1>AACTGTAGGCACCATCAAT)"
    "{{s<=2}}"
    "(?P<umi_1>.{{12}})(?P<discard_2>.*)'"

SnakeMake umi_tools From line 20 of rules/umi_and_trimming.smk

shell:
    "cp {input.r1} {output.r1}"

SnakeMake From line 36 of rules/umi_and_trimming.smk

shell:
    "mkdir -p qc/fastqc; "
    "trim_galore "
    "{params.extra} "
    "--cores {threads} "
    "-o {params.outdir} "
    "{input} "
    ">& {log}"