Variant-calling pipeline for Restriction site-associated DNA sequencing

public public 1yr ago Version: dev 0 bookmarks

Introduction

nf-core/radseq is a bioinformatics best-practice variant-calling pipeline for Restriction site-associated DNA sequencing (RADseq).

The pipeline is built using Nextflow , a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The Nextflow DSL2 implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from nf-core/modules in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!

On release, automated continuous integration tests run the pipeline on a full-sized dataset on the AWS cloud infrastructure. This ensures that the pipeline runs on AWS, has sensible resource allocation defaults set to run on real-world datasets, and permits the persistent storage of results to benchmark between pipeline releases and other analysis sources. The results obtained from the full-sized test can be viewed on the nf-core website .

Pipeline summary

  1. Read QC ( FastQC )

  2. Adapter and quality trimming ( FastP )

  3. Choice of constructing psuedoreference (i) or using existing reference (ii)

    i. CDHIT-est -> Rainbow

    ii. No Preprossesing

  4. Choice of alignment software

    i. BWA

    ii. BWA-mem2

  5. UMI-based deduplicated ( UMI-tools )

  6. Index, merge and index alignments ( SAMtools )

  7. Construct intervals for freebayes ( BEDtools )

  8. Variant calling ( FreeBayes )

  9. Present QC for raw reads ( MultiQC )

Quick Start

  1. Install Nextflow ( >=21.10.3 )

  2. Install any of Docker , Singularity , Podman , Shifter or Charliecloud for full pipeline reproducibility (please only use Conda as a last resort; see docs )

  3. Download the pipeline and test it on a minimal dataset with a single command:

    nextflow run nf-core/radseq -profile test,YOURPROFILE
    

    Note that some form of configuration will be needed so that Nextflow knows how to fetch the required software. This is usually done in the form of a config profile ( YOURPROFILE in the example command above). You can chain multiple config profiles in a comma-separated string.

    • The pipeline comes with config profiles called docker , singularity , podman , shifter , charliecloud and conda which instruct the pipeline to use the named tool for software management. For example, -profile test,docker .

    • Please check nf-core/configs to see if a custom config file to run nf-core pipelines already exists for your Institute. If so, you can simply use -profile <institute> in your command. This will enable either docker or singularity and set the appropriate execution settings for your local compute environment.

    • If you are using singularity and are persistently observing issues downloading Singularity images directly due to timeout or network issues, then you can use the --singularity_pull_docker_container parameter to pull and convert the Docker image instead. Alternatively, you can use the nf-core download command to download images first, before running the pipeline. Setting the NXF_SINGULARITY_CACHEDIR or singularity.cacheDir Nextflow options enables you to store and re-use the images from a central location for future pipeline runs.

    • If you are using conda , it is highly recommended to use the NXF_CONDA_CACHEDIR or conda.cacheDir settings to store the environments in a central location for future pipeline runs.

  4. Start running your own analysis!

    nextflow run nf-core/radseq -profile <docker/singularity/podman/shifter/charliecloud/conda/institute> --input samplesheet.csv
    

Documentation

The nf-core/radseq pipeline comes with documentation about the pipeline usage , parameters and output .

Credits

dDocent was originally written by Jon Puritz and developed into nf-core/radseq by Gabriel Barrett

Contributions and Support

If you would like to contribute to this pipeline, please see the contributing guidelines .

For further information or help, don't hesitate to get in touch on the Slack #radseq channel (you can join with this invite ).

Citations

An extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.

You can cite the nf-core publication as follows:

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x .

Code Snippets

24
25
26
27
28
29
30
31
32
33
"""
bedops --merge ${bed.collect().join(" ")} \\
${args} \\
> ${prefix}.bed

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedops: \$(bedops --version | awk '/version:/{print \$2}')
END_VERSIONS
"""
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
"""
awk '{if (\$1 ~ /Cl/) clus = clus + 1; else  print \$3 "\\t" clus}' ${clstr} | \\
sed -e 's/[>dDocent_Contig_,...]//g' | \\
sort -g -k1 -S 2G > ${prefix}.sort.contig.cluster.ids

paste ${prefix}.sort.contig.cluster.ids ${totaluniqseq} > ${prefix}.contig.cluster.totaluniqseq

# cd-hit TO rainbow cluster format
sort -k2,2 -g ${prefix}.contig.cluster.totaluniqseq | \\
sed -e 's/NNNNNNNNNN/	/g' > ${prefix}.rclstr
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    GNU: \$(sort --version | awk 'NR==1{print \$4}')
END_VERSIONS
"""
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
"""
awk '{if (\$1 ~ /Cl/) clus = clus + 1; else  print \$3 "\\t" clus}' ${clstr} | \\
sed -e 's/[>dDocent_Contig_,...]//g' | \\
sort -g -k1 > ${prefix}.sort.contig.cluster.ids

paste ${prefix}.sort.contig.cluster.ids <(awk '!/>/' ${totaluniqseq}) > ${prefix}.contig.cluster.Funiq

sed -e 's/NNNNNNNNNN/   /g' ${totaluniqseq} | \\
sort -k1 -S 2G | \\
awk '{print \$0 "\\t" NR}'  > ${prefix}.totaluniqseq.CN

join -t \$'\\t' -1 3 -2 1 ${prefix}.contig.cluster.Funiq ${prefix}.totaluniqseq.CN -o 2.3,1.2,2.1,2.2 > ${prefix}.contig.cluster.totaluniqseq

# cd-hit TO rainbow cluster format
sort -k2,2 -g ${prefix}.contig.cluster.totaluniqseq -S 2G | \\
sed -e 's/NNNNNNNNNN/	/g' > ${prefix}..rclstr

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    GNU: \$(sort --version | awk 'NR==1{print \$4}')
END_VERSIONS
"""
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
"""
awk -v x="${withinIndv_MinDepth}" '(\$1 >= x)' *.uniq.seqs | \\
cut -f2 | \\
sed -e 's/NNNNNNNNNN/-/' >  total.uniqs

cut -f 1 -d "-" total.uniqs > total.u.F
cut -f 2 -d "-" total.uniqs > total.u.R

paste total.u.F total.u.R | \\
sort -k1 -S 2G > total.fr

awk -v x=${withinIndv_MinDepth} '\$1 >= x' *.uniq.seqs | \\
cut -f2 | \\
sed -e 's/NNNNNNNNNN/	/g' | \\
cut -f1 | \\
uniq | \\
sort -S 2G | \\
uniq -c > total.f.uniq

join -1 2 -2 1 -o 1.1,1.2,2.2 total.f.uniq total.fr | \\
awk '{print \$1 "\t" \$2 "NNNNNNNNNN" \$3}' | \\
awk -v x=${acrossIndv_MinDepth} '\$1 >= x' > uniq.k.${withinIndv_MinDepth}.c.${acrossIndv_MinDepth}.seqs

sort -k1 -r -n -S 2G uniq.k.${withinIndv_MinDepth}.c.${acrossIndv_MinDepth}.seqs | \\
cut -f2 > totaluniqseq
awk '{c= c + 1; print ">dDocent_Contig_" c "\\n" \$1}' totaluniqseq > ${prefix}_uniq.full.fasta

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
END_VERSIONS
"""
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
"""
awk -v x=${withinIndv_MinDepth} '(\$1 >= x)' *.uniq.seqs | \\
cut -f2 | \\
perl -e 'while (<>) {chomp; \$z{\$_}++;} while((\$k,\$v) = each(%z)) {print "\$v\\t\$k\\n";}' | \\
awk -v x=${acrossIndv_MinDepth} '(\$1 >= x)' > uniq.k.${withinIndv_MinDepth}.c.${acrossIndv_MinDepth}.seqs

# order the sequences for reproducibility 
sort -k1 -r -n -S 2G uniq.k.${withinIndv_MinDepth}.c.${acrossIndv_MinDepth}.seqs | \\
cut -f2 > totaluniqseq 

awk '{c= c + 1; print ">dDocent_Contig_" c "\\n" \$1}' totaluniqseq > ${prefix}_uniq.full.fasta

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    perl: \$(perl --version | sed -n -E '/^This is/ s/.*\\(v([[:digit:].]+)\\).*/\\1/p')
END_VERSIONS
"""
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
"""
cat ${intersect} ${low_cov} > ${prefix}_cov.split.stats
echo "${lengths.join("\n")}" > ${prefix}_lengths.txt
MaxLen=\$(awk '{ print length() | "sort -rn" }' ${prefix}_lengths.txt| head -1)
MaxLen2=\$(( \$MaxLen / 2 ))

TT=\$(( \$MaxLen2 * 1000000 ))
DP=\$(awk '{print \$4}' ${cov} | sort -rn | perl -e '\$d=.001;@l=<>;print \$l[int(\$d*@l)]')
CC=\$( awk -v x=\$DP '\$4 < x' ${cov} | awk '{len=\$3-\$2;lc=len*\$4;tl=tl+lc} END {OFMT = "%.0f";print tl/"'${task.cpus}'"}')

awk -v x=\$DP '\$4 < x' ${prefix}_cov.split.stats | sort -k1,1 -k2,2 | awk -v cutoff=\$CC -v tt=\$TT 'BEGIN{i=1}
    {len=\$3-\$2;lc=len*\$4;cov = cov + lc
    if (NR == 1 && lc > tt) {x="mapped."i".bed";print \$1"\\t"\$2"\\t"\$3 > x; i=i+1; e=1}
    else if ( cov < cutoff && lc < tt) {x="mapped."i".bed";print \$1"\\t"\$2"\\t"\$3 > x; e=0}
    else if (lc > tt && e > 0 ) {x="mapped."i".bed"; print \$1"\\t"\$2"\\t"\$3 > x; cov=0;i=i+1; e=1}
    else if (lc > tt && e < 1 ) {i=i+1; x="mapped."i".bed"; print \$1"\\t"\$2"\\t"\$3 > x; cov=0;i=i+1;e=1}
    else if (cov > cutoff && lc < tt ) {i=i+1; x="mapped."i".bed"; print \$1"\\t"\$2"\\t"\$3 > x; cov=lc;e=0}
    }'
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    perl: \$(perl --version | sed -n -E '/^This is/ s/.*\\(v([[:digit:].]+)\\).*/\\1/p')
END_VERSIONS
"""
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
"""
cat ${intersect} ${low_cov} > ${prefix}_cov.split.stats
echo "${lengths.join("\n")}" > ${prefix}_lengths.txt
MaxLen=\$(awk '{ print length() | "sort -rn" }' ${prefix}_lengths.txt| head -1)
MaxLen2=\$(( \$MaxLen / 2 ))

TT=\$(( \$MaxLen2 * 1000000 ))
DP=\$(awk '{print \$4}' ${cov} | sort -rn | perl -e '\$d=.00005;@l=<>;print \$l[int(\$d*@l)]')
CC=\$( awk -v x=\$DP '\$4 < x' ${cov} | awk '{len=\$3-\$2;lc=len*\$4;tl=tl+lc} END {OFMT = "%.0f";print tl/"'${task.cpus}'"}')

awk -v x=\$DP '\$4 < x' ${prefix}_cov.split.stats | sort -k1,1 -k2,2 | awk -v cutoff=\$CC -v tt=\$TT 'BEGIN{i=1}
    {len=\$3-\$2;lc=len*\$4;cov = cov + lc
    if (NR == 1 && lc > tt) {x="mapped."i".bed";print \$1"\\t"\$2"\\t"\$3 > x; i=i+1; e=1}
    else if ( cov < cutoff && lc < tt) {x="mapped."i".bed";print \$1"\\t"\$2"\\t"\$3 > x; e=0}
    else if (lc > tt && e > 0 ) {x="mapped."i".bed"; print \$1"\\t"\$2"\\t"\$3 > x; cov=0;i=i+1; e=1}
    else if (lc > tt && e < 1 ) {i=i+1; x="mapped."i".bed"; print \$1"\\t"\$2"\\t"\$3 > x; cov=0;i=i+1;e=1}
    else if (cov > cutoff && lc < tt ) {i=i+1; x="mapped."i".bed"; print \$1"\\t"\$2"\\t"\$3 > x; cov=lc;e=0}
    }'
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    perl: \$(perl --version | sed -n -E '/^This is/ s/.*\\(v([[:digit:].]+)\\).*/\\1/p')
END_VERSIONS   
"""
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
"""
gunzip -c ${forward_reads} | \\
awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' | \\
awk '!/>/' > ${prefix}.forward

gunzip -c ${reverse_reads} | \\
awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' | \\
awk '!/>/' > ${prefix}.reverse

paste -d '-' ${prefix}.forward ${prefix}.reverse | \\
awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' | \\
sed -e 's/-/NNNNNNNNNN/' | \\
perl -e 'while (<>) {chomp; \$z{\$_}++;} while((\$k,\$v) = each(%z)) {print "\$v\\t\$k\\n";}' > ${prefix}.uniq.seqs

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    perl: \$(perl --version | sed -n -E '/^This is/ s/.*\\(v([[:digit:].]+)\\).*/\\1/p')
END_VERSIONS
"""
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
        """
        gunzip -c ${forward_reads} | \\
        awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' | \\
        awk '!/>/' > ${prefix}.forward

        gunzip -c ${reverse_reads} | \\
        awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' | \\
        awk '!/>/' > ${prefix}.reverse

        paste ${prefix}.forward ${prefix}.reverse | sort -k1 -S 200M > ${prefix}.fr
		cut -f1 ${prefix}.fr | uniq -c > ${prefix}.f.uniq && cut -f2 ${prefix}.fr > ${prefix}.r
		awk '{for(i=0;i<\$1;i++)print}' ${prefix}.f.uniq > ${prefix}.f.uniq.e

        paste -d '-' ${prefix}.f.uniq.e ${prefix}.r | \\
        awk '!/NNN/'| \\
        sed -e 's/-/NNNNNNNNNN/' | \\
        sed -e 's/^[ \\t]*//' | \\
        sed -e 's/\\s/\\t/g' > ${prefix}.uniq.seqs

        rm ${prefix}.f.uniq.e ${prefix}.f.uniq ${prefix}.r ${prefix}.fr

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
END_VERSIONS
        """
77
78
79
80
81
82
83
84
85
86
87
88
"""
gunzip -c ${forward_reads} | \\
awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' | \\
awk '!/>/' | \\
perl -e 'while (<>) {chomp; \$z{\$_}++;} while((\$k,\$v) = each(%z)) {print "\$v\\t\$k\\n";}' > ${prefix}.uniq.seqs

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    perl: \$(perl --version | sed -n -E '/^This is/ s/.*\\(v([[:digit:].]+)\\).*/\\1/p')
END_VERSIONS
"""
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
"""
awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' ${assembled_fastq} | \\
awk '!/>/' | \\
perl -e 'while (<>) {chomp; \$z{\$_}++;} while((\$k,\$v) = each(%z)) {print "\$v\\t\$k\\n";}' > ${prefix}.uniq.seqs

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
    perl: \$(perl --version | sed -n -E '/^This is/ s/.*\\(v([[:digit:].]+)\\).*/\\1/p')
END_VERSIONS
"""
17
18
19
20
21
22
23
24
25
26
"""
check_samplesheet.py \\
    $samplesheet \\
    samplesheet.valid.csv

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    python: \$(python --version | sed 's/Python //g')
END_VERSIONS
"""
23
24
25
26
27
28
29
30
31
32
33
34
"""
bcftools concat \\
    --output ${prefix}.vcf.gz \\
    $args \\
    --threads $task.cpus \\
    ${vcfs}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bcftools: \$(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*\$//')
END_VERSIONS
"""
38
39
40
41
42
43
44
45
"""
touch ${prefix}.vcf.gz

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bcftools: \$(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*\$//')
END_VERSIONS
"""
NextFlow From line 38 of concat/main.nf
24
25
26
27
28
29
30
31
32
33
34
35
"""
bcftools \\
    sort \\
    --output ${prefix}${interval}.vcf.gz \\
    $args \\
    $vcf

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bcftools: \$(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*\$//')
END_VERSIONS
"""
40
41
42
43
44
45
46
47
"""
touch ${prefix}.vcf.gz

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bcftools: \$(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*\$//')
END_VERSIONS
"""
NextFlow From line 40 of sort/main.nf
29
30
31
32
33
34
35
36
37
38
39
40
41
"""
bcftools stats \\
    $args \\
    $regions_file \\
    $targets_file \\
    $samples_file \\
    $vcf > ${prefix}.bcftools_stats.txt

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bcftools: \$(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*\$//')
END_VERSIONS
"""
46
47
48
49
50
51
52
53
"""
touch ${prefix}.bcftools_stats.txt

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bcftools: \$(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*\$//')
END_VERSIONS
"""
NextFlow From line 46 of stats/main.nf
23
24
25
26
27
28
29
30
31
32
33
34
"""
bedtools \\
    bamtobed \\
    $args \\
    -i $bam \\
    > ${prefix}.bed

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedtools: \$(bedtools --version | sed -e "s/bedtools v//g")
END_VERSIONS
"""
24
25
26
27
28
29
30
31
32
33
34
35
36
37
"""
bedtools \\
    coverage \\
    $args \\
    $reference \\
    -a $input_A \\
    -b $input_B \\
    > ${prefix}.cov

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedtools: \$(echo \$(bedtools --version 2>&1) | sed 's/^.*bedtools v//' ))
END_VERSIONS
"""
27
28
29
30
31
32
33
34
35
36
37
38
39
"""
bedtools \\
    intersect \\
    -a $intervals1 \\
    -b $intervals2 \\
    $args \\
    > ${prefix}.${extension}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedtools: \$(bedtools --version | sed -e "s/bedtools v//g")
END_VERSIONS
"""
46
47
48
49
50
51
52
53
"""
touch ${prefix}.${extension}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedtools: \$(bedtools --version | sed -e "s/bedtools v//g")
END_VERSIONS
"""
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
    """
    echo "${lengths.join("\n")}" > "${prefix}_lengths.txt"
    MaxLen=\$(awk '{ print length() | "sort -rn" }' "${prefix}_lengths.txt" | head -1)
    #split cov.stats file into high and low coverage intervals
	awk '\$4 > ${coverage_threshold}' ${regions} > ${prefix}_cov.high.stats
	awk '\$4 <= ${coverage_threshold}' ${regions} > ${prefix}_cov.low.stats
    MaxLen2=\$(("\$MaxLen" / 2))
    ML1=\$(("\$MaxLen2" + 1))
    bedtools \\
        makewindows \\
        -b ${prefix}_cov.high.stats \\
        -w \$MaxLen2 -s \$ML1 \\
        $args \\
        > ${prefix}.tab

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedtools: \$(bedtools --version | sed -e "s/bedtools v//g")
END_VERSIONS
    """
25
26
27
28
29
30
31
32
33
34
35
36
37
"""
cat ${cov} | \\
bedtools sort -i - -faidx ${faidx} | \\
bedtools \\
    merge \\
    -i - \\
    $args \\
    > ${prefix}.cov
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedtools: \$(bedtools --version | sed -e "s/bedtools v//g")
END_VERSIONS
"""
29
30
31
32
33
34
35
36
37
38
39
40
41
"""
bedtools \\
    sort \\
    -i $intervals \\
    $genome_cmd \\
    $args \\
    > ${prefix}_sorted.${extension}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bedtools: \$(bedtools --version | sed -e "s/bedtools v//g")
END_VERSIONS
"""
22
23
24
25
26
27
28
29
30
31
32
33
34
"""
mkdir bwa
bwa \\
    index \\
    $args \\
    -p bwa/${fasta.baseName} \\
    $fasta

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwa: \$(echo \$(bwa 2>&1) | sed 's/^.*Version: //; s/Contact:.*\$//')
END_VERSIONS
"""
37
38
39
40
41
42
43
44
45
46
47
48
49
50
"""
mkdir bwa

touch bwa/genome.amb
touch bwa/genome.ann
touch bwa/genome.bwt
touch bwa/genome.pac
touch bwa/genome.sa

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwa: \$(echo \$(bwa 2>&1) | sed 's/^.*Version: //; s/Contact:.*\$//')
END_VERSIONS
"""
22
23
24
25
26
27
28
29
30
31
32
33
"""
mkdir bwamem2
bwa-mem2 \\
    index \\
    $args \\
    $fasta -p bwamem2/${fasta}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwamem2: \$(echo \$(bwa-mem2 version 2>&1) | sed 's/.* //')
END_VERSIONS
"""
36
37
38
39
40
41
42
43
44
45
46
47
48
"""
mkdir bwamem2
touch bwamem2/${fasta}.0123
touch bwamem2/${fasta}.ann
touch bwamem2/${fasta}.pac
touch bwamem2/${fasta}.amb
touch bwamem2/${fasta}.bwt.2bit.64

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwamem2: \$(echo \$(bwa-mem2 version 2>&1) | sed 's/.* //')
END_VERSIONS
"""
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
        """
        INDEX=`find -L ./ -name "*.amb" | sed 's/\\.amb\$//'`

        echo "${lengths.join("\n")}" > lengths.txt
        MLEN=\$(awk '{ print length() | "sort -rn" }' lengths.txt | head -1)
        INSERT=\$(( \$MLEN * 2 ))
        INSERTH=\$(( \$INSERT + 100 ))
        INSERTL=\$(( \$INSERT - 100 ))
        SD=\$(( \$INSERT / 5 ))

        bwa-mem2 \\
            mem \\
            $args \\
            -I \$INSERT,\$SD,\$INSERTH,\$INSERTL \\
            -R "@RG\\tID:${prefix}\\tSM:${prefix}\\tPL:Illumina" \\
            -t $task.cpus \\
            \$INDEX \\
            $reads \\
            | samtools view $args2 \\
            | samtools sort $args3 --threads $task.cpus -o ${prefix}.bam -

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwamem2: \$(echo \$(bwa-mem2 version 2>&1) | sed 's/.* //')
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
        """
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
        """
        INDEX=`find -L ./ -name "*.amb" | sed 's/\\.amb\$//'`
        bwa-mem2 \\
            mem \\
            $args \\
            -R "@RG\\tID:${prefix}\\tSM:${prefix}\\tPL:Illumina" \\
            -t $task.cpus \\
            \$INDEX \\
            $reads \\
            | samtools view $args2 \\
            | samtools sort $args3 --threads $task.cpus -o ${prefix}.bam -

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwamem2: \$(echo \$(bwa-mem2 version 2>&1) | sed 's/.* //')
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
        """        
79
80
81
82
83
84
85
86
"""
touch ${prefix}.bam
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwamem2: \$(echo \$(bwa-mem2 version 2>&1) | sed 's/.* //')
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
NextFlow From line 79 of mem/main.nf
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
        """
        INDEX=`find -L ./ -name "*.amb" | sed 's/.amb//'`

        echo "${lengths.join("\n")}" > lengths.txt
        MLEN=\$(awk '{ print length() | "sort -rn" }' lengths.txt | head -1)
        INSERT=\$(( \$MLEN * 2 ))
        INSERTH=\$(( \$INSERT + 100 ))
        INSERTL=\$(( \$INSERT - 100 ))
        SD=\$(( \$INSERT / 5 ))

        bwa mem \\
            $args \\
            -I \$INSERT,\$SD,\$INSERTH,\$INSERTL \\
            -R "@RG\\tID:${prefix}\\tSM:${prefix}\\tPL:Illumina" \\
            -t $task.cpus \\
            \$INDEX \\
            $reads \\
            | samtools view $args2 \\
            | samtools sort $args3 --threads $task.cpus -o ${prefix}.bam -

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwa: \$(echo \$(bwa 2>&1) | sed 's/^.*Version: //; s/Contact:.*\$//')
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
        """
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
        """
        INDEX=`find -L ./ -name "*.amb" | sed 's/.amb//'`

        bwa mem \\
            $args \\
            -R "@RG\\tID:${prefix}\\tSM:${prefix}\\tPL:Illumina" \\
            -t $task.cpus \\
            \$INDEX \\
            $reads \\
            | samtools view $args2 \\
            | samtools sort $args3 --threads $task.cpus -o ${prefix}.bam -

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    bwa: \$(echo \$(bwa 2>&1) | sed 's/^.*Version: //; s/Contact:.*\$//')
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
        """
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
"""
[ ! -f  ${prefix}.fastq.gz ] && ln -sf $reads ${prefix}.fastq.gz
fastp \\
    --stdout \\
    --in1 ${prefix}.fastq.gz \\
    --thread $task.cpus \\
    --json ${prefix}.fastp.json \\
    --html ${prefix}.fastp.html \\
    $fail_fastq \\
    $args \\
    2> ${prefix}.fastp.log \\
| gzip -c > ${prefix}.fastp.fastq.gz
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastp: \$(fastp --version 2>&1 | sed -e "s/fastp //g")
END_VERSIONS
"""
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
"""
MaxLen="\$(awk '!/>/' ${uniq_full_fasta}  | \\
    awk '(NR==1||length<shortest){shortest=length} END {print shortest}')"

fastp \\
    --in1 ${reads} \\
    --out1 ${prefix}.fastp.fastq.gz \\
    --thread ${task.cpus} \\
    ${args} \\
    --disable_quality_filtering \\
    --length_required \$MaxLen \\
    &> fastp.log

# Fastq back to Fasta
gunzip ${prefix}.fastp.fastq.gz
awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' ${prefix}.fastp.fastq | \\
    paste - - | \\
    sort -k1,1 -V | \\
    tr "\\t" "\\n" > ${prefix}.uniq.fasta

awk '!/>/' ${prefix}.uniq.fasta > ${prefix}.totaluniqseq

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastp: \$(fastp --version 2>&1 | sed -e "s/fastp //g")
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
END_VERSIONS
"""
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
"""
MaxLen="\$(awk '!/>/' ${uniq_full_fasta}  | \\
    awk '(NR==1||length<shortest){shortest=length} END {print shortest}')"

fastp \\
    --in1 ${reads} \\
    --out1 ${prefix}.fastp.fastq.gz \\
    --thread ${task.cpus} \\
    ${args} \\
    --disable_quality_filtering \\
    --length_required \$MaxLen \\
    &> fastp.log

# Fastq back to Fasta
gunzip ${prefix}.fastp.fastq.gz
awk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' ${prefix}.fastp.fastq | \\
    paste - - | \\
    sort -k1,1 -V | \\
    tr "\\t" "\\n" > ${prefix}.uniq.fasta

awk '!/>/' ${prefix}.uniq.fasta > ${prefix}.totaluniqseq

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastp: \$(fastp --version 2>&1 | sed -e "s/fastp //g")
    BusyBox: \$(busybox | sed -n -E 's/.*v([[:digit:].]+)\\s\\(.*/\\1/p')
END_VERSIONS
"""    
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
"""
[ ! -f  ${prefix}.fastq.gz ] && ln -sf ${reads} ${prefix}.fastq.gz
fastp \\
    --stdout \\
    --in1 ${prefix}.fastq.gz \\
    --out1  ${prefix}.fastp.fastq.gz \\
    --thread $task.cpus \\
    --json ${prefix}.fastp.json \\
    --html ${prefix}.fastp.html \\
    $fail_fastq \\
    $args \\
    2> ${prefix}.fastp.log
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastp: \$(fastp --version 2>&1 | sed -e "s/fastp //g")
END_VERSIONS
"""
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
"""
[ ! -f  ${prefix}_1.fastq.gz ] && ln -sf ${reads[0]} ${prefix}_1.fastq.gz
[ ! -f  ${prefix}_2.fastq.gz ] && ln -sf ${reads[1]} ${prefix}_2.fastq.gz
fastp \\
    --in1 ${prefix}_1.fastq.gz \\
    --in2 ${prefix}_2.fastq.gz \\
    --out1 ${prefix}_1.fastp.fastq.gz \\
    --out2 ${prefix}_2.fastp.fastq.gz \\
    --json ${prefix}.fastp.json \\
    --html ${prefix}.fastp.html \\
    $fail_fastq \\
    $merge_fastq \\
    --thread $task.cpus \\
    $args \\
    $umi_args \\
    2> ${prefix}.fastp.log
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastp: \$(fastp --version 2>&1 | sed -e "s/fastp //g")
END_VERSIONS
"""
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
"""
[ ! -f  ${prefix}.fastq.gz ] && ln -sf $reads ${prefix}.fastq.gz
fastp \\
    --stdout \\
    --in1 ${prefix}.fastq.gz \\
    --out1  ${prefix}.fastp.fastq.gz \\
    --thread $task.cpus \\
    --json ${prefix}.fastp.json \\
    --html ${prefix}.fastp.html \\
    $fail_fastq \\
    $umi_args \\
    $args \\
    2> ${prefix}.fastp.log
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastp: \$(fastp --version 2>&1 | sed -e "s/fastp //g")
END_VERSIONS
"""  
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
"""
[ ! -f  ${prefix}_1.fastq.gz ] && ln -sf ${reads[0]} ${prefix}_1.fastq.gz
[ ! -f  ${prefix}_2.fastq.gz ] && ln -sf ${reads[1]} ${prefix}_2.fastq.gz
fastp \\
    --in1 ${prefix}_1.fastq.gz \\
    --in2 ${prefix}_2.fastq.gz \\
    --out1 ${prefix}_1.fastp.fastq.gz \\
    --out2 ${prefix}_2.fastp.fastq.gz \\
    --json ${prefix}.fastp.json \\
    --html ${prefix}.fastp.html \\
    $fail_fastq \\
    $merge_fastq \\
    --thread $task.cpus \\
    $args \\
    2> ${prefix}.fastp.log
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastp: \$(fastp --version 2>&1 | sed -e "s/fastp //g")
END_VERSIONS
"""
28
29
30
31
32
33
34
35
36
37
38
"""
printf "%s %s\\n" $rename_to | while read old_name new_name; do
    [ -f "\${new_name}" ] || ln -s \$old_name \$new_name
done
fastqc $args --threads $task.cpus $renamed_files

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastqc: \$( fastqc --version | sed -e "s/FastQC v//g" )
END_VERSIONS
"""
42
43
44
45
46
47
48
49
50
"""
touch ${prefix}.html
touch ${prefix}.zip

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    fastqc: \$( fastqc --version | sed -e "s/FastQC v//g" )
END_VERSIONS
"""
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
"""
freebayes \\
    -f $fasta \\
    $targets_file \\
    $samples_file \\
    $populations_file \\
    $cnv_file \\
    $args \\
    $bam | \\
bgzip -c > ${prefix}${interval}.vcf.gz
cat <<-END_VERSIONS > versions.yml
"${task.process}":
    freebayes: \$(echo \$(freebayes --version 2>&1) | sed 's/version:\s*v//g' )
END_VERSIONS
"""
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
"""
minimap2 \\
    $args \\
    -t $task.cpus \\
    "${reference ?: reads}" \\
    "$reads" \\
    $cigar_paf \\
    $set_cigar_bam \\
    $bam_output


cat <<-END_VERSIONS > versions.yml
"${task.process}":
    minimap2: \$(minimap2 --version 2>&1)
END_VERSIONS
"""
22
23
24
25
26
27
28
29
30
31
32
33
"""
minimap2 \\
    -t $task.cpus \\
    -d ${fasta.baseName}.mmi \\
    $args \\
    $fasta

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    minimap2: \$(minimap2 --version 2>&1)
END_VERSIONS
"""
28
29
30
31
32
33
34
35
36
37
38
39
40
"""
multiqc \\
    --force \\
    $args \\
    $config \\
    $extra_config \\
    .

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    multiqc: \$( multiqc --version | sed -e "s/multiqc, version //g" )
END_VERSIONS
"""
43
44
45
46
47
48
49
50
51
52
"""
touch multiqc_data
touch multiqc_plots
touch multiqc_report.html

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    multiqc: \$( multiqc --version | sed -e "s/multiqc, version //g" )
END_VERSIONS
"""
23
24
25
26
27
28
29
30
31
32
33
"""
samtools \\
    faidx \\
    $args \\
    $fasta

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
36
37
38
39
40
41
42
43
"""
touch ${fasta}.fai
cat <<-END_VERSIONS > versions.yml

"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
NextFlow From line 36 of faidx/main.nf
23
24
25
26
27
28
29
30
31
32
33
34
"""
samtools \\
    flagstat \\
    --threads ${task.cpus} \\
    $bam \\
    > ${prefix}.flagstat

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
24
25
26
27
28
29
30
31
32
33
34
35
"""
samtools \\
    idxstats \\
    --threads ${task.cpus-1} \\
    $bam \\
    > ${prefix}.idxstats

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
24
25
26
27
28
29
30
31
32
33
34
35
"""
samtools \\
    index \\
    -@ ${task.cpus-1} \\
    $args \\
    $input

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
38
39
40
41
42
43
44
45
46
47
"""
touch ${input}.bai
touch ${input}.crai
touch ${input}.csi

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
NextFlow From line 38 of index/main.nf
30
31
32
33
34
35
36
37
38
39
40
41
42
43
"""
samtools \\
    merge \\
    --threads ${task.cpus-1} \\
    $args \\
    ${reference} \\
    ${prefix}.${file_type} \\
    $input_files

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
48
49
50
51
52
53
54
55
"""
touch ${prefix}.${file_type}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
NextFlow From line 48 of merge/main.nf
25
26
27
28
29
30
31
32
33
34
35
36
37
"""
samtools \\
    stats \\
    --threads ${task.cpus} \\
    ${reference} \\
    ${input} \\
    > ${prefix}.stats

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
41
42
43
44
45
46
47
48
"""
touch ${prefix}.stats

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
END_VERSIONS
"""
NextFlow From line 41 of stats/main.nf
28
29
30
31
32
33
34
35
36
37
38
39
"""
seqtk \\
    seq \\
    $args \\
    $fastx | \\
    gzip -c > ${prefix}.seqtk-seq.${extension}.gz

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    seqtk: \$(echo \$(seqtk 2>&1) | sed 's/^.*Version: //; s/ .*\$//')
END_VERSIONS
"""
23
24
25
26
27
28
29
30
"""
tabix $args $tab

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    tabix: \$(echo \$(tabix -h 2>&1) | sed 's/^.*Version: //; s/ .*\$//')
END_VERSIONS
"""
34
35
36
37
38
39
40
41
"""
touch ${tab}.tbi
cat <<-END_VERSIONS > versions.yml

"${task.process}":
    tabix: \$(echo \$(tabix -h 2>&1) | sed 's/^.*Version: //; s/ .*\$//')
END_VERSIONS
"""
NextFlow From line 34 of tabix/main.nf
31
32
33
34
35
36
37
38
39
40
41
42
43
44
"""
PYTHONHASHSEED=0 umi_tools \\
    dedup \\
    -I $bam \\
    -S ${prefix}_dedup.bam \\
    $stats \\
    $paired \\
    $args

cat <<-END_VERSIONS > versions.yml
"${task.process}":
    umitools: \$(umi_tools --version 2>&1 | sed 's/^.*UMI-tools version://; s/ *\$//')
END_VERSIONS
"""
ShowHide 47 more snippets with no or duplicated tags.

Login to post a comment if you would like to share your experience with this workflow.

Do you know this workflow well? If so, you can request seller status , and start supporting this workflow.

Free

Created: 1yr ago
Updated: 1yr ago
Maitainers: public
URL: https://nf-co.re/radseq
Name: radseq
Version: dev
Badge:
workflow icon

Insert copied code into your website to add a link to this workflow.

Downloaded: 0
Copyright: Public Domain
License: None
  • Future updates

Related Workflows

cellranger-snakemake-gke
snakemake workflow to run cellranger on a given bucket using gke.
A Snakemake workflow for running cellranger on a given bucket using Google Kubernetes Engine. The usage of this workflow ...