SPAdes 3.1.1 Manual

1. About SPAdes

SPAdes – St. Petersburg genome assembler – is intended for both standard isolates and single-cell MDA bacteria assemblies. This manual will help you to install and run SPAdes. SPAdes version 3.1.1 was released under GPLv2 on May 28, 2014 and can be downloaded from http://bioinf.spbau.ru/en/spades.

1.1 Supported data types

The current version of SPAdes works with Illumina or IonTorrent reads and is capable of providing hybrid assemblies using PacBio and Sanger reads. You can also provide additional contigs that will be used as long reads.

Version 3.1.1 of SPAdes supports paired-end reads, mate-pairs and unpaired reads. SPAdes can take as input several paired-end and mate-pair libraries simultaneously.

Note, that SPAdes was initially designed for small genomes. It was tested on single-cell and standard bacterial and fungal data sets. SPAdes is not intended for larger genomes (e.g. mammalian size genomes) and metagenomic projects. For such purposes you can use it at your own risk.

SPAdes has also a separate module for assembling highly polymorphic diploid genomes. For more information see dipSPAdes manual.

1.2 SPAdes pipeline

SPAdes comes in several separate modules:

• BayesHammer – read error correction tool for Illumina reads, which works well on both single-cell and standard data sets.
• IonHammer – read error correction tool for IonTorrent data, which also works on both types of data.
• SPAdes – iterative short-read genome assembly module; values of K are selected automatically based on the read length and data set type.
• MismatchCorrector – a tool which improves mismatch and short indel rates in resulting contigs and scaffolds; this module uses the BWA tool [Li H. and Durbin R., 2009]; MismatchCorrector is turned off by default, but we recommend to turn it on (see SPAdes options section).
• dipSPAdes – module for assembling highly polymorphic diploid genomes; for more information see dipSPAdes manual.

We recommend to run SPAdes with BayesHammer/IonHammer to obtain high-quality assemblies. However, if you use your own read correction tool, it is possible to turn error correction module off. It is also possible to use only the read error correction stage, if you wish to use another assembler. See the SPAdes options section.

1.3 SPAdes' performance

In this section we give approximate data about SPAdes' performance on two data sets:

• Standard isolate E. coli; 6.2Gb, 28M reads, 2x100bp, insert size ~ 215bp
• MDA single-cell E. coli; 6.3 Gb, 29M reads, 2x100bp, insert size ~ 270bp

We ran SPAdes with default parameters using 16 threads on a server with Intel Xeon 2.27GHz processors. BayesHammer runs in approximately 1 hour and takes up to 15Gb of RAM to perform read error correction on each data set. Assembly takes 35 minutes and 4Gb of RAM for the E. coli isolate data set. The E. coli single-cell data set takes 55 minutes and 6Gb of RAM. MismatchCorrector runs for about an hour on standard E. coli, about 2 hours on the single-cell data set, and requires 8Gb of RAM. All modules also require additional disk space for temporary files. See the table below for more precise values.

Data set	E. coli isolate			E. coli single-cell
Stage	Time (min)	Peak RAM     usage (Gb)	Additional     disk space (Gb)	Time (min)	Peak RAM     usage (Gb)	Additional     disk space (Gb)
BayesHammer	60	10	10	75	10	10
SPAdes	55	3	6	75	5	8
MismatchCorrector	60	9	21	120	9	21

Notes:

• Running SPAdes without preliminary read error correction (e.g. without BayesHammer or IonHammer) will likely require more time and memory.
• Each module removes its temporary files as soon as it finishes.
• SPAdes 3.1 uses 512 Mb per thread for buffers, thus memory consumption may be greater. However, if you set memory limit manually SPAdes will use smaller buffers.

2. Installation

SPAdes requires a 64-bit Linux system or Mac OS and Python (supported versions are 2.4, 2.5, 2.6, 2.7, 3.2 and 3.3) to be pre-installed on it. To obtain SPAdes you can either download binaries or download source code and compile it yourself.

2.1 Downloading SPAdes Linux binaries

To download SPAdes Linux binaries and extract them, go to the directory in which you wish SPAdes to be installed and run:

    wget http://spades.bioinf.spbau.ru/release3.1.1/SPAdes-3.1.1-Linux.tar.gz
    tar -xzf SPAdes-3.1.1-Linux.tar.gz
    cd SPAdes-3.1.1-Linux/bin/

In this case you do not need to run any installation scripts – SPAdes is ready to use. The following files will be placed in the bin directory:

• spades.py (main executable script)
• hammer (read error correcting module for Illumina reads)
• ionhammer (read error correcting module for IonTorrent reads)
• spades (assembly module)
• bwa-spades (BWA alignment module which is required for mismatch correction)
• dipspades.py (main executable script for dipSPAdes)
• dipspades (assembly module for highly polymorphic diploid genomes)

We also suggest adding SPAdes installation directory to the PATH variable.

2.2 Downloading SPAdes binaries for Mac

To obtain SPAdes binaries for Mac, go to the directory in which you wish SPAdes to be installed and run:

    curl http://spades.bioinf.spbau.ru/release3.1.1/SPAdes-3.1.1-Darwin.tar.gz -o SPAdes-3.1.1-Darwin.tar.gz
    tar -zxf SPAdes-3.1.1-Darwin.tar.gz
    cd SPAdes-3.1.1-Darwin/bin/

Just as in Linux, SPAdes is ready to use and no further installation steps are required. You will get the same files in the bin directory:

• spades.py (main executable script)
• hammer (read error correcting module for Illumina reads)
• ionhammer (read error correcting module for IonTorrent reads)
• spades (assembly module)
• bwa-spades (BWA alignment module which is required for mismatch correction)
• dipspades.py (main executable script for dipSPAdes)
• dipspades (assembly module for highly polymorphic diploid genomes)

We also suggest adding SPAdes installation directory to the PATH variable.

2.3 Downloading and compiling SPAdes source code

If you wish to compile SPAdes by yourself you will need the following libraries to be pre-installed:

• g++ (version 4.7 or higher)
• cmake (version 2.8.8 or higher)
• zlib
• libbz2

If you meet these requirements, you can download the SPAdes source code:

    wget http://spades.bioinf.spbau.ru/release3.1.1/SPAdes-3.1.1.tar.gz
    tar -xzf SPAdes-3.1.1.tar.gz
    cd SPAdes-3.1.1

and build it with the following script:

    ./spades_compile.sh

SPAdes will be built in the directory ./bin. If you wish to install SPAdes into another directory, you can specify full path of destination folder by running the following command in bash or sh:

    PREFIX=<destination_dir> ./spades_compile.sh

for example:

    PREFIX=/usr/local ./spades_compile.sh

which will install SPAdes into /usr/local/bin.

After installation you will get the same files in ./bin (or <destination_dir>/bin if you specified PREFIX) directory:

• spades.py (main executable script)
• hammer (read error correcting module for Illumina reads)
• ionhammer (read error correcting module for IonTorrent reads)
• spades (assembly module)
• bwa-spades (BWA alignment module which is required for mismatch correction)
• dipspades.py (main executable script for dipSPAdes)
• dipspades (assembly module for highly polymorphic diploid genomes)

We also suggest adding SPAdes installation directory to the PATH variable.

2.4 Verifying your installation

For testing purposes, SPAdes comes with a toy data set (reads that align to first 1000 bp of E. coli). To try SPAdes on this data set, run:

    <spades installation dir>/spades.py --test

If you added SPAdes installation directory to the PATH variable, you can run:

    spades.py --test

If the installation is successful, you will find the following information at the end of the log:

===== Assembling finished. 

 * Corrected reads are in spades_test/corrected/
 * Assembled contigs are in spades_test/contigs.fasta (contigs.fastg)
 * Assembled scaffolds are in spades_test/scaffolds.fasta (scaffolds.fastg)

======= SPAdes pipeline finished.

SPAdes log can be found here: /home/andrey/ablab/algorithmic-biology/assembler/spades_test/spades.log

Thank you for using SPAdes!

3. Running SPAdes

3.1 SPAdes input

SPAdes takes as input paired-end reads, mate-pairs and single (unpaired) reads in FASTA and FASTQ. For IonTorrent data SPAdes also supports unpaired reads in unmapped BAM format (like the one produced by Torrent Server). However, in order to run read error correction, reads should be in FASTQ or BAM format. Sanger and PacBio CCS reads can be provided in both formats since SPAdes does not run error correction for these types of data.

To run SPAdes 3.1.1 you need at least one library of the following types:

• Illumina paired-end/high-quality mate-pairs/unpaired reads
• IonTorrent paired-end/high-quality mate-pairs/unpaired reads
• PacBio CCS reads

Illumina and IonTorrent libraries should not be assembled together. All other types of input data are compatible. SPAdes should not be used if only PacBio CLR reads, Sanger reads or additional contigs are available.

SPAdes supports mate-pair only assembly. However, we recommend to use only high-quality mate-pair libraries in this case (e.g. that do not have a paired-end part). We tested mate-pair only pipeline using Illumina Nextera mate-pairs. See more here.

Notes:

• It is not recommended to run SPAdes on PacBio reads with low coverage (less than 5).
• We suggest not to run SPAdes on PacBio reads for large genomes.
• SPAdes accepts gzip-compressed files.

Read-pair libraries

By using command line interface, you can specify up to five different paired-end libraries, up to five mate-pair libraries and also up to five high-quality mate-pair ones. If you wish to use more, you can use YAML data set file. We further refer to paired-end and mate-pair libraries simply as to read-pair libraries.

By default, SPAdes assumes that paired-end reads have forward-reverse (fr) orientation and mate-pairs have reverse-forward (rf) orientation. However, different orientations can be set for any library by using SPAdes options.

To distinguish reads in pairs we refer to them as left and right reads. For forward-reverse orientation, the forward reads correspond to the left reads and the reverse reads, to the right. Similarly, in reverse-forward orientation left and right reads correspond to reverse and forward reads, respectively, etc.

Each read-pair library can be stored in several files or several pairs of files. Paired reads can be organized in two different ways:

• In file pairs. In this case left and right reads are placed in different files and go in the same order in respective files.
• In merged files. In this case, the reads are interlaced, so that each right read goes after the corresponding paired left read.

For example, Illumina produces paired-end reads in two files: s_1_1_sequence.txt and s_1_2_sequence.txt. If you choose to store reads in file pairs make sure that for every read from s_1_1_sequence.txt the corresponding paired read from s_1_2_sequence.txt is placed in the respective paired file on the same line number. If you choose to use merged files, every read from s_1_1_sequence.txt should be followed by the corresponding paired read from s_1_2_sequence.txt.

Unpaired (single-read) libraries

By using command line interface, you can specify up to five different single-read libraries. To input more libraries, you can use YAML data set file.

Single librairies are assumed to have high quality and a reasonable coverage. For example, you can provide PacBio CSS reads as a single-read library. Additionally, if you have merged a paired-end library with overlapping read-pairs (for example, using FLASh), you can provide the resulting reads as a single-read library.

Note, that you should not specify PacBio CLR, Sanger reads or additional contigs as single-read libraries, each of them has a separate option.

PacBio reads

SPAdes can take as an input an unlimited number of PacBio libraries.

PacBio CLR reads are used for hybrid assemblies (e.g. with Illumina or IonTorrent). There is no need to pre-correct PacBio CLR reads. You just need to have filtered subreads in FASTQ/FASTA format. Provide these filtered subreads using --pacbio option. SPAdes will use PacBio CLR reads for gap closure and repeat resolution.

PacBio CCS reads or pre-corrected PacBio CLR reads can be simply provided as single reads to SPAdes.

Additional contigs

In case you have contigs of the same genome generated by other assembler(s) and you wish to merge them into SPAdes assembly, you can specify additional contigs using --trusted-contigs or --untrusted-contigs. First option is used when high quality contigs are available. These contigs will be used for graph construction, gap closure and repeat resolution. Second option is used for less reliable contigs that may have more errors or contigs of unknown quality. These contigs will be used only for gap closure and repeat resolution. The number of additional contigs is unlimited.

Note, that SPAdes does not perform assembly using genomes of closely-related species. Only contigs of the same genome should be specified.

3.2 SPAdes command line options

To run SPAdes from the command line, type

    spades.py [options] -o <output_dir>

Basic options

-o <output_dir>
    Specify the output directory. Required option.

--sc
    This flag is required for MDA (single-cell) data.

--iontorrent
    This flag is required when assembling IonTorrent data. Allows BAM files as input. Carefully read section 3.3 before using this option.

--test
    Runs SPAdes on the toy data set; see section 2.3.

-h (or --help)
    Prints help.

--careful
    Tries to reduce the number of mismatches and short indels. Also runs MismatchCorrector – a post processing tool, which uses BWA tool (comes with SPAdes). This option is recommended.

--continue
    Continues SPAdes run from the specified output folder starting from the last available check-point. Check-points are made after:

• error correction module is finished
• iteration for each specified K value of assembly module is finished
• mismatch correction is finished for contigs or scaffolds

--restart-from <check_point>
    Restart SPAdes run from the specified output folder starting from the specified check-point. Check-points are:

• ec – start from error correction
• as – restart assembly module from the first iteration
• k<int> – restart from the iteration with specified k values, e.g. k55
• mc – restart mismatch correction

--disable-gzip-output
    Forces read error correction module not to compress the corrected reads. If this options is not set, corrected reads will be in *.fastq.gz format.

Input data

--12 <file_name>
    File with interlaced forward and reverse paired-end reads.

-1 <file_name>
    File with forward reads.

-2 <file_name>
    File with reverse reads.

-s <file_name>
    File with unpaired reads.

•  Single-read libraries

--s<#> <file_name>
    File for single-read library number <#> (<#> = 1,2,3,4,5). For example, for the first paired-end library the option is: --s1 <file_name>
    Do not use -s options for single-read libraries, since it specifies unpaired reads for the first paired-end library.

•  Paired-end libraries

--pe<#>-12 <file_name>
    File with interlaced reads for paired-end library number <#> (<#> = 1,2,3,4,5). For example, for the first single-read library the option is: --pe1-12 <file_name>


--pe<#>-1 <file_name>
    File with left reads for paired-end library number <#> (<#> = 1,2,3,4,5).

--pe<#>-2 <file_name>
    File with right reads for paired-end library number <#> (<#> = 1,2,3,4,5).

--pe<#>-s <file_name>
    File with unpaired reads from paired-end library number <#> (<#> = 1,2,3,4,5)
    For example, paired reads can become unpaired during the error correction procedure.

--pe<#>-<or> <file_name>
    Orientation of reads for paired-end library number <#> (<#> = 1,2,3,4,5; <or> = "fr","rf","ff").
    The default orientation for paired-end libraries is forward-reverse. For example, to specify reverse-forward orientation for the second paired-end library, you should use the flag: --pe2-rf


•  Mate-pair libraries

--mp<#>-12 <file_name>
    File with interlaced reads for mate-pair library number <#> (<#> = 1,2,3,4,5).

--mp<#>-1 <file_name>
    File with left reads for mate-pair library number <#> (<#> = 1,2,3,4,5).

--mp<#>-2 <file_name>
    File with right reads for mate-pair library number <#> (<#> = 1,2,3,4,5).

--mp<#>-<or> <file_name>
    Orientation of reads for mate-pair library number <#> (<#> = 1,2,3,4,5; <or> = "fr","rf","ff").
    The default orientation for mate-pair libraries is reverse-forward. For example, to specify forward-forward orientation for the first mate-pair library, you should use the flag: --mp1-ff


•  High-quality mate-pair libraries (can be used for mate-pair only assembly)

--hqmp<#>-12 <file_name>
    File with interlaced reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5).

--hqmp<#>-1 <file_name>
    File with left reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5).

--hqmp<#>-2 <file_name>
    File with right reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5).

--hqmp<#>-s <file_name>
    File with unpaired reads from high-quality mate-pair library number <#> (<#> = 1,2,3,4,5)


--hqmp<#>-<or> <file_name>
    Orientation of reads for high-quality mate-pair library number <#> (<#> = 1,2,3,4,5; <or> = "fr","rf","ff").
    The default orientation for high-quality mate-pair libraries is also reverse-forward. For example, to specify forward-reverse orientation for the first high-quality mate-pair library, you should use the flag: --hqmp1-fr

--pacbio <file_name>
    File with PacBio CLR reads. For PacBio CCS reads use -s option. More information on PacBio reads is provided in section 3.1.

--sanger <file_name>
    File with Sanger reads

--trusted-contigs <file_name>
    Reliable contigs of the same genome, which are likely to have no misassemblies and small rate of other errors (e.g. mismatches and indels). This option is not intended for contigs of the related species.

--untrusted-contigs <file_name>
    Contigs of the same genome, quality of which is average or unknown. Contigs of poor quality can be used but may introduce errors in the assembly. This option is also not intended for contigs of the related species.

An alternative way to specify an input data set for SPAdes is to create a YAML data set file. By using a YAML file you can provide an unlimited number of paired-end, mate-pair and unpaired libraries. Basically, YAML data set file is a text file, in which input libraries are provided as a comma-separated list in square brackets. Each library is provided in braces as a comma-separated list of attributes. The following attributes are available:

• orientation ("fr", "rf", "ff")
• type ("paired-end", "mate-pairs", "hq-mate-pairs", "single", "pacbio", "sanger", "trusted-contigs", "untrusted-contigs")
• interlaced reads (comma-separated list of files with interlaced reads)
• left reads (comma-separated list of files with left reads)
• right reads (comma-separated list of files with right reads)
• single reads (comma-separated list of files with single reads)

To properly specify a library you should provide its type and at least one file with reads. Orientation is an optional attribute. Its default value is "fr" (forward-reverse) for paired-end libraries and "rf" (reverse-forward) for mate-pair libraries.

The value for each attribute is given after a colon. Comma-separated lists of files should be given in square brackets. For each file you should provide its full path in double quotes. Make sure that files with right reads are given in the same order as corresponding files with left reads.

For example, if you have one paired-end library splitted into two pairs of files:

    lib_pe1_left_1.fastq
    lib_pe1_right_1.fastq
    lib_pe1_left_2.fastq
    lib_pe1_right_2.fastq

    lib_mp1_left.fastq
    lib_mp1_right.fastq

    pacbio_css.fastq
    pacbio_clr.fastq

    [
      {
        orientation: "fr",
        type: "paired-end",
        right reads: [
          "/FULL_PATH_TO_DATASET/lib_pe1_right_1.fastq",
          "/FULL_PATH_TO_DATASET/lib_pe1_right_2.fastq" 
        ],
        left reads: [
          "/FULL_PATH_TO_DATASET/lib_pe1_left_1.fastq",
          "/FULL_PATH_TO_DATASET/lib_pe1_left_2.fastq" 
        ]
      },
      {
        orientation: "rf",
        type: "mate-pairs",
        right reads: [
          "/FULL_PATH_TO_DATASET/lib_mp1_right.fastq" 
        ],
        left reads: [
          "/FULL_PATH_TO_DATASET/lib_mp1_left.fastq"
        ]
      },
      {
        type: "single",
        single reads: [
          "/FULL_PATH_TO_DATASET/pacbio_ccs.fastq" 
        ]
      },
      {
        type: "pacbio",
        single reads: [
          "/FULL_PATH_TO_DATASET/pacbio_clr.fastq" 
        ]
      }
    ]

Once you have created a YAML file save it with .yaml extension (e.g. as my_data_set.yaml) and run SPAdes using the --dataset option:
--dataset <your YAML file>

Note that the --dataset option cannot be used with any other options for specifying input data.

Advanced options

-k <int,int,...>
    Comma-separated list of k-mer sizes to be used (all values must be odd, less than 128 and listed in ascending order). If --sc is set the default value are 21,33,55. For multicell data sets K values are automatically selected using maximum read length (see note for assembling long Illumina paired reads for details). To properly select K values for IonTorrent data read section 3.3.

--phred-offset <33 or 64>
    PHRED quality offset for the input reads, can be either 33 or 64. It will be auto-detected if it is not specified.

    spades.py --pe1-1 ../share/spades/test_dataset/ecoli_1K_1.fq.gz \
    --pe1-2 ../share/spades/test_dataset/ecoli_1K_2.fq.gz -o spades_test

    lib1_forward_1.fastq
    lib1_reverse_1.fastq
    lib1_forward_2.fastq
    lib1_reverse_2.fastq

    spades.py --pe1-1 lib1_forward_1.fastq --pe1-2 lib1_reverse_1.fastq \
    --pe1-1 lib1_forward_2.fastq --pe1-2 lib1_reverse_2.fastq \
    -o spades_output

    spades.py --pe1-12 lib1_1.fastq --pe1-12 lib1_2.fastq \
    --pe1-s lib1_unpaired_1.fastq --pe1-s lib1_unpaired_2.fastq \
    -o spades_output    

    lib_pe1_left.fastq
    lib_pe1_right.fastq

    lib_mp1_left.fastq
    lib_mp1_right.fastq

    lib_mp2_left.fastq
    lib_mp2_right.fastq

    spades.py --pe1-1 lib_pe1_left.fastq --pe1-2 lib_pe1_right.fastq \
    --mp1-1 lib_mp1_left.fastq --mp1-2 lib_mp1_right.fastq \
    --mp2-1 lib_mp2_left.fastq --mp2-2 lib_mp2_right.fastq \
    -o spades_output

    it_reads.fastq
    pacbio_clr.fastq
    contigs.fasta

    spades.py --iontorrent -s it_reads.fastq \
    --pacbio pacbio_clr.fastq --trusted-contigs contigs.fastq \
    -o spades_output

    unpaired1_1.fastq
    unpaired1_2.fastq
    unpaired1_3.fasta

    spades.py --s1 unpaired1_1.fastq \
    --s1 unpaired1_2.fastq --s1 unpaired1_3.fastq \
    -o spades_output

    spades.py -k 21,33,55,77 --careful --only-assembler <your reads> -o spades_output

    spades.py -k 21,33,55,77 --careful <your reads> -o spades_output

    spades.py -k 21,33,55,77,99,127 --careful --only-assembler <your reads> -o spades_output

    spades.py -k 21,33,55,77,99,127 --careful <your reads> -o spades_output

    contigs.fasta, contigs.fastg – resulting contigs
    scaffolds.fasta, scaffolds.fastg – resulting scaffolds 

    corrected/ – files from read error correction
        configs/ – configuration files for read error correction
        corrected.yaml – internal configuration file
        Output files with corrected reads

    params.txt – information about SPAdes parameters in this run
    spades.log – SPAdes log
    dataset.info – internal configuration file
    input_dataset.yaml – internal YAML data set file
    K<##>/ – directory containing files from the run with K=<##>