needle Wiki The master copies of EMBOSS documentation are available at http://emboss.open-bio.org/wiki/Appdocs on the EMBOSS Wiki. Please help by correcting and extending the Wiki pages. Function Needleman-Wunsch global alignment of two sequences Description needle reads two input sequences and writes their optimal global sequence alignment to file. It uses the Needleman-Wunsch alignment algorithm to find the optimum alignment (including gaps) of two sequences along their entire length. The algorithm uses a dynamic programming method to ensure the alignment is optimum, by exploring all possible alignments and choosing the best. A scoring matrix is read that contains values for every possible residue or nucleotide match. Needle finds the alignment with the maximum possible score where the score of an alignment is equal to the sum of the matches taken from the scoring matrix, minus penalties arising from opening and extending gaps in the aligned sequences. The substitution matrix and gap opening and extension penalties are user-specified. Algorithm The Needleman-Wunsch algorithm is a member of the class of algorithms that can calculate the best score and alignment of two sequences in the order of mn steps, where n and m are the sequence lengths. These dynamic programming algorithms were first developed for protein sequence comparison by Needleman and Wunsch, though similar methods were independently devised during the late 1960's and early 1970's for use in the fields of speech processing and computer science. An important problem is the treatment of gaps, i.e., spaces inserted to optimise the alignment score. A penalty is subtracted from the score for each gap opened (the 'gap open' penalty) and a penalty is subtracted from the score for the total number of gap spaces multiplied by a cost (the 'gap extension' penalty). Typically, the cost of extending a gap is set to be 5-10 times lower than the cost for opening a gap. Penalty for a gap of n positions is calculated using the following formula: gap opening penalty + (n - 1) * gap extension penalty In a Needleman-Wunsch global alignment, the entire length of each sequence is aligned. The sequences might be partially overlapping or one sequence might be aligned entirely internally to the other. There is no penalty for the hanging ends of the overlap. In bioinformatics, it is usually reasonable to assume that the sequences are incomplete and there should be no penalty for failing to align the missing bases. Usage Here is a sample session with needle % needle tsw:hba_human tsw:hbb_human Needleman-Wunsch global alignment of two sequences Gap opening penalty [10.0]: Gap extension penalty [0.5]: Output alignment [hba_human.needle]: Go to the input files for this example Go to the output files for this example Command line arguments Needleman-Wunsch global alignment of two sequences Version: EMBOSS:6.4.0.0 Standard (Mandatory) qualifiers: [-asequence] sequence Sequence filename and optional format, or reference (input USA) [-bsequence] seqall Sequence(s) filename and optional format, or reference (input USA) -gapopen float [10.0 for any sequence] The gap open penalty is the score taken away when a gap is created. The best value depends on the choice of comparison matrix. The default value assumes you are using the EBLOSUM62 matrix for protein sequences, and the EDNAFULL matrix for nucleotide sequences. (Floating point number from 1.0 to 100.0) -gapextend float [0.5 for any sequence] The gap extension, penalty is added to the standard gap penalty for each base or residue in the gap. This is how long gaps are penalized. Usually you will expect a few long gaps rather than many short gaps, so the gap extension penalty should be lower than the gap penalty. An exception is where one or both sequences are single reads with possible sequencing errors in which case you would expect many single base gaps. You can get this result by setting the gap open penalty to zero (or very low) and using the gap extension penalty to control gap scoring. (Floating point number from 0.0 to 10.0) [-outfile] align [*.needle] Output alignment file name (default -aformat srspair) Additional (Optional) qualifiers: -datafile matrixf [EBLOSUM62 for protein, EDNAFULL for DNA] This is the scoring matrix file used when comparing sequences. By default it is the file 'EBLOSUM62' (for proteins) or the file 'EDNAFULL' (for nucleic sequences). These files are found in the 'data' directory of the EMBOSS installation. -endweight boolean [N] Apply end gap penalties. -endopen float [10.0 for any sequence] The end gap open penalty is the score taken away when an end gap is created. The best value depends on the choice of comparison matrix. The default value assumes you are using the EBLOSUM62 matrix for protein sequences, and the EDNAFULL matrix for nucleotide sequences. (Floating point number from 1.0 to 100.0) -endextend float [0.5 for any sequence] The end gap extension, penalty is added to the end gap penalty for each base or residue in the end gap. This is how long end gaps are penalized. (Floating point number from 0.0 to 10.0) Advanced (Unprompted) qualifiers: -[no]brief boolean [Y] Brief identity and similarity Associated qualifiers: "-asequence" associated qualifiers -sbegin1 integer Start of the sequence to be used -send1 integer End of the sequence to be used -sreverse1 boolean Reverse (if DNA) -sask1 boolean Ask for begin/end/reverse -snucleotide1 boolean Sequence is nucleotide -sprotein1 boolean Sequence is protein -slower1 boolean Make lower case -supper1 boolean Make upper case -sformat1 string Input sequence format -sdbname1 string Database name -sid1 string Entryname -ufo1 string UFO features -fformat1 string Features format -fopenfile1 string Features file name "-bsequence" associated qualifiers -sbegin2 integer Start of each sequence to be used -send2 integer End of each sequence to be used -sreverse2 boolean Reverse (if DNA) -sask2 boolean Ask for begin/end/reverse -snucleotide2 boolean Sequence is nucleotide -sprotein2 boolean Sequence is protein -slower2 boolean Make lower case -supper2 boolean Make upper case -sformat2 string Input sequence format -sdbname2 string Database name -sid2 string Entryname -ufo2 string UFO features -fformat2 string Features format -fopenfile2 string Features file name "-outfile" associated qualifiers -aformat3 string Alignment format -aextension3 string File name extension -adirectory3 string Output directory -aname3 string Base file name -awidth3 integer Alignment width -aaccshow3 boolean Show accession number in the header -adesshow3 boolean Show description in the header -ausashow3 boolean Show the full USA in the alignment -aglobal3 boolean Show the full sequence in alignment General qualifiers: -auto boolean Turn off prompts -stdout boolean Write first file to standard output -filter boolean Read first file from standard input, write first file to standard output -options boolean Prompt for standard and additional values -debug boolean Write debug output to program.dbg -verbose boolean Report some/full command line options -help boolean Report command line options and exit. More information on associated and general qualifiers can be found with -help -verbose -warning boolean Report warnings -error boolean Report errors -fatal boolean Report fatal errors -die boolean Report dying program messages -version boolean Report version number and exit Input file format needle reads in two nucleotide or protein sequences. The second input can be more than one sequence to align to the first input sequence. The input is a standard EMBOSS sequence query (also known as a 'USA'). Major sequence database sources defined as standard in EMBOSS installations include srs:embl, srs:uniprot and ensembl Data can also be read from sequence output in any supported format written by an EMBOSS or third-party application. The input format can be specified by using the command-line qualifier -sformat xxx, where 'xxx' is replaced by the name of the required format. The available format names are: gff (gff3), gff2, embl (em), genbank (gb, refseq), ddbj, refseqp, pir (nbrf), swissprot (swiss, sw), dasgff and debug. See: http://emboss.sf.net/docs/themes/SequenceFormats.html for further information on sequence formats. Input files for usage example 'tsw:hba_human' is a sequence entry in the example protein database 'tsw' Database entry: tsw:hba_human ID HBA_HUMAN Reviewed; 142 AA. AC P69905; P01922; Q1HDT5; Q3MIF5; Q53F97; Q96KF1; Q9NYR7; Q9UCM0; DT 21-JUL-1986, integrated into UniProtKB/Swiss-Prot. DT 23-JAN-2007, sequence version 2. DT 15-JUN-2010, entry version 86. DE RecName: Full=Hemoglobin subunit alpha; DE AltName: Full=Hemoglobin alpha chain; DE AltName: Full=Alpha-globin; GN Name=HBA1; GN and GN Name=HBA2; OS Homo sapiens (Human). OC Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi; OC Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini; OC Catarrhini; Hominidae; Homo. OX NCBI_TaxID=9606; RN [1] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA] (HBA1). RX MEDLINE=81088339; PubMed=7448866; DOI=10.1016/0092-8674(80)90347-5; RA Michelson A.M., Orkin S.H.; RT "The 3' untranslated regions of the duplicated human alpha-globin RT genes are unexpectedly divergent."; RL Cell 22:371-377(1980). RN [2] RP NUCLEOTIDE SEQUENCE [MRNA] (HBA2). RX MEDLINE=80137531; PubMed=6244294; RA Wilson J.T., Wilson L.B., Reddy V.B., Cavallesco C., Ghosh P.K., RA Deriel J.K., Forget B.G., Weissman S.M.; RT "Nucleotide sequence of the coding portion of human alpha globin RT messenger RNA."; RL J. Biol. Chem. 255:2807-2815(1980). RN [3] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA] (HBA2). RX MEDLINE=81175088; PubMed=6452630; DOI=10.1073/pnas.77.12.7054; RA Liebhaber S.A., Goossens M.J., Kan Y.W.; RT "Cloning and complete nucleotide sequence of human 5'-alpha-globin RT gene."; RL Proc. Natl. Acad. Sci. U.S.A. 77:7054-7058(1980). RN [4] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RX PubMed=6946451; DOI=10.1073/pnas.78.8.5041; RA Orkin S.H., Goff S.C., Hechtman R.L.; RT "Mutation in an intervening sequence splice junction in man."; RL Proc. Natl. Acad. Sci. U.S.A. 78:5041-5045(1981). RN [5] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA], AND VARIANT LYS-32. RX MEDLINE=21303311; PubMed=11410421; RA Zhao Y., Xu X.; RT "Alpha2(CD31 AGG-->AAG, Arg-->Lys) causing non-deletional alpha- RT thalassemia in a Chinese family with HbH disease."; [Part of this file has been deleted for brevity] FT /FTId=VAR_002841. FT VARIANT 132 132 S -> P (in Questembert; highly unstable; FT causes alpha-thalassemia). FT /FTId=VAR_002843. FT VARIANT 134 134 S -> R (in Val de Marne; O(2) affinity FT up). FT /FTId=VAR_002844. FT VARIANT 136 136 V -> E (in Pavie). FT /FTId=VAR_002845. FT VARIANT 137 137 L -> M (in Chicago). FT /FTId=VAR_002846. FT VARIANT 137 137 L -> P (in Bibba; unstable; causes alpha- FT thalassemia). FT /FTId=VAR_002847. FT VARIANT 137 137 L -> R (in Toyama). FT /FTId=VAR_035242. FT VARIANT 139 139 S -> P (in Attleboro; O(2) affinity up). FT /FTId=VAR_002848. FT VARIANT 140 140 K -> E (in Hanamaki; O(2) affinity up). FT /FTId=VAR_002849. FT VARIANT 140 140 K -> T (in Tokoname; O(2) affinity up). FT /FTId=VAR_002850. FT VARIANT 141 141 Y -> H (in Rouen/Ethiopia; O(2) affinity FT up). FT /FTId=VAR_002851. FT VARIANT 142 142 R -> C (in Nunobiki; O(2) affinity up). FT /FTId=VAR_002852. FT VARIANT 142 142 R -> H (in Suresnes; O(2) affinity up). FT /FTId=VAR_002854. FT VARIANT 142 142 R -> L (in Legnano; O(2) affinity up). FT /FTId=VAR_002853. FT VARIANT 142 142 R -> P (in Singapore). FT /FTId=VAR_002855. FT CONFLICT 10 10 N -> H (in Ref. 13; BAD97112). FT HELIX 5 16 FT HELIX 17 21 FT HELIX 22 36 FT HELIX 38 43 FT HELIX 54 72 FT HELIX 74 76 FT HELIX 77 80 FT HELIX 82 90 FT HELIX 97 113 FT TURN 115 117 FT HELIX 120 137 FT TURN 138 140 SQ SEQUENCE 142 AA; 15258 MW; 15E13666573BBBAE CRC64; MVLSPADKTN VKAAWGKVGA HAGEYGAEAL ERMFLSFPTT KTYFPHFDLS HGSAQVKGHG KKVADALTNA VAHVDDMPNA LSALSDLHAH KLRVDPVNFK LLSHCLLVTL AAHLPAEFTP AVHASLDKFL ASVSTVLTSK YR // Database entry: tsw:hbb_human ID HBB_HUMAN Reviewed; 147 AA. AC P68871; A4GX73; B2ZUE0; P02023; Q13852; Q14481; Q14510; Q45KT0; AC Q549N7; Q6FI08; Q6R7N2; Q8IZI1; Q9BX96; Q9UCD6; Q9UCP8; Q9UCP9; DT 21-JUL-1986, integrated into UniProtKB/Swiss-Prot. DT 23-JAN-2007, sequence version 2. DT 15-JUN-2010, entry version 88. DE RecName: Full=Hemoglobin subunit beta; DE AltName: Full=Hemoglobin beta chain; DE AltName: Full=Beta-globin; DE Contains: DE RecName: Full=LVV-hemorphin-7; GN Name=HBB; OS Homo sapiens (Human). OC Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi; OC Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini; OC Catarrhini; Hominidae; Homo. OX NCBI_TaxID=9606; RN [1] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RX MEDLINE=77126403; PubMed=1019344; RA Marotta C., Forget B., Cohen-Solal M., Weissman S.M.; RT "Nucleotide sequence analysis of coding and noncoding regions of human RT beta-globin mRNA."; RL Prog. Nucleic Acid Res. Mol. Biol. 19:165-175(1976). RN [2] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RX MEDLINE=81064667; PubMed=6254664; DOI=10.1016/0092-8674(80)90428-6; RA Lawn R.M., Efstratiadis A., O'Connell C., Maniatis T.; RT "The nucleotide sequence of the human beta-globin gene."; RL Cell 21:647-651(1980). RN [3] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA], AND VARIANT LYS-7. RX PubMed=16175509; DOI=10.1086/491748; RA Wood E.T., Stover D.A., Slatkin M., Nachman M.W., Hammer M.F.; RT "The beta-globin recombinational hotspot reduces the effects of strong RT selection around HbC, a recently arisen mutation providing resistance RT to malaria."; RL Am. J. Hum. Genet. 77:637-642(2005). RN [4] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RA Lu L., Hu Z.H., Du C.S., Fu Y.S.; RT "DNA sequence of the human beta-globin gene isolated from a healthy RT Chinese."; RL Submitted (JUN-1997) to the EMBL/GenBank/DDBJ databases. RN [5] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA], AND VARIANT ARG-113. RA Cabeda J.M., Correia C., Estevinho A., Cardoso C., Amorim M.L., RA Cleto E., Vale L., Coimbra E., Pinho L., Justica B.; RT "Unexpected patterns of globin mutations in thalassemia patients from RT north of Portugal."; [Part of this file has been deleted for brevity] FT VARIANT 141 141 A -> V (in Puttelange; polycythemia; O(2) FT affinity up). FT /FTId=VAR_003082. FT VARIANT 142 142 L -> R (in Olmsted; unstable). FT /FTId=VAR_003083. FT VARIANT 143 143 A -> D (in Ohio; O(2) affinity up). FT /FTId=VAR_003084. FT VARIANT 144 144 H -> D (in Rancho Mirage). FT /FTId=VAR_003085. FT VARIANT 144 144 H -> P (in Syracuse; O(2) affinity up). FT /FTId=VAR_003087. FT VARIANT 144 144 H -> Q (in Little Rock; O(2) affinity FT up). FT /FTId=VAR_003086. FT VARIANT 144 144 H -> R (in Abruzzo; O(2) affinity up). FT /FTId=VAR_003088. FT VARIANT 145 145 K -> E (in Mito; O(2) affinity up). FT /FTId=VAR_003089. FT VARIANT 146 146 Y -> C (in Rainier; O(2) affinity up). FT /FTId=VAR_003090. FT VARIANT 146 146 Y -> H (in Bethesda; O(2) affinity up). FT /FTId=VAR_003091. FT VARIANT 147 147 H -> D (in Hiroshima; O(2) affinity up). FT /FTId=VAR_003092. FT VARIANT 147 147 H -> L (in Cowtown; O(2) affinity up). FT /FTId=VAR_003093. FT VARIANT 147 147 H -> P (in York; O(2) affinity up). FT /FTId=VAR_003094. FT VARIANT 147 147 H -> Q (in Kodaira; O(2) affinity up). FT /FTId=VAR_003095. FT CONFLICT 26 26 Missing (in Ref. 15; ACD39349). FT CONFLICT 42 42 F -> L (in Ref. 13; AAR96398). FT HELIX 6 16 FT TURN 21 23 FT HELIX 24 35 FT HELIX 37 42 FT HELIX 44 46 FT HELIX 52 57 FT HELIX 59 77 FT TURN 78 80 FT HELIX 82 94 FT TURN 95 97 FT HELIX 102 119 FT HELIX 120 122 FT HELIX 125 142 FT HELIX 144 146 SQ SEQUENCE 147 AA; 15998 MW; A31F6D621C6556A1 CRC64; MVHLTPEEKS AVTALWGKVN VDEVGGEALG RLLVVYPWTQ RFFESFGDLS TPDAVMGNPK VKAHGKKVLG AFSDGLAHLD NLKGTFATLS ELHCDKLHVD PENFRLLGNV LVCVLAHHFG KEFTPPVQAA YQKVVAGVAN ALAHKYH // Output file format The output is a standard EMBOSS alignment file. The results can be output in one of several styles by using the command-line qualifier -aformat xxx, where 'xxx' is replaced by the name of the required format. Some of the alignment formats can cope with an unlimited number of sequences, while others are only for pairs of sequences. The available multiple alignment format names are: multiple, simple, fasta, msf, clustal, mega, meganon, nexus,, nexusnon, phylip, phylipnon, selex, treecon, tcoffee, debug, srs. The available pairwise alignment format names are: pair, markx0, markx1, markx2, markx3, markx10, match, sam, bam, score, srspair See: http://emboss.sf.net/docs/themes/AlignFormats.html for further information on alignment formats. Output files for usage example File: hba_human.needle ######################################## # Program: needle # Rundate: Fri 15 Jul 2011 12:00:00 # Commandline: needle # [-asequence] tsw:hba_human # [-bsequence] tsw:hbb_human # Align_format: srspair # Report_file: hba_human.needle ######################################## #======================================= # # Aligned_sequences: 2 # 1: HBA_HUMAN # 2: HBB_HUMAN # Matrix: EBLOSUM62 # Gap_penalty: 10.0 # Extend_penalty: 0.5 # # Length: 149 # Identity: 65/149 (43.6%) # Similarity: 90/149 (60.4%) # Gaps: 9/149 ( 6.0%) # Score: 292.5 # # #======================================= HBA_HUMAN 1 MV-LSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHF-D 48 || |:|.:|:.|.|.|||| :..|.|.|||.|:.:.:|.|:.:|..| | HBB_HUMAN 1 MVHLTPEEKSAVTALWGKV--NVDEVGGEALGRLLVVYPWTQRFFESFGD 48 HBA_HUMAN 49 LS-----HGSAQVKGHGKKVADALTNAVAHVDDMPNALSALSDLHAHKLR 93 || .|:.:||.|||||..|.::.:||:|::....:.||:||..||. HBB_HUMAN 49 LSTPDAVMGNPKVKAHGKKVLGAFSDGLAHLDNLKGTFATLSELHCDKLH 98 HBA_HUMAN 94 VDPVNFKLLSHCLLVTLAAHLPAEFTPAVHASLDKFLASVSTVLTSKYR 142 |||.||:||.:.|:..||.|...||||.|.|:..|.:|.|:..|..||. HBB_HUMAN 99 VDPENFRLLGNVLVCVLAHHFGKEFTPPVQAAYQKVVAGVANALAHKYH 147 #--------------------------------------- #--------------------------------------- The Identity: is the percentage of identical matches between the two sequences over the reported aligned region (including any gaps in the length). The Similarity: is the percentage of matches between the two sequences over the reported aligned region (including any gaps in the length). Data files For protein sequences EBLOSUM62 is used for the substitution matrix. For nucleotide sequence, EDNAFULL is used. Others can be specified. EMBOSS data files are distributed with the application and stored in the standard EMBOSS data directory, which is defined by the EMBOSS environment variable EMBOSS_DATA. To see the available EMBOSS data files, run: % embossdata -showall To fetch one of the data files (for example 'Exxx.dat') into your current directory for you to inspect or modify, run: % embossdata -fetch -file Exxx.dat Users can provide their own data files in their own directories. Project specific files can be put in the current directory, or for tidier directory listings in a subdirectory called ".embossdata". Files for all EMBOSS runs can be put in the user's home directory, or again in a subdirectory called ".embossdata". The directories are searched in the following order: * . (your current directory) * .embossdata (under your current directory) * ~/ (your home directory) * ~/.embossdata Notes needle is a true implementation of the Needleman-Wunsch algorithm and so produces a full path matrix. It therefore cannot be used with genome sized sequences unless you've a lot of memory and a lot of time. References 1. Needleman, S. B. and Wunsch, C. D. (1970) J. Mol. Biol. 48, 443-453. 2. Kruskal, J. B. (1983) An overview of squence comparison In D. Sankoff and J. B. Kruskal, (ed.), Time warps, string edits and macromolecules: the theory and practice of sequence comparison, pp. 1-44 Addison Wesley. Warnings needle is for aligning two sequences over their entire length. This works best with closely related sequences. If you use needle to align very distantly-related sequences, it will produce a result but much of the alignment may have little or no biological significance. A true Needleman Wunsch implementation like needle needs memory proportional to the product of the sequence lengths. For two sequences of length 10,000,000 and 1,000 it therefore needs memory proportional to 10,000,000,000 characters. Two arrays of this size are produced, one of ints and one of floats so multiply that figure by 8 to get the memory usage in bytes. That doesn't include other overheads. Therefore only use water and needle for accurate alignment of reasonably short sequences. If you run out of memory, try using stretcher instead. Diagnostic Error Messages Uncaught exception Assertion failed raised at ajmem.c:xxx Probably means you have run out of memory. Try using stretcher if this happens. Exit status 0 upon successful completion. Known bugs None. See also Program name Description est2genome Align EST sequences to genomic DNA sequence needleall Many-to-many pairwise alignments of two sequence sets stretcher Needleman-Wunsch rapid global alignment of two sequences When you want an alignment that covers the whole length of both sequences, use needle. When you are trying to find the best region of similarity between two sequences, use water. stretcher is a more suitable program to use to find global alignments of very long sequences. Author(s) Alan Bleasby European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, UK Please report all bugs to the EMBOSS bug team (emboss-bug (c) emboss.open-bio.org) not to the original author. History Completed 8th July 1999. Modified 26th July 1999 - scoring tweaked. Modified 22 Oct 2000 - %ID and %Similarity scores added. Target users This program is intended to be used by everyone and everything, from naive users to embedded scripts. Comments None