water Wiki The master copies of EMBOSS documentation are available at http://emboss.open-bio.org/wiki/Appdocs on the EMBOSS Wiki. Please help by correcting and extending the Wiki pages. Function Smith-Waterman local alignment of sequences Description water uses the Smith-Waterman algorithm (modified for speed enhancments) to calculate the local alignment of a sequence to one or more other sequences. The gap insertion penalty, gap extension penalty and substitution matrix used to calculate the alignments are specified. The output is a standard EMBOSS alignment file. Algorithm The Smith-Waterman algorithm is a member of the class of algorithms that can calculate the best score and local alignment in the order of mn steps, where n and m are the lengths of the two sequences. These dynamic programming algorithms were first developed for protein sequence comparison by Smith and Waterman, though similar methods were independently devised during the late 1960's and early 1970's for use in the fields of speech processing and computer science. Dynamic programming methods ensure the optimal local alignment by exploring all possible alignments and choosing the best. It does this by reading in a scoring matrix that contains values for every possible residue or nucleotide match. water finds an alignment with the maximum possible score where the score of an alignment is equal to the sum of the matches taken from the scoring matrix. An important problem is the treatment of gaps, i.e., spaces inserted to optimise the alignment score. A penalty is subtracted from the score for each gap opened (the 'gap open' penalty) and a penalty is subtracted from the score for the total number of gap spaces multiplied by a cost (the 'gap extension' penalty). Typically, the cost of extending a gap is set to be 5-10 times lower than the cost for opening a gap. There are two ways to compute a penalty for a gap of n positions : gap opening penalty + (n - 1) * gap extension penalty gap penalty + n * gap length penalty The two methods are basically equivalent. The first way is used by EMBOSS and WU-BLAST. The second way is used by NCBI-BLAST, GCG, Staden and CLUSTAL. Fasta used it for a long time the first way, but Prof. Pearson decided recently to shift to the second. The Smith-Waterman algorithm contains no negative scores in the path matrix it creates. The algorithm starts the alignment at the highest path matrix score and works backwards until a cell contains zero. See the Reference Smith et al. for details. Usage Here is a sample session with water % water tsw:hba_human tsw:hbb_human Smith-Waterman local alignment of sequences Gap opening penalty [10.0]: Gap extension penalty [0.5]: Output alignment [hba_human.water]: Go to the input files for this example Go to the output files for this example Command line arguments Smith-Waterman local alignment of sequences Version: EMBOSS:6.4.0.0 Standard (Mandatory) qualifiers: [-asequence] sequence Sequence filename and optional format, or reference (input USA) [-bsequence] seqall Sequence(s) filename and optional format, or reference (input USA) -gapopen float [10.0 for any sequence] The gap open penalty is the score taken away when a gap is created. The best value depends on the choice of comparison matrix. The default value assumes you are using the EBLOSUM62 matrix for protein sequences, and the EDNAFULL matrix for nucleotide sequences. (Number from 0.000 to 100.000) -gapextend float [0.5 for any sequence] The gap extension penalty is added to the standard gap penalty for each base or residue in the gap. This is how long gaps are penalized. Usually you will expect a few long gaps rather than many short gaps, so the gap extension penalty should be lower than the gap penalty. An exception is where one or both sequences are single reads with possible sequencing errors in which case you would expect many single base gaps. You can get this result by setting the gap open penalty to zero (or very low) and using the gap extension penalty to control gap scoring. (Number from 0.000 to 10.000) [-outfile] align [*.water] Output alignment file name (default -aformat srspair) Additional (Optional) qualifiers: -datafile matrixf [EBLOSUM62 for protein, EDNAFULL for DNA] This is the scoring matrix file used when comparing sequences. By default it is the file 'EBLOSUM62' (for proteins) or the file 'EDNAFULL' (for nucleic sequences). These files are found in the 'data' directory of the EMBOSS installation. Advanced (Unprompted) qualifiers: -[no]brief boolean [Y] Brief identity and similarity Associated qualifiers: "-asequence" associated qualifiers -sbegin1 integer Start of the sequence to be used -send1 integer End of the sequence to be used -sreverse1 boolean Reverse (if DNA) -sask1 boolean Ask for begin/end/reverse -snucleotide1 boolean Sequence is nucleotide -sprotein1 boolean Sequence is protein -slower1 boolean Make lower case -supper1 boolean Make upper case -sformat1 string Input sequence format -sdbname1 string Database name -sid1 string Entryname -ufo1 string UFO features -fformat1 string Features format -fopenfile1 string Features file name "-bsequence" associated qualifiers -sbegin2 integer Start of each sequence to be used -send2 integer End of each sequence to be used -sreverse2 boolean Reverse (if DNA) -sask2 boolean Ask for begin/end/reverse -snucleotide2 boolean Sequence is nucleotide -sprotein2 boolean Sequence is protein -slower2 boolean Make lower case -supper2 boolean Make upper case -sformat2 string Input sequence format -sdbname2 string Database name -sid2 string Entryname -ufo2 string UFO features -fformat2 string Features format -fopenfile2 string Features file name "-outfile" associated qualifiers -aformat3 string Alignment format -aextension3 string File name extension -adirectory3 string Output directory -aname3 string Base file name -awidth3 integer Alignment width -aaccshow3 boolean Show accession number in the header -adesshow3 boolean Show description in the header -ausashow3 boolean Show the full USA in the alignment -aglobal3 boolean Show the full sequence in alignment General qualifiers: -auto boolean Turn off prompts -stdout boolean Write first file to standard output -filter boolean Read first file from standard input, write first file to standard output -options boolean Prompt for standard and additional values -debug boolean Write debug output to program.dbg -verbose boolean Report some/full command line options -help boolean Report command line options and exit. More information on associated and general qualifiers can be found with -help -verbose -warning boolean Report warnings -error boolean Report errors -fatal boolean Report fatal errors -die boolean Report dying program messages -version boolean Report version number and exit Input file format water reads a nucleotide or protein sequence and one or more sequence to be aligned to it. The input is a standard EMBOSS sequence query (also known as a 'USA'). Major sequence database sources defined as standard in EMBOSS installations include srs:embl, srs:uniprot and ensembl Data can also be read from sequence output in any supported format written by an EMBOSS or third-party application. The input format can be specified by using the command-line qualifier -sformat xxx, where 'xxx' is replaced by the name of the required format. The available format names are: gff (gff3), gff2, embl (em), genbank (gb, refseq), ddbj, refseqp, pir (nbrf), swissprot (swiss, sw), dasgff and debug. See: http://emboss.sf.net/docs/themes/SequenceFormats.html for further information on sequence formats. Input files for usage example 'tsw:hba_human' is a sequence entry in the example protein database 'tsw' Database entry: tsw:hba_human ID HBA_HUMAN Reviewed; 142 AA. AC P69905; P01922; Q1HDT5; Q3MIF5; Q53F97; Q96KF1; Q9NYR7; Q9UCM0; DT 21-JUL-1986, integrated into UniProtKB/Swiss-Prot. DT 23-JAN-2007, sequence version 2. DT 15-JUN-2010, entry version 86. DE RecName: Full=Hemoglobin subunit alpha; DE AltName: Full=Hemoglobin alpha chain; DE AltName: Full=Alpha-globin; GN Name=HBA1; GN and GN Name=HBA2; OS Homo sapiens (Human). OC Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi; OC Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini; OC Catarrhini; Hominidae; Homo. OX NCBI_TaxID=9606; RN [1] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA] (HBA1). RX MEDLINE=81088339; PubMed=7448866; DOI=10.1016/0092-8674(80)90347-5; RA Michelson A.M., Orkin S.H.; RT "The 3' untranslated regions of the duplicated human alpha-globin RT genes are unexpectedly divergent."; RL Cell 22:371-377(1980). RN [2] RP NUCLEOTIDE SEQUENCE [MRNA] (HBA2). RX MEDLINE=80137531; PubMed=6244294; RA Wilson J.T., Wilson L.B., Reddy V.B., Cavallesco C., Ghosh P.K., RA Deriel J.K., Forget B.G., Weissman S.M.; RT "Nucleotide sequence of the coding portion of human alpha globin RT messenger RNA."; RL J. Biol. Chem. 255:2807-2815(1980). RN [3] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA] (HBA2). RX MEDLINE=81175088; PubMed=6452630; DOI=10.1073/pnas.77.12.7054; RA Liebhaber S.A., Goossens M.J., Kan Y.W.; RT "Cloning and complete nucleotide sequence of human 5'-alpha-globin RT gene."; RL Proc. Natl. Acad. Sci. U.S.A. 77:7054-7058(1980). RN [4] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RX PubMed=6946451; DOI=10.1073/pnas.78.8.5041; RA Orkin S.H., Goff S.C., Hechtman R.L.; RT "Mutation in an intervening sequence splice junction in man."; RL Proc. Natl. Acad. Sci. U.S.A. 78:5041-5045(1981). RN [5] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA], AND VARIANT LYS-32. RX MEDLINE=21303311; PubMed=11410421; RA Zhao Y., Xu X.; RT "Alpha2(CD31 AGG-->AAG, Arg-->Lys) causing non-deletional alpha- RT thalassemia in a Chinese family with HbH disease."; [Part of this file has been deleted for brevity] FT /FTId=VAR_002841. FT VARIANT 132 132 S -> P (in Questembert; highly unstable; FT causes alpha-thalassemia). FT /FTId=VAR_002843. FT VARIANT 134 134 S -> R (in Val de Marne; O(2) affinity FT up). FT /FTId=VAR_002844. FT VARIANT 136 136 V -> E (in Pavie). FT /FTId=VAR_002845. FT VARIANT 137 137 L -> M (in Chicago). FT /FTId=VAR_002846. FT VARIANT 137 137 L -> P (in Bibba; unstable; causes alpha- FT thalassemia). FT /FTId=VAR_002847. FT VARIANT 137 137 L -> R (in Toyama). FT /FTId=VAR_035242. FT VARIANT 139 139 S -> P (in Attleboro; O(2) affinity up). FT /FTId=VAR_002848. FT VARIANT 140 140 K -> E (in Hanamaki; O(2) affinity up). FT /FTId=VAR_002849. FT VARIANT 140 140 K -> T (in Tokoname; O(2) affinity up). FT /FTId=VAR_002850. FT VARIANT 141 141 Y -> H (in Rouen/Ethiopia; O(2) affinity FT up). FT /FTId=VAR_002851. FT VARIANT 142 142 R -> C (in Nunobiki; O(2) affinity up). FT /FTId=VAR_002852. FT VARIANT 142 142 R -> H (in Suresnes; O(2) affinity up). FT /FTId=VAR_002854. FT VARIANT 142 142 R -> L (in Legnano; O(2) affinity up). FT /FTId=VAR_002853. FT VARIANT 142 142 R -> P (in Singapore). FT /FTId=VAR_002855. FT CONFLICT 10 10 N -> H (in Ref. 13; BAD97112). FT HELIX 5 16 FT HELIX 17 21 FT HELIX 22 36 FT HELIX 38 43 FT HELIX 54 72 FT HELIX 74 76 FT HELIX 77 80 FT HELIX 82 90 FT HELIX 97 113 FT TURN 115 117 FT HELIX 120 137 FT TURN 138 140 SQ SEQUENCE 142 AA; 15258 MW; 15E13666573BBBAE CRC64; MVLSPADKTN VKAAWGKVGA HAGEYGAEAL ERMFLSFPTT KTYFPHFDLS HGSAQVKGHG KKVADALTNA VAHVDDMPNA LSALSDLHAH KLRVDPVNFK LLSHCLLVTL AAHLPAEFTP AVHASLDKFL ASVSTVLTSK YR // Database entry: tsw:hbb_human ID HBB_HUMAN Reviewed; 147 AA. AC P68871; A4GX73; B2ZUE0; P02023; Q13852; Q14481; Q14510; Q45KT0; AC Q549N7; Q6FI08; Q6R7N2; Q8IZI1; Q9BX96; Q9UCD6; Q9UCP8; Q9UCP9; DT 21-JUL-1986, integrated into UniProtKB/Swiss-Prot. DT 23-JAN-2007, sequence version 2. DT 15-JUN-2010, entry version 88. DE RecName: Full=Hemoglobin subunit beta; DE AltName: Full=Hemoglobin beta chain; DE AltName: Full=Beta-globin; DE Contains: DE RecName: Full=LVV-hemorphin-7; GN Name=HBB; OS Homo sapiens (Human). OC Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi; OC Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini; OC Catarrhini; Hominidae; Homo. OX NCBI_TaxID=9606; RN [1] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RX MEDLINE=77126403; PubMed=1019344; RA Marotta C., Forget B., Cohen-Solal M., Weissman S.M.; RT "Nucleotide sequence analysis of coding and noncoding regions of human RT beta-globin mRNA."; RL Prog. Nucleic Acid Res. Mol. Biol. 19:165-175(1976). RN [2] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RX MEDLINE=81064667; PubMed=6254664; DOI=10.1016/0092-8674(80)90428-6; RA Lawn R.M., Efstratiadis A., O'Connell C., Maniatis T.; RT "The nucleotide sequence of the human beta-globin gene."; RL Cell 21:647-651(1980). RN [3] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA], AND VARIANT LYS-7. RX PubMed=16175509; DOI=10.1086/491748; RA Wood E.T., Stover D.A., Slatkin M., Nachman M.W., Hammer M.F.; RT "The beta-globin recombinational hotspot reduces the effects of strong RT selection around HbC, a recently arisen mutation providing resistance RT to malaria."; RL Am. J. Hum. Genet. 77:637-642(2005). RN [4] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA]. RA Lu L., Hu Z.H., Du C.S., Fu Y.S.; RT "DNA sequence of the human beta-globin gene isolated from a healthy RT Chinese."; RL Submitted (JUN-1997) to the EMBL/GenBank/DDBJ databases. RN [5] RP NUCLEOTIDE SEQUENCE [GENOMIC DNA], AND VARIANT ARG-113. RA Cabeda J.M., Correia C., Estevinho A., Cardoso C., Amorim M.L., RA Cleto E., Vale L., Coimbra E., Pinho L., Justica B.; RT "Unexpected patterns of globin mutations in thalassemia patients from RT north of Portugal."; [Part of this file has been deleted for brevity] FT VARIANT 141 141 A -> V (in Puttelange; polycythemia; O(2) FT affinity up). FT /FTId=VAR_003082. FT VARIANT 142 142 L -> R (in Olmsted; unstable). FT /FTId=VAR_003083. FT VARIANT 143 143 A -> D (in Ohio; O(2) affinity up). FT /FTId=VAR_003084. FT VARIANT 144 144 H -> D (in Rancho Mirage). FT /FTId=VAR_003085. FT VARIANT 144 144 H -> P (in Syracuse; O(2) affinity up). FT /FTId=VAR_003087. FT VARIANT 144 144 H -> Q (in Little Rock; O(2) affinity FT up). FT /FTId=VAR_003086. FT VARIANT 144 144 H -> R (in Abruzzo; O(2) affinity up). FT /FTId=VAR_003088. FT VARIANT 145 145 K -> E (in Mito; O(2) affinity up). FT /FTId=VAR_003089. FT VARIANT 146 146 Y -> C (in Rainier; O(2) affinity up). FT /FTId=VAR_003090. FT VARIANT 146 146 Y -> H (in Bethesda; O(2) affinity up). FT /FTId=VAR_003091. FT VARIANT 147 147 H -> D (in Hiroshima; O(2) affinity up). FT /FTId=VAR_003092. FT VARIANT 147 147 H -> L (in Cowtown; O(2) affinity up). FT /FTId=VAR_003093. FT VARIANT 147 147 H -> P (in York; O(2) affinity up). FT /FTId=VAR_003094. FT VARIANT 147 147 H -> Q (in Kodaira; O(2) affinity up). FT /FTId=VAR_003095. FT CONFLICT 26 26 Missing (in Ref. 15; ACD39349). FT CONFLICT 42 42 F -> L (in Ref. 13; AAR96398). FT HELIX 6 16 FT TURN 21 23 FT HELIX 24 35 FT HELIX 37 42 FT HELIX 44 46 FT HELIX 52 57 FT HELIX 59 77 FT TURN 78 80 FT HELIX 82 94 FT TURN 95 97 FT HELIX 102 119 FT HELIX 120 122 FT HELIX 125 142 FT HELIX 144 146 SQ SEQUENCE 147 AA; 15998 MW; A31F6D621C6556A1 CRC64; MVHLTPEEKS AVTALWGKVN VDEVGGEALG RLLVVYPWTQ RFFESFGDLS TPDAVMGNPK VKAHGKKVLG AFSDGLAHLD NLKGTFATLS ELHCDKLHVD PENFRLLGNV LVCVLAHHFG KEFTPPVQAA YQKVVAGVAN ALAHKYH // Output file format The output is a standard EMBOSS alignment file. The results can be output in one of several styles by using the command-line qualifier -aformat xxx, where 'xxx' is replaced by the name of the required format. Some of the alignment formats can cope with an unlimited number of sequences, while others are only for pairs of sequences. The available multiple alignment format names are: multiple, simple, fasta, msf, clustal, mega, meganon, nexus,, nexusnon, phylip, phylipnon, selex, treecon, tcoffee, debug, srs. The available pairwise alignment format names are: pair, markx0, markx1, markx2, markx3, markx10, match, sam, bam, score, srspair See: http://emboss.sf.net/docs/themes/AlignFormats.html for further information on alignment formats. By default the output is in 'srspair' format. Output files for usage example File: hba_human.water ######################################## # Program: water # Rundate: Fri 15 Jul 2011 12:00:00 # Commandline: water # [-asequence] tsw:hba_human # [-bsequence] tsw:hbb_human # Align_format: srspair # Report_file: hba_human.water ######################################## #======================================= # # Aligned_sequences: 2 # 1: HBA_HUMAN # 2: HBB_HUMAN # Matrix: EBLOSUM62 # Gap_penalty: 10.0 # Extend_penalty: 0.5 # # Length: 145 # Identity: 63/145 (43.4%) # Similarity: 88/145 (60.7%) # Gaps: 8/145 ( 5.5%) # Score: 293.5 # # #======================================= HBA_HUMAN 3 LSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHF-DLS- 50 |:|.:|:.|.|.|||| :..|.|.|||.|:.:.:|.|:.:|..| ||| HBB_HUMAN 4 LTPEEKSAVTALWGKV--NVDEVGGEALGRLLVVYPWTQRFFESFGDLST 51 HBA_HUMAN 51 ----HGSAQVKGHGKKVADALTNAVAHVDDMPNALSALSDLHAHKLRVDP 96 .|:.:||.|||||..|.::.:||:|::....:.||:||..||.||| HBB_HUMAN 52 PDAVMGNPKVKAHGKKVLGAFSDGLAHLDNLKGTFATLSELHCDKLHVDP 101 HBA_HUMAN 97 VNFKLLSHCLLVTLAAHLPAEFTPAVHASLDKFLASVSTVLTSKY 141 .||:||.:.|:..||.|...||||.|.|:..|.:|.|:..|..|| HBB_HUMAN 102 ENFRLLGNVLVCVLAHHFGKEFTPPVQAAYQKVVAGVANALAHKY 146 #--------------------------------------- #--------------------------------------- The Identity: is the percentage of identical matches between the two sequences over the reported aligned region (including any gaps in the length). The Similarity: is the percentage of matches between the two sequences over the reported aligned region (including any gaps in the length). Data files For protein sequences EBLOSUM62 is used for the substitution matrix. For nucleotide sequence, EDNAFULL is used. Others can be specified. EMBOSS data files are distributed with the application and stored in the standard EMBOSS data directory, which is defined by the EMBOSS environment variable EMBOSS_DATA. To see the available EMBOSS data files, run: % embossdata -showall To fetch one of the data files (for example 'Exxx.dat') into your current directory for you to inspect or modify, run: % embossdata -fetch -file Exxx.dat Users can provide their own data files in their own directories. Project specific files can be put in the current directory, or for tidier directory listings in a subdirectory called ".embossdata". Files for all EMBOSS runs can be put in the user's home directory, or again in a subdirectory called ".embossdata". The directories are searched in the following order: * . (your current directory) * .embossdata (under your current directory) * ~/ (your home directory) * ~/.embossdata Notes A local alignment searches for regions of local similarity between two sequences and need not include the entire length of the sequences. Local alignment methods are very useful for scanning databases or other circumsatnces when you wish to find matches between small regions of sequences, for example between protein domains. References 1. Smith TF, Waterman MS (1981) J. Mol. Biol 147(1);195-7 Warnings Local alignment methods only report the best matching areas between two sequences - there may be a large number of alternative local alignments that do not score as highly. If two proteins share more than one common region, for example one has a single copy of a particular domain while the other has two copies, it may be possible to "miss" the second and subsequent alignments. You will be able to see this situation if you have done a dotplot and your local alignment does not show all the features you expected to see. water is for aligning the best matching subsequences of two sequences. It does not necessarily align whole sequences against each other; you should use needle if you wish to align closely related sequences along their whole lengths. A true Smith Waterman implementation like water needs memory proportional to the product of the sequence lengths. For two sequences of length 10,000,000 and 1,000 it therefore needs memory proportional to 10,000,000,000 characters. Two arrays of this size are produced, one of ints and one of floats so multiply that figure by 8 to get the memory usage in bytes. That doesn't include other overheads. Therefore only use water and needle for accurate alignment of reasonably short sequences. It should not be used with very large sequences unless you have a lot of memory and a lot of time. If you run out of memory, try using supermatcher or matcher instead. Diagnostic Error Messages Uncaught exception Assertion failed raised at ajmem.c:xxx Probably means you have run out of memory. Try using supermatcher or matcher if this happens. Exit status 0 if successful. Known bugs None. See also Program name Description matcher Waterman-Eggert local alignment of two sequences seqmatchall All-against-all word comparison of a sequence set supermatcher Calculate approximate local pair-wise alignments of larger sequences wordfinder Match large sequences against one or more other sequences wordmatch Finds regions of identity (exact matches) of two sequences matcher is a local alignment program that gives as good an alignment as water but it uses far less memory. However, water runs twice as fast as matcher. supermatcher is designed for local alignments of very large sequences. It gives good results as long as there is not a significant amount of insertion or deletion in the alignment. Author(s) Alan Bleasby European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, UK Please report all bugs to the EMBOSS bug team (emboss-bug (c) emboss.open-bio.org) not to the original author. History Completed 7th July 1999. Modified 27th July 1999 - tweaking scoring. Modified 22 Oct 2000 - added ID and Similarity scores. Target users This program is intended to be used by everyone and everything, from naive users to embedded scripts. Comments None