======== SOAPdenovo2 ========== ['SOAPdenovo-63mer', 'all', '-s', 'SOAPdenovo2.config', '-o', 'SOAPdenovo2', '-K', '21', '-N', '184000000', '-R', '-M', '1', '-L', '200', '-p', '32'] Start time: 2021-06-05 09:07:18.145862 Version 2.04: released on July 13th, 2012 Compile Apr 5 2019 17:00:46 ******************** Pregraph ******************** Parameters: pregraph -s SOAPdenovo2.config -K 21 -p 32 -R -o SOAPdenovo2 In SOAPdenovo2.config, 5 lib(s), maximum read length 100, maximum name length 256. 32 thread(s) initialized. Import reads from file: /local/genbank/workspace/reads.all/2008SE.fq Import reads from file: /local/genbank/workspace/reads.all/20090202.pollux.unpaired.fq Import reads from file: /local/genbank/workspace/reads.all/367GB.fq Import reads from file: /local/genbank/workspace/reads.all/454.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB4kbUnpaired.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_8_1_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_8_2_valid_2P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_6_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_6_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_8_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_8_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_6_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_6_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_8_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_8_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB2kb_R1.illumina-1.8+_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB2kb_R2.illumina-1.8+_valid_2P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB4kb_R1.illumina-1.8+_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB4kb_R2.illumina-1.8+_valid_2P.fq.corrected.fq Time spent on hashing reads: 415s, 92789812 read(s) processed. LIB(s) information: [LIB] 0, avg_ins 0, reverse 0. [LIB] 1, avg_ins 339, reverse 0. [LIB] 2, avg_ins 339, reverse 0. [LIB] 3, avg_ins 2000, reverse 0. [LIB] 4, avg_ins 4000, reverse 0. 559953974 node(s) allocated, 5763302452 kmer(s) in reads, 5763302452 kmer(s) processed. done hashing nodes 530775535 linear node(s) marked. Time spent on marking linear nodes: 9s. Time spent on pre-graph construction: 425s. Start to remove frequency-one-kmer tips shorter than 42. Total 7232194 tip(s) removed. 32 thread(s) initialized. 4532883 linear node(s) marked. Start to remove tips with minority links. 875694 tip(s) removed in cycle 1. 1714 tip(s) removed in cycle 2. 1 tip(s) removed in cycle 3. 0 tip(s) removed in cycle 4. Total 877409 tip(s) removed. 32 thread(s) initialized. 0 linear node(s) marked. Time spent on removing tips: 315s. --- 20000000 edge(s) built. --- 30000000 edge(s) built. --- 40000000 edge(s) built. 49352297 (24683679) edge(s) and 4060341 extra node(s) constructed. Time spent on constructing edges: 733s. In file: SOAPdenovo2.config, max seq len 100, max name len 256. 32 thread(s) initialized. Import reads from file: /local/genbank/workspace/reads.all/2008SE.fq Import reads from file: /local/genbank/workspace/reads.all/20090202.pollux.unpaired.fq Import reads from file: /local/genbank/workspace/reads.all/367GB.fq Import reads from file: /local/genbank/workspace/reads.all/454.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB4kbUnpaired.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_8_1_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_8_2_valid_2P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_6_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_6_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_8_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_8_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_6_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_6_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_8_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_8_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB2kb_R1.illumina-1.8+_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB2kb_R2.illumina-1.8+_valid_2P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB4kb_R1.illumina-1.8+_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB4kb_R2.illumina-1.8+_valid_2P.fq.corrected.fq 92789812 read(s) processed. Time spent on: importing reads: 78s, chopping reads to kmers: 16s, searching kmers: 263s, aligning reads to edges: 45s, searching (K+1)mers: 117s, adding pre-arcs: 99s, recording read paths: 95s. 1838995989 marker(s) output. Reads alignment done, 1187186 read(s) deleted, 46198778 pre-arc(s) added. LIB(s) information: [LIB] 0, avg_ins 0, reverse 0. [LIB] 1, avg_ins 339, reverse 0. [LIB] 2, avg_ins 339, reverse 0. [LIB] 3, avg_ins 2000, reverse 0. [LIB] 4, avg_ins 4000, reverse 0. Time spent on aligning reads: 762s. 17234961 vertex(es) output. Overall time spent on constructing pre-graph: 37m. ******************** Contig ******************** Parameters: contig -g SOAPdenovo2 -M 1 -R -s SOAPdenovo2.config -p 32 There are 17234961 kmer(s) in vertex file. There are 49352297 edge(s) in edge file. Kmers sorted. 49352297 edge(s) input. 62664542 pre-arcs loaded. 895680528 markers overall. 895680528 markers loaded. 12336475 none-palindrome edge(s) swapped, 0 palindrome edge(s) processed. 49352297 edge(s) sorted. Arcs sorted. 274082 repeat(s) are solvable, 548586 more edge(s). 1097172 dead arc(s) removed. Time spent on solving repeat: 17s. Start to pinch bubbles, cutoff 0.100000, MAX NODE NUM 3, MAX DIFF NUM 2. .............100000 bubbles merged. .............200000 bubbles merged. .............300000 bubbles merged. .............400000 bubbles merged. .............500000 bubbles merged. 1334134 start points, 43591810 dheap nodes. 13274615 pair(s) found, 614604 pair of path(s) compared, 517824 pair(s) merged. Sequence comparison failed: Path crossing deleted edge 0 Length difference of two paths greater than two 46310 Mismatch score greater than cutoff (2) 17938 Mismatch score ratio greater than cutoff (0.1) 0 Path length shorter than (Kmer-1) 32532 DFibHeap: 2482852 node(s) allocated. 3083177 edge(s) concatenated in cycle 1. 85740 edge(s) concatenated in cycle 2. 14 edge(s) concatenated in cycle 3. 0 edge(s) concatenated in cycle 4. Time spent on pinching bubbles: 174s. Start to destroy weak inner edges. 4128737 weak inner edge(s) destroyed in cycle 1. 16689 weak inner edge(s) destroyed in cycle 2. 161 weak inner edge(s) destroyed in cycle 3. 5 weak inner edge(s) destroyed in cycle 4. 0 weak inner edge(s) destroyed in cycle 5. 8245452 dead arc(s) removed. 489553 inner edge(s) with coverage lower than or equal to 1 destroyed. 1011686 dead arc(s) removed. 6583829 edge(s) concatenated in cycle 1. 397561 edge(s) concatenated in cycle 2. 1891 edge(s) concatenated in cycle 3. 0 edge(s) concatenated in cycle 4. Before compacting, 49900883 edge(s) existed. After compacting, 19263275 edge(s) left. Strict: 0, cutoff length: 42. 1322009 tips cut in cycle 1. 86649 tips cut in cycle 2. 13353 tips cut in cycle 3. 3684 tips cut in cycle 4. 1239 tips cut in cycle 5. 521 tips cut in cycle 6. 242 tips cut in cycle 7. 129 tips cut in cycle 8. 96 tips cut in cycle 9. 78 tips cut in cycle 10. 66 tips cut in cycle 11. 51 tips cut in cycle 12. 45 tips cut in cycle 13. 35 tips cut in cycle 14. 56 tips cut in cycle 15. 107 tips cut in cycle 16. 229 tips cut in cycle 17. 1871 tips cut in cycle 18. 3440 tips cut in cycle 19. 1925 tips cut in cycle 20. 496 tips cut in cycle 21. 86 tips cut in cycle 22. 10 tips cut in cycle 23. 3 tips cut in cycle 24. 2 tips cut in cycle 25. 0 tips cut in cycle 26. 1240348 dead arc(s) removed. 845036 edge(s) concatenated in cycle 1. 4584 edge(s) concatenated in cycle 2. 1 edge(s) concatenated in cycle 3. 0 edge(s) concatenated in cycle 4. Before compacting, 19263275 edge(s) existed. After compacting, 14691189 edge(s) left. There are 1394279 contig(s) longer than 100, sum up 273942106 bp, with average length 196. The longest length is 3952 bp, contig N50 is 212 bp,contig N90 is 109 bp. 7350755 contig(s) longer than 22 output. Time spent on constructing contig: 12m. ******************** Map ******************** Parameters: map -s SOAPdenovo2.config -g SOAPdenovo2 -p 32 -K 21 Kmer size: 21. Contig length cutoff: 23. 7350755 contig(s), maximum sequence length 3952, minimum sequence length 22, maximum name length 10. Time spent on parsing contigs file: 3s. 32 thread(s) initialized. Time spent on hashing contigs: 32s. 367211393 node(s) allocated, 375166224 kmer(s) in contigs, 375166224 kmer(s) processed. Time spent on graph construction: 35s. Time spent on aligning long reads: 0s. In file: SOAPdenovo2.config, max seq len 100, max name len 256 32 thread(s) initialized. 14691189 edge(s) in the graph. Import reads from file: /local/genbank/workspace/reads.all/20090202_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_7_2_2P.fastq.corrected.fq Current insert size is 339, map_len is 32. Import reads from file: /local/genbank/workspace/reads.all/20090202_8_1_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/20090202_8_2_valid_2P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_6_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_6_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_8_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae6.2GB_8_2_2P.fastq.corrected.fq Current insert size is 339, map_len is 32. Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_6_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_6_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_7_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_7_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_8_1_1P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/Algae7.2GB_8_2_2P.fastq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB2kb_R1.illumina-1.8+_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB2kb_R2.illumina-1.8+_valid_2P.fq.corrected.fq Current insert size is 2000, map_len is 35. Import reads from file: /local/genbank/workspace/reads.all/AB4kb_R1.illumina-1.8+_valid_1P.fq.corrected.fq Import reads from file: /local/genbank/workspace/reads.all/AB4kb_R2.illumina-1.8+_valid_2P.fq.corrected.fq Current insert size is 4000, map_len is 35. Total reads 70134528 Reads in gaps 28264748 Ratio 40.3% Reads on contigs 43796577 Ratio 62.4% 4 pe insert size, the largest boundary is 70134528. LIB(s) information: [LIB] 0, avg_ins 0, reverse 0. [LIB] 1, avg_ins 339, reverse 0. [LIB] 2, avg_ins 339, reverse 0. [LIB] 3, avg_ins 2000, reverse 0. [LIB] 4, avg_ins 4000, reverse 0. Time spent on aligning reads: 668s. Overall time spent on alignment: 11m. ******************** Scaff ******************** Parameters: scaff -g SOAPdenovo2 -p 32 -L 200 -N 184000000 gzip: stdout: Broken pipe Files for scaffold construction are OK. There are 4 grad(s), 70134528 read(s), max read len 100. Kmer size: 21 There are 14691189 edge(s) in edge file. Mask contigs with coverage lower than 0.8 or higher than 16.0, and strict length 100. Average contig coverage is 8, 6220830 contig(s) masked. Mask contigs shorter than 200, 7800153 contig(s) masked. 16939963 arc(s) loaded, average weight is 7. 7350755 contig(s) loaded. Done loading updated edges. Time spent on loading updated edges: 56s. ***************************************************** Start to load paired-end reads information. For insert size: 339 Total PE links 1619591 Normal PE links on same contig 443801 Incorrect oriented PE links 779 PE links of too small insert size 10743 PE links of too large insert size 0 Correct PE links 1164224 Accumulated connections 1411154 Use contigs longer than 339 to estimate insert size: PE links 311424 Average insert size 235 SD 70 705577 new connections. For insert size: 339 Total PE links 6604676 Normal PE links on same contig 1877974 Incorrect oriented PE links 1102 PE links of too small insert size 8940 PE links of too large insert size 0 Correct PE links 4716479 Accumulated connections 3649402 Use contigs longer than 339 to estimate insert size: PE links 1241854 Average insert size 238 SD 69 1824701 new connections. For insert size: 2000 Total PE links 5791153 Normal PE links on same contig 85842 Incorrect oriented PE links 110 PE links of too small insert size 22 PE links of too large insert size 0 Correct PE links 5705152 Accumulated connections 10848874 Use contigs longer than 2000 to estimate insert size: PE links 19 Too few PE links. 5424437 new connections. For insert size: 4000 Total PE links 3325873 Normal PE links on same contig 468534 Incorrect oriented PE links 1419 PE links of too small insert size 2 PE links of too large insert size 0 Correct PE links 2855532 Accumulated connections 2696094 Use contigs longer than 4000 to estimate insert size: PE links 0 Too few PE links. 1348047 new connections. All paired-end reads information loaded. Time spent on loading paired-end reads information: 58s. ***************************************************** Start to construct scaffolds. *************************** For insert size: 339 Total PE links 2530280 PE links to masked contigs 2398967 On same scaffold PE links 0 Report from smallScaf: 0 scaffolds by smallPE. *************************** For insert size: 2000 Total PE links 5424438 PE links to masked contigs 5118978 On same scaffold PE links 0 *************************** For insert size: 4000 Total PE links 1348047 PE links to masked contigs 1185229 On same scaffold PE links 0 Cutoff of PE links to make a reliable connection: 5 Report from checkScaf: 0 scaffold segments broken. Active connections 1046358 Weak connections 913814 Weak ratio 87.3% 3 circles removed. Start to remove transitive connection. Total contigs 14691189 Masked contigs 14020995 Remained contigs 670194 None-outgoing-connection contigs 544131 (81.190071%) Single-outgoing-connection contigs 118711 Multi-outgoing-connection contigs 337 Cycle 1 Two-outgoing-connection contigs 7015 Potential transitive connections 5586 Transitive connections 4281 Transitive ratio 61.0% Cycle 2 Two-outgoing-connection contigs 2616 Potential transitive connections 1297 Transitive connections 20 Transitive ratio 0.8% Cycle 3 Two-outgoing-connection contigs 2596 Potential transitive connections 1277 Transitive connections 0 Transitive ratio 0.0% Start to linearize sub-graph. Picked sub-graphs 2127 Connection-conflict 40 Significant overlapping 1118 Eligible 0 Bubble structures 0 Mask repeats: Puzzles 905 Masked contigs 584 Start to remove transitive connection. Total contigs 14691189 Masked contigs 14022163 Remained contigs 669026 None-outgoing-connection contigs 540583 (80.801491%) Single-outgoing-connection contigs 128223 Multi-outgoing-connection contigs 12 Cycle 1 Two-outgoing-connection contigs 208 Potential transitive connections 159 Transitive connections 53 Transitive ratio 25.5% Cycle 2 Two-outgoing-connection contigs 155 Potential transitive connections 106 Transitive connections 0 Transitive ratio 0.0% Start to linearize sub-graph. Picked sub-graphs 164 Connection-conflict 6 Significant overlapping 143 Eligible 0 Bubble structures 0 Non-strict linearization. Start to linearize sub-graph. Picked sub-graphs 114 Connection-conflict 7 Significant overlapping 72 Eligible 0 Bubble structures 0 Start to mask puzzles. Masked contigs 46 Remained puzzles 0 Freezing done. Recover contigs. Total recovered contigs 133 Single-route cases 133 Multi-route cases 0 All links loaded. Time spent on constructing scaffolds: 25s. The final rank ******************************* Scaffold number 39457 In-scaffold contig number 1394050 Total scaffold length 102798852 Average scaffold length 2605 Filled gap number 7261 Longest scaffold 28486 Scaffold and singleton number 1330663 Scaffold and singleton length 308714489 Average length 232 N50 281 N90 97 Weak points 0 ******************************* 1000 scaffolds processed. 2000 scaffolds processed. 3000 scaffolds processed. 4000 scaffolds processed. 5000 scaffolds processed. 6000 scaffolds processed. 7000 scaffolds processed. 8000 scaffolds processed. 9000 scaffolds processed. 10000 scaffolds processed. 11000 scaffolds processed. 12000 scaffolds processed. 13000 scaffolds processed. 14000 scaffolds processed. 15000 scaffolds processed. 16000 scaffolds processed. 17000 scaffolds processed. 18000 scaffolds processed. 19000 scaffolds processed. 20000 scaffolds processed. 21000 scaffolds processed. 22000 scaffolds processed. 23000 scaffolds processed. 24000 scaffolds processed. 25000 scaffolds processed. 26000 scaffolds processed. 27000 scaffolds processed. 28000 scaffolds processed. 29000 scaffolds processed. 30000 scaffolds processed. 31000 scaffolds processed. 32000 scaffolds processed. 33000 scaffolds processed. 34000 scaffolds processed. 35000 scaffolds processed. 36000 scaffolds processed. 37000 scaffolds processed. 38000 scaffolds processed. 39000 scaffolds processed. Done with 39457 scaffolds, 0 gaps finished, 63874 gaps overall. Overall time spent on constructing scaffolds: 109m. Time for the whole pipeline: 171m. Finish time: 2021-06-05 11:59:02.958473 Elapsed time: 2:51:44.812611