### Pipeline run code and environment: * Command: `/home/mjs5kd/.local/bin/refgenie build mm10/hisat2_index` * Compute host: udc-ba25-18 * Working dir: /sfs/qumulo/qproject/shefflab/deploy/rg.databio.org_full/rg.databio.org * Outfolder: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/_refgenie_build/ * Pipeline started at: (04-07 19:01:04) elapsed: 0.0 _TIME_ ### Version log: * Python version: 3.6.6 * Pypiper dir: `/sfs/qumulo/qhome/mjs5kd/.local/lib/python3.6/site-packages/pypiper` * Pypiper version: 0.12.1 * Pipeline dir: `/sfs/qumulo/qhome/mjs5kd/.local/bin` * Pipeline version: None ### Arguments passed to pipeline: * `asset_registry_paths`: `['mm10/hisat2_index']` * `assets`: `None` * `command`: `build` * `config_file`: `refgenie.yaml` * `docker`: `False` * `files`: `None` * `genome`: `None` * `genome_config`: `None` * `genome_description`: `None` * `logdev`: `False` * `new_start`: `False` * `outfolder`: `/project/shefflab/deploy/rg.databio.org_full/genomes/data` * `params`: `None` * `recipe`: `None` * `recover`: `False` * `requirements`: `False` * `silent`: `False` * `skip_read_lock`: `False` * `tag_description`: `None` * `verbosity`: `None` * `volumes`: `None` ---------------------------------------- Target to produce: `/project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/_refgenie_build/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1_hisat2_index__default.flag` > `hisat2-build /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/fasta/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.fa /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1` (31688)
[33mWARNING:[0m Skipping mount /opt/singularity/3.5.2/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container Settings: Output files: "/project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.*.ht2" Line rate: 6 (line is 64 bytes) Lines per side: 1 (side is 64 bytes) Offset rate: 4 (one in 16) FTable chars: 10 Strings: unpacked Local offset rate: 3 (one in 8) Local fTable chars: 6 Local sequence length: 57344 Local sequence overlap between two consecutive indexes: 1024 Endianness: little Actual local endianness: little Sanity checking: disabled Assertions: disabled Random seed: 0 Sizeofs: void*:8, int:4, long:8, size_t:8 Input files DNA, FASTA: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/fasta/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.fa Reading reference sizes Time reading reference sizes: 00:00:25 Calculating joined length Writing header Reserving space for joined string Joining reference sequences Time to join reference sequences: 00:00:13 Time to read SNPs and splice sites: 00:00:00 Using parameters --bmax 497396907 --dcv 1024 Doing ahead-of-time memory usage test Passed! Constructing with these parameters: --bmax 497396907 --dcv 1024 Constructing suffix-array element generator Building DifferenceCoverSample Building sPrime Building sPrimeOrder V-Sorting samples V-Sorting samples time: 00:01:49 Allocating rank array Ranking v-sort output Ranking v-sort output time: 00:00:30 Invoking Larsson-Sadakane on ranks Invoking Larsson-Sadakane on ranks time: 00:00:53 Sanity-checking and returning Building samples Reserving space for 12 sample suffixes Generating random suffixes QSorting 12 sample offsets, eliminating duplicates QSorting sample offsets, eliminating duplicates time: 00:00:00 Multikey QSorting 12 samples (Using difference cover) Multikey QSorting samples time: 00:00:00 Calculating bucket sizes Splitting and merging Splitting and merging time: 00:00:00 Avg bucket size: 3.78969e+08 (target: 497396906) Converting suffix-array elements to index image Allocating ftab, absorbFtab Entering GFM loop Getting block 1 of 7 Reserving size (497396907) for bucket 1 Calculating Z arrays for bucket 1 Entering block accumulator loop for bucket 1: bucket 1: 10% bucket 1: 20% bucket 1: 30% bucket 1: 40% bucket 1: 50% bucket 1: 60% bucket 1: 70% bucket 1: 80% bucket 1: 90% bucket 1: 100% Sorting block of length 349801096 for bucket 1 (Using difference cover) Sorting block time: 00:06:41 Returning block of 349801097 for bucket 1 Getting block 2 of 7 Reserving size (497396907) for bucket 2 Calculating Z arrays for bucket 2 Entering block accumulator loop for bucket 2: bucket 2: 10% bucket 2: 20% bucket 2: 30% bucket 2: 40% bucket 2: 50% bucket 2: 60% bucket 2: 70% bucket 2: 80% bucket 2: 90% bucket 2: 100% Sorting block of length 400051560 for bucket 2 (Using difference cover) Sorting block time: 00:07:59 Returning block of 400051561 for bucket 2 Getting block 3 of 7 Reserving size (497396907) for bucket 3 Calculating Z arrays for bucket 3 Entering block accumulator loop for bucket 3: bucket 3: 10% bucket 3: 20% bucket 3: 30% bucket 3: 40% bucket 3: 50% bucket 3: 60% bucket 3: 70% bucket 3: 80% bucket 3: 90% bucket 3: 100% Sorting block of length 451877143 for bucket 3 (Using difference cover) Sorting block time: 00:08:59 Returning block of 451877144 for bucket 3 Getting block 4 of 7 Reserving size (497396907) for bucket 4 Calculating Z arrays for bucket 4 Entering block accumulator loop for bucket 4: bucket 4: 10% bucket 4: 20% bucket 4: 30% bucket 4: 40% bucket 4: 50% bucket 4: 60% bucket 4: 70% bucket 4: 80% bucket 4: 90% bucket 4: 100% Sorting block of length 452272567 for bucket 4 (Using difference cover) Sorting block time: 00:08:58 Returning block of 452272568 for bucket 4 Getting block 5 of 7 Reserving size (497396907) for bucket 5 Calculating Z arrays for bucket 5 Entering block accumulator loop for bucket 5: bucket 5: 10% bucket 5: 20% bucket 5: 30% bucket 5: 40% bucket 5: 50% bucket 5: 60% bucket 5: 70% bucket 5: 80% bucket 5: 90% bucket 5: 100% Sorting block of length 371808568 for bucket 5 (Using difference cover) Sorting block time: 00:07:08 Returning block of 371808569 for bucket 5 Getting block 6 of 7 Reserving size (497396907) for bucket 6 Calculating Z arrays for bucket 6 Entering block accumulator loop for bucket 6: bucket 6: 10% bucket 6: 20% bucket 6: 30% bucket 6: 40% bucket 6: 50% bucket 6: 60% bucket 6: 70% bucket 6: 80% bucket 6: 90% bucket 6: 100% Sorting block of length 485965051 for bucket 6 (Using difference cover) Sorting block time: 00:09:56 Returning block of 485965052 for bucket 6 Getting block 7 of 7 Reserving size (497396907) for bucket 7 Calculating Z arrays for bucket 7 Entering block accumulator loop for bucket 7: bucket 7: 10% bucket 7: 20% bucket 7: 30% bucket 7: 40% bucket 7: 50% bucket 7: 60% bucket 7: 70% bucket 7: 80% bucket 7: 90% bucket 7: 100% Sorting block of length 141007509 for bucket 7 (Using difference cover) Sorting block time: 00:02:48 Returning block of 141007510 for bucket 7 Exited GFM loop fchr[A]: 0 fchr[C]: 773280124 fchr[G]: 1325927941 fchr[T]: 1878618059 fchr[$]: 2652783500 Exiting GFM::buildToDisk() Returning from initFromVector Wrote 888471943 bytes to primary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.1.ht2 Wrote 663195880 bytes to secondary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.2.ht2 Re-opening _in1 and _in2 as input streams Returning from GFM constructor Returning from initFromVector Wrote 1165549977 bytes to primary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.5.ht2 Wrote 675339278 bytes to secondary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.6.ht2 Re-opening _in5 and _in5 as input streams Returning from HGFM constructor Headers: len: 2652783500 gbwtLen: 2652783501 nodes: 2652783501 sz: 663195875 gbwtSz: 663195876 lineRate: 6 offRate: 4 offMask: 0xfffffff0 ftabChars: 10 eftabLen: 0 eftabSz: 0 ftabLen: 1048577 ftabSz: 4194308 offsLen: 165798969 offsSz: 663195876 lineSz: 64 sideSz: 64 sideGbwtSz: 48 sideGbwtLen: 192 numSides: 13816581 numLines: 13816581 gbwtTotLen: 884261184 gbwtTotSz: 884261184 reverse: 0 linearFM: Yes Total time for call to driver() for forward index: 01:21:48Command completed. Elapsed time: 1:21:49. Running peak memory: 5.052GB. PID: 31688; Command: hisat2-build; Return code: 0; Memory used: 5.052GB > `touch /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/_refgenie_build/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1_hisat2_index__default.flag` (14738)
Command completed. Elapsed time: 0:00:00. Running peak memory: 5.052GB. PID: 14738; Command: touch; Return code: 0; Memory used: 0.001GB Asset digest: 7ee392f36e92676d4572b6d95c7774fe Default tag for '0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index' set to: default ### Pipeline completed. Epilogue * Elapsed time (this run): 1:21:59 * Total elapsed time (all runs): 1:21:49 * Peak memory (this run): 5.0516 GB * Pipeline completed time: 2021-04-07 20:23:03 Finished building 'hisat2_index' asset Created alias directories: - /project/shefflab/deploy/rg.databio.org_full/genomes/alias/mm10/hisat2_index/default