### Pipeline run code and environment: * Command: `/home/mjs5kd/.local/bin/refgenie build mm10/hisat2_index` * Compute host: udc-ba25-18 * Working dir: /sfs/qumulo/qproject/shefflab/deploy/rg.databio.org_full/rg.databio.org * Outfolder: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/_refgenie_build/ * Pipeline started at: (04-07 19:01:04) elapsed: 0.0 _TIME_ ### Version log: * Python version: 3.6.6 * Pypiper dir: `/sfs/qumulo/qhome/mjs5kd/.local/lib/python3.6/site-packages/pypiper` * Pypiper version: 0.12.1 * Pipeline dir: `/sfs/qumulo/qhome/mjs5kd/.local/bin` * Pipeline version: None ### Arguments passed to pipeline: * `asset_registry_paths`: `['mm10/hisat2_index']` * `assets`: `None` * `command`: `build` * `config_file`: `refgenie.yaml` * `docker`: `False` * `files`: `None` * `genome`: `None` * `genome_config`: `None` * `genome_description`: `None` * `logdev`: `False` * `new_start`: `False` * `outfolder`: `/project/shefflab/deploy/rg.databio.org_full/genomes/data` * `params`: `None` * `recipe`: `None` * `recover`: `False` * `requirements`: `False` * `silent`: `False` * `skip_read_lock`: `False` * `tag_description`: `None` * `verbosity`: `None` * `volumes`: `None` ---------------------------------------- Target to produce: `/project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/_refgenie_build/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1_hisat2_index__default.flag` > `hisat2-build /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/fasta/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.fa /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1` (31688)
WARNING: Skipping mount /opt/singularity/3.5.2/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
Settings:
  Output files: "/project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.*.ht2"
  Line rate: 6 (line is 64 bytes)
  Lines per side: 1 (side is 64 bytes)
  Offset rate: 4 (one in 16)
  FTable chars: 10
  Strings: unpacked
  Local offset rate: 3 (one in 8)
  Local fTable chars: 6
  Local sequence length: 57344
  Local sequence overlap between two consecutive indexes: 1024
  Endianness: little
  Actual local endianness: little
  Sanity checking: disabled
  Assertions: disabled
  Random seed: 0
  Sizeofs: void*:8, int:4, long:8, size_t:8
Input files DNA, FASTA:
  /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/fasta/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.fa
Reading reference sizes
  Time reading reference sizes: 00:00:25
Calculating joined length
Writing header
Reserving space for joined string
Joining reference sequences
  Time to join reference sequences: 00:00:13
  Time to read SNPs and splice sites: 00:00:00
Using parameters --bmax 497396907 --dcv 1024
  Doing ahead-of-time memory usage test
  Passed!  Constructing with these parameters: --bmax 497396907 --dcv 1024
Constructing suffix-array element generator
Building DifferenceCoverSample
  Building sPrime
  Building sPrimeOrder
  V-Sorting samples
  V-Sorting samples time: 00:01:49
  Allocating rank array
  Ranking v-sort output
  Ranking v-sort output time: 00:00:30
  Invoking Larsson-Sadakane on ranks
  Invoking Larsson-Sadakane on ranks time: 00:00:53
  Sanity-checking and returning
Building samples
Reserving space for 12 sample suffixes
Generating random suffixes
QSorting 12 sample offsets, eliminating duplicates
QSorting sample offsets, eliminating duplicates time: 00:00:00
Multikey QSorting 12 samples
  (Using difference cover)
  Multikey QSorting samples time: 00:00:00
Calculating bucket sizes
Splitting and merging
  Splitting and merging time: 00:00:00
Avg bucket size: 3.78969e+08 (target: 497396906)
Converting suffix-array elements to index image
Allocating ftab, absorbFtab
Entering GFM loop
Getting block 1 of 7
  Reserving size (497396907) for bucket 1
  Calculating Z arrays for bucket 1
  Entering block accumulator loop for bucket 1:
  bucket 1: 10%
  bucket 1: 20%
  bucket 1: 30%
  bucket 1: 40%
  bucket 1: 50%
  bucket 1: 60%
  bucket 1: 70%
  bucket 1: 80%
  bucket 1: 90%
  bucket 1: 100%
  Sorting block of length 349801096 for bucket 1
  (Using difference cover)
  Sorting block time: 00:06:41
Returning block of 349801097 for bucket 1
Getting block 2 of 7
  Reserving size (497396907) for bucket 2
  Calculating Z arrays for bucket 2
  Entering block accumulator loop for bucket 2:
  bucket 2: 10%
  bucket 2: 20%
  bucket 2: 30%
  bucket 2: 40%
  bucket 2: 50%
  bucket 2: 60%
  bucket 2: 70%
  bucket 2: 80%
  bucket 2: 90%
  bucket 2: 100%
  Sorting block of length 400051560 for bucket 2
  (Using difference cover)
  Sorting block time: 00:07:59
Returning block of 400051561 for bucket 2
Getting block 3 of 7
  Reserving size (497396907) for bucket 3
  Calculating Z arrays for bucket 3
  Entering block accumulator loop for bucket 3:
  bucket 3: 10%
  bucket 3: 20%
  bucket 3: 30%
  bucket 3: 40%
  bucket 3: 50%
  bucket 3: 60%
  bucket 3: 70%
  bucket 3: 80%
  bucket 3: 90%
  bucket 3: 100%
  Sorting block of length 451877143 for bucket 3
  (Using difference cover)
  Sorting block time: 00:08:59
Returning block of 451877144 for bucket 3
Getting block 4 of 7
  Reserving size (497396907) for bucket 4
  Calculating Z arrays for bucket 4
  Entering block accumulator loop for bucket 4:
  bucket 4: 10%
  bucket 4: 20%
  bucket 4: 30%
  bucket 4: 40%
  bucket 4: 50%
  bucket 4: 60%
  bucket 4: 70%
  bucket 4: 80%
  bucket 4: 90%
  bucket 4: 100%
  Sorting block of length 452272567 for bucket 4
  (Using difference cover)
  Sorting block time: 00:08:58
Returning block of 452272568 for bucket 4
Getting block 5 of 7
  Reserving size (497396907) for bucket 5
  Calculating Z arrays for bucket 5
  Entering block accumulator loop for bucket 5:
  bucket 5: 10%
  bucket 5: 20%
  bucket 5: 30%
  bucket 5: 40%
  bucket 5: 50%
  bucket 5: 60%
  bucket 5: 70%
  bucket 5: 80%
  bucket 5: 90%
  bucket 5: 100%
  Sorting block of length 371808568 for bucket 5
  (Using difference cover)
  Sorting block time: 00:07:08
Returning block of 371808569 for bucket 5
Getting block 6 of 7
  Reserving size (497396907) for bucket 6
  Calculating Z arrays for bucket 6
  Entering block accumulator loop for bucket 6:
  bucket 6: 10%
  bucket 6: 20%
  bucket 6: 30%
  bucket 6: 40%
  bucket 6: 50%
  bucket 6: 60%
  bucket 6: 70%
  bucket 6: 80%
  bucket 6: 90%
  bucket 6: 100%
  Sorting block of length 485965051 for bucket 6
  (Using difference cover)
  Sorting block time: 00:09:56
Returning block of 485965052 for bucket 6
Getting block 7 of 7
  Reserving size (497396907) for bucket 7
  Calculating Z arrays for bucket 7
  Entering block accumulator loop for bucket 7:
  bucket 7: 10%
  bucket 7: 20%
  bucket 7: 30%
  bucket 7: 40%
  bucket 7: 50%
  bucket 7: 60%
  bucket 7: 70%
  bucket 7: 80%
  bucket 7: 90%
  bucket 7: 100%
  Sorting block of length 141007509 for bucket 7
  (Using difference cover)
  Sorting block time: 00:02:48
Returning block of 141007510 for bucket 7
Exited GFM loop
fchr[A]: 0
fchr[C]: 773280124
fchr[G]: 1325927941
fchr[T]: 1878618059
fchr[$]: 2652783500
Exiting GFM::buildToDisk()
Returning from initFromVector
Wrote 888471943 bytes to primary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.1.ht2
Wrote 663195880 bytes to secondary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.2.ht2
Re-opening _in1 and _in2 as input streams
Returning from GFM constructor
Returning from initFromVector
Wrote 1165549977 bytes to primary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.5.ht2
Wrote 675339278 bytes to secondary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1.6.ht2
Re-opening _in5 and _in5 as input streams
Returning from HGFM constructor
Headers:
    len: 2652783500
    gbwtLen: 2652783501
    nodes: 2652783501
    sz: 663195875
    gbwtSz: 663195876
    lineRate: 6
    offRate: 4
    offMask: 0xfffffff0
    ftabChars: 10
    eftabLen: 0
    eftabSz: 0
    ftabLen: 1048577
    ftabSz: 4194308
    offsLen: 165798969
    offsSz: 663195876
    lineSz: 64
    sideSz: 64
    sideGbwtSz: 48
    sideGbwtLen: 192
    numSides: 13816581
    numLines: 13816581
    gbwtTotLen: 884261184
    gbwtTotSz: 884261184
    reverse: 0
    linearFM: Yes
Total time for call to driver() for forward index: 01:21:48
Command completed. Elapsed time: 1:21:49. Running peak memory: 5.052GB. PID: 31688; Command: hisat2-build; Return code: 0; Memory used: 5.052GB > `touch /project/shefflab/deploy/rg.databio.org_full/genomes/data/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index/default/_refgenie_build/0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1_hisat2_index__default.flag` (14738)

Command completed. Elapsed time: 0:00:00. Running peak memory: 5.052GB.  
  PID: 14738;	Command: touch;	Return code: 0;	Memory used: 0.001GB

Asset digest: 7ee392f36e92676d4572b6d95c7774fe
Default tag for '0f10d83b1050c08dd53189986f60970b92a315aa7a16a6f1/hisat2_index' set to: default

### Pipeline completed. Epilogue
*        Elapsed time (this run):  1:21:59
*  Total elapsed time (all runs):  1:21:49
*         Peak memory (this run):  5.0516 GB
*        Pipeline completed time: 2021-04-07 20:23:03
Finished building 'hisat2_index' asset
Created alias directories: 
 - /project/shefflab/deploy/rg.databio.org_full/genomes/alias/mm10/hisat2_index/default