Leo Martins https://leomrtns.github.io/blog/ Notes from the workbench — mathematics, algorithms, and computational biology. quarto-1.10.18 Mon, 05 Jan 2026 00:00:00 GMT Back to blogging Leo Martins https://leomrtns.github.io/posts/back-to-blogging/ I decided to get back into blogging, since perhaps it’s the only semi-permanent documentation of what we have been socially discussing, academically and otherwise.

I was thinking about what went wrong with blogging, and there are several reasons. The mostly cited ones are a lack of support for RSS and its more widespread usage (after the end of Google Reader), and the rise of social media platforms like Twitter, which are quicker to digest than whole paragraphs. Erick articulates it better than I can, but there are two other personal reasons why I believe I stopped blogging. The first was the volatility of references, and the second was a perceived lack of civility in the comment sections.

I guess we got used to the first one, where links from one year ago are no longer valid, although at that time me, as an academic, assumed that all sentences should be supported by a link (reference needed). Going forward I’ll relax this requirement. A side-effect is that unlike before, we can update blog posts instead of creating a new post just with an addendum. As for the lack of civility, it moved from blog comments to social media. Which on the upside is not a resposibility of the hosting blog to moderate anymore. For personal blogs it does not change much, but now we are more comfortable with blocking, deleting comments, or ignoring trolls even when disguised as sea lions.

I still have some stuff to do, like deciding if I should invest in a blog with jupyter-notebook capabilities (I still have some technical blog posts using the unmaintained fastpages ). Or checking if the RSS gives us the full post —at some point article feeds started forcing you to click back to the original site to read the full content, I guess fitting into my first complaint of volatile content.

]]> Personal https://leomrtns.github.io/posts/back-to-blogging/ Mon, 05 Jan 2026 00:00:00 GMT Welcome Leo Martins https://leomrtns.github.io/posts/welcome/ I am Leonardo de Oliveira Martins, a Bayesian computational biologist specialised in Evolutionary Genomics. I am a Tenure Track Fellow at the Signal Processing Group of the Department of Electrical and Electronic Engineering, University of Liverpool.

I used to be a senior researcher at the Quadram Institute Biosciences, where I developed, implemented, and streamlined hig-performing computational methods for the evolutionary analysis of microbial data sets.

]]> Misc https://leomrtns.github.io/posts/welcome/ Sun, 04 Jan 2026 00:00:00 GMT Not all invariant sites are created the same Leo Martins https://leomrtns.github.io/posts/invariant-sites/ A while back we had an interesting discussion regarding the exclusion of invariant sites in maximum likelihood phylogenetic inference. Since this discussion usually comes up again and again, I would like to add a bit of context.

First, I understand that there are situations where it is unavoidable to exclude “uninteresting” sites to speed up calculations, and in many practical situations we expect the results to be unchanged. In these cases, and listening to the justification for it, I wonder why using a slower likelihood method instead of a much faster distance-based method. But that’s a different story.

So, in a nutshell, the problem lies when we have a large alignment where most of the sites (columns in the alignment) are constant, i.e. there is no polymorphism (SNPs) and thus we assume they do not affect the tree estimation. Their removal should not affect the topology, but the branch lengths are indeed affected by the number of constant sites.

Removing constant sites biases your data set towards fast-evolving sites, thus the name ASC (“ascertainment bias” is a type of selection bias). The obvious effect is that all estimated branch lengths will be very large, since they are given in substitutions per site and at least one of the sequences has a SNP at a given position. For instance, if you have two genomes of 1000bp but only 10 sites are polymorphic (i.e. SNPs, the other 900 are identical on both sequences), then their distance is 1/100 substitutions per site; however if you only look at the 10 SNPs then their distance is 1 substitution per site!

When you have several sequences and a tree it gets more complicated since maximum likelihood methods like IQTREE2 estimate the instantaneous rate matrix — from the HKY model, for instance. Which describes the probability of A becoming a C etc. and it must take into account the many constant sites in order to have a good estimate of the Prob(A->A) and so forth. If there are no constant sites the rate matrix will inflate Prob(A->C) and collapse Prob(A->A) and thus branch lengths will be inflated in a non-linear way for most models (otherwise we could just correct by dividing branch lengths by an amount, like 100 in example above). The topology may also be affected. The bias correction is done by replacing the unconditional probabilities Prob(A->C) by conditional ones Prob(A->C “given that” all constant sites were removed).

Removing invariant sites and telling the tree inference program about it

One solution for this problem is to calculate how many (constant) sites you are excluding and then adding this information to the phylogenetic inference software (the link above contains instructions on how to to this with IQTREE2; the same can be done for BEAST or RaxML).

We may be tempted to use an explicit “ascertainment bias correction” model, but we have to be careful since the ASC method implemented in iqtree2 assumes that we do not know how many constant sites we are excluding (see https://www.biorxiv.org/content/10.1101/186478v1, and http://dx.doi.org/10.1093/sysbio/syv053 for details). Which is true for morphological data, but not in general for genomes, since we can count (snp-sites -C) the constant sites we are excluding. Therefore we have more precise estimation by informing iqtree2 the number of excluded sites (the option “-fconst”) as discussed in the link above https://bitsandbugs.org/2019/11/06/two-easy-ways-to-run-iq-tree-with-the-correct-number-of-constant-sites/. Notice that -fconst creates artificial columns in the alignment with the constant sites, and therefore is not compatible with +ASC. That is, if you call iqtree with both +ASC and -fconst then ASC will complain that there are constant sites in the alignment (which were added by -fconst!)

RaxML handles more ASC models: https://github.com/amkozlov/raxml-ng/wiki/Input-data#evolutionary-model, where the STAM model is equivalent to -fconst and the LEWIS model is equiv to +ASC in IQTREE2.

Sometimes it’s better not to do anything

My suggestion in the discussion linked above and in general, is to leave the original data as is without trying to remove sites and correcting later. This is because if the sites are “truly” invariant then they won’t add to the computational time (as explained by Alexis Stamatakis in https://groups.google.com/g/raxml/c/JuwXlf576Sw for instance). This is also what IQTREE2 does, as mentioned above: it adds four artificial columns to the alignment weighted by their counts. This is because most likelihood algorithms compress the alignments into distinct “patterns”, weighted by how many times we see them in the alignment.

There is an important difference, however, which is the inclusion of gaps and Ns in the “invariant” sites. For instance, take the following alignments (one line per sequence, spaces to ease visualisation):

alignment 1
>seq1 NNNNCCTCCC CGGTTCCAAC CTTCCAAAAA
>seq2 NNNNCCTCCC CGGTTCCAAC CCCTTATAAA
>seq3 ----CCTCCC CGGTTCCTTG GCCTTAACAA
>seq4 ----CCTCCC CGGAAGGTTG GCCTTAAAGA
>seq5 ----CCTCCT TTTAAGGTTG GCCTTAAAAT

alignment 2
>seq1 AAAACCTCCC CGGTTCCAAC CTTCCAAAAA
>seq2 NNNNCCTCCC CGGTTCCAAC CCCTTATAAA
>seq3 ----CCTCCC CGGTTCCTTG GCCTTAACAA
>seq4 ----CCTCCC CGGAAGGTTG GCCTTAAAGA
>seq5 ----CCTCCT TTTAAGGTTG GCCTTAAAAT

alignment 3
>seq1 NNNNCCTCCC CGGTTCCAAC CTTCCAAAAA
>seq2 AAAACCTCCC CGGTTCCAAC CCCTTATAAA
>seq3 AAAACCTCCC CGGTTCCTTG GCCTTAACAA
>seq4 ----CCTCCC CGGAAGGTTG GCCTTAAAGA
>seq5 ----CCTCCT TTTAAGGTTG GCCTTAAAAT

The only difference between the alignments are the first four sites: in alignment 1 we only have gaps and Ns. In alignments 2 and 3 we have AAAA in a few samples, instead. All 3 alignments are identical after removal of constant sites with snp-sites, since the first four sites do not have SNPs! Which is what I meant in the discussion linked above when I said

One thing to consider is that a column composed of “AAAA” is distinct from one composed of “AANN” or “AA-A”, although all 3 of them will be removed by snp-sites -c.

If we fix the topology and the branch lengths (iqtree2 -s ${file}.aln -te fixed.tre -blfix -m JC) then the log-likelihood of the three alignments will be -115.559 for alignment 1, -121.104 for alignment 2, and -122.123 for alignment 3. So the likelihood is different for the three alignments (I’m using the Jukes-Cantor model to avoid handling extra parameters). Using snp-sites to remove the invariant sites and to count the frequencies would not fix the problem since alignments 2 and 3 have the same number of “invariant sites”: 5,4,0,1. To make a long story short, I ran the 3 alignments under a fixed tree (topology and branch lengths) as above and asking IQTREE2 to optimise the branch lengths (iqtree2 -s ${i} -te fixed.tre --prefix ${i%aln}best_length -m JC). I also used only the constant sites with the snp-sites correction, as in

snp-sites -c $i > ${i%aln}snpsites
iqtree2 -fconst $(snp-sites -C ${i}) -s ${i%aln}snpsites -m JC --prefix ${i%aln}const

We have the results as below, where we can see that not only the likelihoods, but the tree branch lengths are different:

iqtree file log-likelihood ML tree
alignment1.aln -115.559 (seq1:0.1886473604,seq2:0.0303274053,(seq3:0.0284331913,(seq4:0.0258689619,seq5:0.2463693196):0.2138883204):0.2087029810);
alignment1.best_length -115.559 (seq1:0.1886473604,seq2:0.0303274053,(seq3:0.0284331913,(seq4:0.0258689619,seq5:0.2463693196):0.2138883204):0.2087029810);
alignment1.const -115.559 (seq1:0.1886475517,seq2:0.0303274177,(seq3:0.0284339867,(seq4:0.0258696236,seq5:0.2463695763):0.2138875505):0.2087022795);
alignment2.aln -121.104 (seq1:0.1886473604,seq2:0.0303274053,(seq3:0.0284331913,(seq4:0.0258689619,seq5:0.2463693196):0.2138883204):0.2087029810);
alignment2.best_length -121.104 (seq1:0.1886473604,seq2:0.0303274053,(seq3:0.0284331913,(seq4:0.0258689619,seq5:0.2463693196):0.2138883204):0.2087029810);
alignment2.const -124.433 (seq1:0.1577276882,seq2:0.0281664193,(seq3:0.0272139809,(seq4:0.0254304079,seq5:0.2041776675):0.1736347105):0.1705027642);
alignment3.aln -122.123 (seq1:0.1886473604,seq2:0.0303274053,(seq3:0.0284331913,(seq4:0.0258689619,seq5:0.2463693196):0.2138883204):0.2087029810);
alignment3.best_length -122.045 (seq1:0.1888748916,seq2:0.0261918048,(seq3:0.0246320132,(seq4:0.0257728057,seq5:0.2464608488):0.2140593199):0.1757918053);
alignment3.const -124.433 (seq1:0.1577274163,seq2:0.0281664193,(seq3:0.0272140466,(seq4:0.0254297568,seq5:0.2041783024):0.1736349260):0.1705029865);

Notice that all trees are identical for alignment*.aln since we fix the tree and branch lengths. Also, alignments 2 and 3 have practically identical trees after the SNP-sites correction (const trees), the difference in branch lengths is negligible due to stochastic effects. However the optimal branch lengths are quite distinct under a full optimisation (best_length trees). This is because the Prob(A->N) and Prob(A->A) are not the same, which will affect the estimation of model parameters and branch lengths.

So it boils down to the amount of gaps and uncertainty in our data set: on one extreme, we can assume that all constant sites which were removed were bona fide ACGT only; on the other hand, removal of “constant” sites may lump into the same category truly invariant sites with regions of high uncertainty. In the former case, we don’t need to remove them since most software (BEAST, IQTREE, RaxML) can easily merge constant sites into a few patterns with no detriment of speed. In the latter case, the patterns of gaps and ambiguous bases within an alignment column can inform the model and branch lengths and thus should not be removed. (Notice that in this case the speed will suffer more since every pattern of Ns and indels must be calculated separately).

Thus if you use snp-sites to remove constant sites and you observe a fantastic speedup, you should worry that it removed more than truly constant sites. Because, as we saw above, it does not consider gaps and Ns. To recap:

Alignment A with truly, bona fide constant sites:
>seq1 N-AC
>seq2 N-AC
>seq3 --AC
>seq4 --AC
>seq5 --AC

Alignment B with no true constant sites: they look constant (no SNPs!) but have distinct phylogenetic signal:
>seq1 AAAA----
>seq2 N-A---AA
>seq3 --AA---A
>seq4 --A-A---
>seq5 ----AA--

The sites from alignment B represent each a distinct pattern, and are not merged by the phylogenetic software. They contain unique phylogenetic information. However they would all be excluded from a SNPs-only alignment, leading to a speed up.

What to do then?

So maybe your data set has more gaps and uncertainty than you would hope. Most real data sets look much more like alignment B than alignment A. What I usually do is to first estimate the tree with SNPs only, and then use this tree as an initial (or fixed) topology for the whole data set, where branch lengths and model parameters can be optimised. This is because the SNPs are good enough for estimating the topology —tree branching, without branch lengths— , which is the most time-consuming part.

I also avoid at all costs using the standard generation of parsimony trees in IQTREE, since it does not use the site patterns (https://groups.google.com/g/iqtree/c/vjyLDpc1e1Q/m/68svHwSMAQAJ) and can be extremely slow. Another approach might be to remove sites which look constant (i.e. no SNPs) and with high levels of uncertainty/gaps. This could help cases where the tree inference is slow even under a fixed topology.

All files and script used are available as an XZ compressed tar file.

]]> phylogenetics https://leomrtns.github.io/posts/invariant-sites/ Tue, 16 Aug 2022 00:00:00 GMT Hash functions and pseudo-random number generation Leo Martins https://leomrtns.github.io/posts/random-hash/

005 Hash functions and pseudo-random number generation

This notebook uses the low-level phylogenetic library biomcmc-lib (commit da29c5c).

Here I show two small tests, one with generation of an initial set of random numbers, and another on popcount() speeds.

testing the generation of a deterministic array of random numbers

In biomcmc-lib there are a few vectors with “random numbers” (some actually from random.org, some random prime numbers used in hash functions, etc.). The function biomcmc_salt_vector32_from_spice_table() will populate a vector with these number in an order specified by the particular seed[4] values.

Code 
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/biomcmc-lib/lib
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/biomcmc-lib/lib/.libs/libbiomcmc.a
//%cflags:-lm
#include <biomcmc.h>

int
main (int argc, char **argv)
{
    uint32_t i,j;
    uint32_t seeds[]={0,1,2,3}, vec[32], nvec=34;
    uint8_t *c;
    size_t size = sizeof (vec);
    
    for (i=0;i<2;i++) {
        seeds[0] = i;
        for (j=0;j<4;j++) printf ("%8x ",seeds[j]);
        printf ("original seeds\n");
        biomcmc_salt_vector32_from_spice_table (vec, nvec, seeds);
        printf("\nseed: %u\n", i);
        //for (j=0; j<nvec;j++) {printf("%12u ", vec[j]); if (!((j+1)%8)) printf ("\n"); }
        c = (uint8_t*) vec;
        size = sizeof (vec);
        for (; (size > 0); c++, size--) {printf ("%4x ", *c); if (!((size-1)%16)) printf ("\n"); }
    }
    for (i=0;i<15;i++) {
        for (j=0;j<4;j++) printf ("%8x ",biomcmc_hashint_salted (j, i));
        printf ("<< for hash %u\n", i);
    }
        
    return EXIT_SUCCESS;
}
       0        1        2        3 original seeds

seed: 0
  15   da   f0   20   87   60   7f   24   ff   f0   9c   e2   87   7e    1    0 
  ff   ff   4f   46   ff   ff   27   23   d8   9a   b1   55   99    2   99   d7 
  85    5    c   d0   a6    f   a6   c4   9a   99   99   99   52   5a    7   8c 
  ae   11   92   3d   8f   e7    2    0   1d   60    0    0   1d   60    0    0 
  ed   6e   1c    1   fb   a5   71    7    9   f8   e6   5e   7b    a   9f    7 
  ff   ff   4b   59   ff   ff   f3   32   d7    5   f2   28   b7   27   3d   c8 
  fa   9e   6e   d6   4f   64   3f   6e   5c   52   ba   39   34   f8   d7   3a 
  2c   fc   1f   a0   46   7e    9    0   71   f7    2    0   1d   60    0    0 
       1 b621b33c 9cc8151d 5ab560c5 original seeds

seed: 1
  55   93   67   35   87   60   7f   24   2e   d1   9d   e2   7b   b3   31   35 
  ff   ff   7f   4c   ff   ff   da   33   fa   2f   b9   34   4d   26   48   46 
  fa   9e   6e   d6    e   8d   25   79   9f   da   f7   50   ee   a6   50   e1 
  e1   36   ce   6d   68   19   10    0    2   90   fd   2d   55   bd    4    0 
  89    a   50   1a    5   9e   41   2e   63   d1   e7   5e   eb   25   d4   19 
  ff   ff   92   6c   ff   ff   b9   30   c2   98   28   91   66   a9   5a   6f 
  fa   9e   6e   d6   30   6c   db   ea   99   3d   cc   5a   cf   a6   fc   ed 
  c0   13   ac   84   6a    0    9    0   a2   47   42   1a   f8   2b    1    0 
c0a9496a 27922c9d c6793575 87d06fbe << for hash 0
6b4ed927 b48681b6 e267b84c 4f6e0e9c << for hash 1
caa3caa3 12d60bf6 25ac1fe5 3882835c << for hash 2
    d0dc   2a6ee1   55234f   7f181c << for hash 3
deb66b58 deb66ab9 deb66a9a deb66afb << for hash 4
2ba588a6 2ba58337 2ba51840 2ba510d1 << for hash 5
acefdd39 ec26e4d2 75fe3d8c ce4259f1 << for hash 6
4636b9c9 62baf5a0 ff4d1170 2bf062cf << for hash 7
b7255c83 a63ec54b 2a674713 c0186ef5 << for hash 8
ad92d5f0 8e953d2f 8f005635 cb773d1b << for hash 9
e6546b64 e6546b64 e6546b64 e6546b64 << for hash 10
c0a9496a 27922c9d c6793575 87d06fbe << for hash 11
c0a9496a 27922c9d c6793575 87d06fbe << for hash 12
c0a9496a 27922c9d c6793575 87d06fbe << for hash 13
c0a9496a 27922c9d c6793575 87d06fbe << for hash 14

Counting the number of active bits (pop_count())

I have recently implemented a few faster versions of 64 bits popcounts, and below I compare them with my first implementation — in the context of “bipartitions”, which are vectors of bitstrings.

First I check if they are calculating the right thing (number of “ones” in a vector of 64 bit integers). And then I see which is faster. I expected pop0() to be slowest (it is linear on the number of active bits), and the others to be similar. Which is what we observe. The one I’ve chosen to be the new “default” on biomcmc-lib is pop1(), which I’ve seen in Jue Ruan’s RedBean and also in novoBreak. The previous default was the i &= i -1 trick described in K & R. The other two are from Andrew Dalke.

Code 
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/biomcmc-lib/lib
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/biomcmc-lib/lib/.libs/libbiomcmc.a
//%cflags:-lm
#include <biomcmc.h>

int
main (int argc, char **argv)
{
    uint32_t i,j, n_iter = 10000;
    clock_t time0, time1;
    bipartition bp = new_bipartition (70);
    biomcmc_random_number_init (0ULL);
    printf ("STEP 1 : check that all pop counts are correct\n");
    for (i=0; i<4; i++) {
        for (j=0; j < bp->n->ints; j++) {
            bp->bs[j] = biomcmc_rng_get ();
        }
        bipartition_print_to_stdout (bp);
        printf ("\t%4d %4d %4d %4d\n", 
                bipartition_count_n_ones_pop0(bp),
                bipartition_count_n_ones_pop1(bp),
                bipartition_count_n_ones_pop2(bp),
                bipartition_count_n_ones_pop3(bp));
    }
    del_bipartition (bp); bp = new_bipartition (50000); // bigger bipartition
    printf ("STEP 2: time pop counts (pop1 is default, and we call through a wrapper to discount this overhead)\n");
    time0 = clock ();
    for (i=0; i < n_iter; i++) {
        for (j=0; j < bp->n->ints; j++) bp->bs[j] = biomcmc_rng_get ();
        bipartition_count_n_ones (bp);
    }
    time1 = clock (); fprintf (stderr, "pop1: %.8f secs\n", (double)(time1-time0)/(double)CLOCKS_PER_SEC);
    time0 = time1;
    for (i=0; i < n_iter; i++) {
        for (j=0; j < bp->n->ints; j++) bp->bs[j] = biomcmc_rng_get ();
        bipartition_count_n_ones_pop0 (bp);
    }
    time1 = clock (); fprintf (stderr, "pop0: %.8f secs\n", (double)(time1-time0)/(double)CLOCKS_PER_SEC);
    time0 = time1;
    
    for (i=0; i < n_iter; i++) {
        for (j=0; j < bp->n->ints; j++) bp->bs[j] = biomcmc_rng_get ();
        bipartition_count_n_ones_pop2 (bp);
    }
    time1 = clock (); fprintf (stderr, "pop2: %.8f secs\n", (double)(time1-time0)/(double)CLOCKS_PER_SEC);
    time0 = time1;
    
    for (i=0; i < n_iter; i++) {
        for (j=0; j < bp->n->ints; j++) bp->bs[j] = biomcmc_rng_get ();
        bipartition_count_n_ones_pop3 (bp);
    }
    time1 = clock (); fprintf (stderr, "pop3: %.8f secs\n", (double)(time1-time0)/(double)CLOCKS_PER_SEC);
    time0 = time1;
    
    del_bipartition (bp);
    biomcmc_random_number_finalize ();
    return EXIT_SUCCESS;
}
STEP 1 : check that all pop counts are correct
1111101010010110110000001011111000010001001000010111110100110101.001010[0]    34   34   34   34
1011001001011101101010010000011101001100100001111101011110110010.011010[34]       36   36   36   36
0000101001000100011000111111100000011010001001000110110100001110.011010[36]       29   29   29   29
1010001100100100100001000000010101111111101000011110111110001100.100100[29]       32   32   32   32
STEP 2: time pop counts (pop1 is default, and we call through a wrapper to discount this overhead)
pop1: 0.19077800 secs
pop0: 0.37243400 secs
pop2: 0.20049800 secs
pop3: 0.19488300 secs
]]> C https://leomrtns.github.io/posts/random-hash/ Fri, 01 Apr 2022 00:00:00 GMT kalign3 as a library and colour printing Leo Martins https://leomrtns.github.io/posts/kalign3/

004 kalign and colour printing

incorporating kalign3 as a library

This notebook uses the low-level phylogenetic library biomcmc-lib (commit 5975331).

Recently I (re-)discovered the kalign software for multiple sequence alignment by Timo Lassmann, and was happily surprised with the code simplicity and a liberal license (GPL-3.0-or-later).

Therefore I decided to incorporate its code into a library, that I can access using a char_vector:: from the biomcmc-lib library. Currently I am working on a derived library/software called “cumaru” (still private), with a modified version of kalign algorithms.

Autotools

First, a digresssion on the configuration aspects of cumaru (or another software importing these libraries).

This is needed by configure.ac as well as the described files, in order for kalign3 to use the AVX extensions.

# M4 macros for checking of CPU features from kalign3
m4_include([m4/ax_gcc_x86_avx_xgetbv.m4])
m4_include([m4/ax_gcc_x86_cpuid.m4])
m4_include([m4/ax_check_compile_flag.m4])
m4_include([m4/ax_ext.m4])
m4_include([m4/ax_openmp.m4])
AX_EXT

For importing biomcmc-lib, you have two options: 1. downloading the (upstream) software through github, with option --recursive (default for final users of software). This will download biomcmc-lib into ${srcdir}/submodules/biomcmc-lib. 2. download biomcmc-lib independently into ${srcdir}/biomcmc-lib or somewhere else and then link it to there. This is how I do it, since I use the same directory for several projects that rely on it.

If you opted for (1) above (i.e. you just cloned the repository recursively), then the configuration option below will create a link as in option (2).

AC_CHECK_FILE([${srcdir}/biomcmc-lib],[],[ln -s submodules/biomcmc-lib ${srcdir}/biomcmc-lib])
AC_CHECK_FILE([${srcdir}/biomcmc-lib/configure.ac],[], [AC_MSG_ERROR(["biomcmc-lib submodule missing, please git clone --recursive or link by hand to location of source code"])])
dnl Call biomcmc-lib ./configure script recursively.
AC_CONFIG_SUBDIRS([biomcmc-lib])
AC_SUBST([BIOMCMCLIB], [biomcmc-lib])

Then the file kalign/Makefile.am has the following information, to create a static local library. This library will encapsulate biomcmc-lib and will be available to the main software as libalign.la:

AM_CPPFLAGS = $(GTKDEPS_CFLAGS)  -I$(srcdir)/../@BIOMCMCLIB@/lib  @OPENMP_CPPFLAGS@ @ZLIB_LIBS@
AM_CFLAGS = @SIMD_FLAGS@ @AM_CFLAGS@  @OPENMP_CFLAGS@
LDADD = $(GTKDEPS_LIBS) @ZLIB_LIBS@ ../biomcmc-lib/lib/libbiomcmc.la  $(AM_LDFLAGS)

common_headers = kalign.h \
tldevel.h rng.h global.h \
alignment_parameters.h \
bisectingKmeans.h \
sequence_distance.h \
alignment.h bpm.h

common_src = run_kalign.c \
tldevel.c rng.c \
alignment_parameters.c \
bisectingKmeans.c \
sequence_distance.c \
alignment.c bpm.c

noinst_LTLIBRARIES = libkalign.la   ## noinst_LT: linked statically (not installed globally)
libkalign_la_SOURCES = config.h $(common_headers) $(common_src)

And the src/Makefile.am (with the final software) can be something like:

AM_CPPFLAGS = $(GTKDEPS_CFLAGS) -I$(srcdir)/../kalign -I$(srcdir)/../@BIOMCMCLIB@/lib @OPENMP_CPPFLAGS@  @ZLIB_LIBS@ 
AM_CFLAGS = @AM_CFLAGS@ @OPENMP_CFLAGS@ @CHECK_CFLAGS@
LDADD = $(GTKDEPS_LIBS) @CHECK_LIBS@  ../kalign/libkalign.la ../biomcmc-lib/lib/libbiomcmc.la @ZLIB_LIBS@  $(AM_LDFLAGS)

bin_PROGRAMS = cumaru 
cumaru_SOURCES = main.c kseq.h
cumaru_LDADD = $(LDADD)

Notice that we need to include the path to the local biomcmc-lib as well, since it is also statically linked.

The code below performs multiple sequence alignment from hand-fed sequences (in practice, fasta reading usually come from kseq.h).

Code 
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/leomrtns.cumaru/kalign
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/kalign
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/kalign/.libs/libkalign.a
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build.191216/biomcmc-lib/lib/.libs/libbiomcmc.a
//%cflags:-lm
#include <kalign.h>

int
main (int argc, char **argv)
{
  int i;
  clock_t time0, time1;
  char_vector seqname = new_char_vector (1);
  char_vector dna = new_char_vector (1);
  char_vector align = NULL;

  time0 = clock ();

  char_vector_add_string (dna, "ATCGAAAGAATTAGGCTTAAGCTAAAAGCTTATGACCACAGAGTTCTAGACCGTACAGAT");
  char_vector_add_string (seqname, "Campy");
  char_vector_add_string (dna, "AGAAAGAATTAGGCTTAAGCTAAGCTTATGACCACAGAGTTCTAGACCGTACAGTT");
  char_vector_add_string (seqname, "Vibrio");
  char_vector_add_string (dna, "ATGGAAAGAATTAGGCTTAAGCCGGAGCTTATGACCACAGAGTTCTAGAGCGTACAGTT");
  char_vector_add_string (seqname, "Strepto");
  char_vector_add_string (dna, "ATGGAAAGAATTAGGCTTAAGCTAAAAGCTTATGACTTCATTCTGTACAGTT");
  char_vector_add_string (seqname, "Staph");

  align = kalign3_from_char_vector (dna);
  time1 = clock (); 
  fprintf (stderr, "finished in  %lf secs\n", (double)(time1-time0)/(double)(CLOCKS_PER_SEC)); 
  fflush(stderr);

  for (i= 0; i < align->nstrings; i++) printf (">%s\n%s\n", seqname->string[i], align->string[i]);

  del_char_vector (dna);
  del_char_vector (align);
  del_char_vector (seqname);
  return EXIT_SUCCESS;
}
finished in  0.000583 secs
>Campy
ATCGAAAGAATTAGGCTTAAGCTAAAAGCTTATGACCACAGAGTTCTAGACCGTACAGAT
>Vibrio
AG--AAAGAATTAGGCTTAAGCTAA--GCTTATGACCACAGAGTTCTAGACCGTACAGTT
>Strepto
ATGGAAAGAATTAGGCTTAAGCCGG-AGCTTATGACCACAGAGTTCTAGAGCGTACAGTT
>Staph
ATGGAAAGAATTAGGCTTAAGCTAAAAGCTTATGACTTCA---TTCT-----GTACAGTT

Printing colours

This week I’ve been refactoring the random and hash modules of biomcmc-lib. I am also adding user-friendly functionality — linux fortunes and coloured printing. Here we can see my test with colours in printf().

Code 
#include <stdio.h>
const char *mytext[] = {"this has ` , $ percent \% and ' while", "so  \%d ____^H^H^H^Hmany"};
const char col_reset[] = "\e[0m";
const char *rownames[7] = {"regular", "bold", "underline", "background", "highbckgrnd", "hightext", "boldhightxt"};
const char *prt_col[][8]={ // 0-black   1-red   2-grn   3-yel   4-blu   5-mag   6-cyn   7-white
   {"\e[0;30m",  "\e[0;31m",  "\e[0;32m",  "\e[0;33m",  "\e[0;34m",  "\e[0;35m",  "\e[0;36m",  "\e[0;37m"},  // 0 regular text
   {"\e[1;30m",  "\e[1;31m",  "\e[1;32m",  "\e[1;33m",  "\e[1;34m",  "\e[1;35m",  "\e[1;36m",  "\e[1;37m"},  // 1 regular bold text
   {"\e[4;30m",  "\e[4;31m",  "\e[4;32m",  "\e[4;33m",  "\e[4;34m",  "\e[4;35m",  "\e[4;36m",  "\e[4;37m"},  // 2 regular underline text
   {"\e[40m",    "\e[41m",    "\e[42m",    "\e[43m",    "\e[44m",    "\e[45m",    "\e[46m",    "\e[47m"},    // 3 regular background
   {"\e[0;100m", "\e[0;101m", "\e[0;102m", "\e[0;103m", "\e[0;104m", "\e[0;105m", "\e[0;106m", "\e[0;107m"}, // 4 high intensity underground
   {"\e[0;90m",  "\e[0;91m",  "\e[0;92m",  "\e[0;93m",  "\e[0;94m",  "\e[0;95m",  "\e[0;96m",  "\e[0;97m"},  // 5 high intensity text 
   {"\e[1;90m",  "\e[1;91m",  "\e[1;92m",  "\e[1;93m",  "\e[1;94m",  "\e[1;95m",  "\e[1;96m",  "\e[1;97m"}   // 6 bold high intensity text
};

int main() {
    int i,j;
    for (i=0;i<8;i++) {
        printf ("\n%d\n", i);
        for (j=0; j < 7; j++) printf ("%15s : %s>%s<>%s<%s\n",rownames[j],prt_col[j][i],mytext[0], mytext[1], col_reset);
    };
    // test with marsaglia constants (should print prime numbers)
    printf ("%5d %5d %5d", (18000<<16)-1, (18000<<15)-1, (18030<<16)-1);
}
0

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<



1

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<



2

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<



3

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<



4

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<



5

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<



6

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<



7

        regular : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

           bold : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

      underline : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

     background : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    highbckgrnd : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

       hightext : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

    boldhightxt : >this has ` , $ percent % and ' while<>so  %d ____^H^H^H^Hmany<

1179647999 589823999 1181614079

And this is the same output in my terminal (black background):

]]> C https://leomrtns.github.io/posts/kalign3/ Tue, 01 Mar 2022 00:00:00 GMT Minimal example of a rolling hash Leo Martins https://leomrtns.github.io/posts/rolling-hash/

003 Minimal example of a rolling hash

This notebook uses the low-level phylogenetic library biomcmc-lib (commit 5975331).

The rolling hash (or expoential hash) is a technique to slide through a sequence while updating its hash value using only information from the current hash and the next value. One of the most popular algorithms is the Rabin-Karp algorithm. Below I start with the imnplementation in linclust/MMseqs2

In DNA context, each base is represented by a random number. In MMseqs they work with a reduced amino acid alphabet of 13 groups

Prototype

Here I describe a prototype (future capability of the library), not implemented yet. I am testing the rolling hash function but also shorter types uint8_t etc. The idea is that each of the testing functionalities will be incorporated into biomcmc-lib.

Below is a working version that follows closely minclust.

Code 
//%cflags:-lm
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib/.libs/libbiomcmc.a
#include <biomcmc.h>

uint16_t RAND[21] = {0x4567, 0x23c6, 0x9869, 0x4873, 0xdc51, 0x5cff, 0x944a, 0x58ec,
                           0x1f29, 0x7ccd, 0x58ba, 0xd7ab, 0x41f2, 0x1efb, 0xa9e3, 0xe146,
                           0x007c, 0x62c2, 0x0854, 0x27f8, 0x231b};// 16 bit random numbers

#define RoL(val, numbits) (val << numbits) ^ (val >> (32 - numbits))

// Transform each letter x[i] to a fixed random number RAND[x[i]] to ensure instantaneous mixing into the 16 bits
// Do XOR with RAND[x[i]] and 5-bit rotate left for each i from 1 to k
uint32_t circ_hash (const uint8_t * x, uint8_t length, const uint8_t rol){
    uint16_t h = 0x0;
    h = h ^ RAND[x[0]];                  // XOR h and ki
    for (uint8_t i = 1; i < length; ++i){
        h = RoL(h, rol);
        h ^= RAND[x[i]];                   // XOR h and ki
    }
    return h;
}

// Rolling hash for CRC hash variant: Computes hash value for next key x[0:length-1] from previous value
// hash( x[-1:length-2] ) and x_first = x[-1]
uint32_t circ_hash_next (const uint8_t * x, uint8_t length, uint8_t x_first, uint16_t h, const uint8_t rol){
    // undo INITIAL_VALUE and first letter x[0] of old key
    h ^= RoL(RAND[x_first], (5*(length-1)) % 16);// circularly permute all letters x[1:length-1] to 5 positions to left
    // h ^= RoL(RAND[x_first], (5*(length-1)) & 15); // since x%y = x & (y-1)
    h =  RoL(h, rol);// add new, last letter of new key x[1:length]
    h ^= RAND[x[length-1]];
    return h;
}

int main (){
    uint8_t i, j;
    uint8_t my_seq1[] = {1,2,3,1,2, 3,1,2,3,1, 1,1,1,1,1, 1,2,1,2,3, 1,2,1,2,1, 2,1,2}; // n=28
    uint8_t my_seq2[] = {1,2,3,4,5, 2,3,4,5,2, 3,4,5,1,1, 4,5,2,5,2, 3,4,3,2,3, 2,3,2}; 
    uint16_t my_h1 = circ_hash((const uint8_t*) my_seq1, 3, 5);
    uint16_t my_h2 = circ_hash((const uint8_t *) my_seq2, 3, 7);
    for (j = 0; j < 25; j++) {
        printf ("%3d >>  ", j);
        for (i=j;i<j+3;i++) printf ("%d ", my_seq1[i]); 
        printf (" \t %8u  \t >>  ",my_h1); // show current kmer and hash
        for (i=j;i<j+3;i++) printf ("%d ", my_seq2[i]); 
        printf (" \t %8u\n",my_h2); // show current kmer and hash
        my_h1 = circ_hash_next ((const uint8_t *) my_seq1 + j + 1, 3, my_seq1[j], my_h1, 5);
        my_h2 = circ_hash_next ((const uint8_t *) my_seq2 + j + 1, 3, my_seq2[j], my_h2, 7);
    }
}
  0 >>  1 2 3       23891    >>  1 2 3          64755
  1 >>  2 3 1       35238    >>  2 3 4          42449
  2 >>  3 1 2       11433    >>  3 4 5          46207
  3 >>  1 2 3       23891    >>  4 5 2          42985
  4 >>  2 3 1       35238    >>  5 2 3          48371
  5 >>  3 1 2       11433    >>  2 3 4          42449
  6 >>  1 2 3       23891    >>  3 4 5          46207
  7 >>  2 3 1       35238    >>  4 5 2          42985
  8 >>  3 1 1       38662    >>  5 2 3          48371
  9 >>  1 1 1       17158    >>  2 3 4          42449
 10 >>  1 1 1       17158    >>  3 4 5          46207
 11 >>  1 1 1       17158    >>  4 5 1           7238
 12 >>  1 1 1       17158    >>  5 1 1            198
 13 >>  1 1 1       17158    >>  1 1 4          48977
 14 >>  1 1 2       63657    >>  1 4 5          62591
 15 >>  1 2 1       14054    >>  4 5 2          42985
 16 >>  2 1 2       17577    >>  5 2 5          43135
 17 >>  1 2 3       23891    >>  2 5 2          42985
 18 >>  2 3 1       35238    >>  5 2 3          48371
 19 >>  3 1 2       11433    >>  2 3 4          42449
 20 >>  1 2 1       14054    >>  3 4 3          41203
 21 >>  2 1 2       17577    >>  4 3 2          57833
 22 >>  1 2 1       14054    >>  3 2 3          48371
 23 >>  2 1 2       17577    >>  2 3 2          57833
 24 >>  1 2 1       14054    >>  3 2 3          48371

The code above assumes a 16-bits hash, cast to 32 bits (due to the RoL macro). Below we use all 32 bits of hash, and also use the fact that for non-negative values, x % y == x & (y-1) which may be faster.

Code 
//%cflags:-lm
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib/.libs/libbiomcmc.a
#include <biomcmc.h>

uint32_t RAND[21] = {0x4567, 0x23c6, 0x9869, 0x4873, 0xdc51, 0x5cff, 0x944a, 0x58ec,
                           0x1f29, 0x7ccd, 0x58ba, 0xd7ab, 0x41f2, 0x1efb, 0xa9e3, 0xe146,
                           0x007c, 0x62c2, 0x0854, 0x27f8, 0x231b};

#define RoL(val, numbits) (val << numbits) ^ (val >> (32 - numbits))
uint32_t circ_hash (const uint8_t * x, uint8_t length, const uint8_t rol){
    uint32_t h = 0x0;
    h = h ^ RAND[x[0]];                  // XOR h and ki
    for (uint8_t i = 1; i < length; ++i){
        h = RoL(h, rol);
        h ^= RAND[x[i]];                 // XOR h and ki
    }
    return h;
}

uint32_t circ_hash_next (const uint8_t * x, uint8_t length, uint8_t x_first, uint32_t h, const uint8_t rol){
    uint8_t remain = (rol * (length-1)) & 31U; // since x % y = x & (y-1)
    // undo INITIAL_VALUE and first letter x[0] of old key
    //h ^= RoL(RAND[x_first], (rol * (length-1)) % 32);// circularly permute all letters x[1:length-1] to 5 positions to left
    h ^= RoL(RAND[x_first], remain);
    h =  RoL(h, rol);// add new, last letter of new key x[1:length]
    h ^= RAND[x[length-1]];
    return h;
}

int main (){
    uint8_t i, j;
    uint8_t my_seq1[] = {1,2,3,1,2, 3,1,2,3,1, 1,1,1,1,1, 1,2,1,2,3, 1,2,1,2,1, 2,1,2}; // n=28
    uint8_t my_seq2[] = {1,2,3,4,5, 2,3,4,5,2, 3,4,5,1,1, 4,5,2,5,2, 3,4,3,2,3, 2,3,2}; 
    uint32_t my_h1 = circ_hash((const uint8_t *) my_seq1, 4, 5);
    uint32_t my_h2 = circ_hash((const uint8_t *) my_seq2, 4, 7);
    for (j = 0; j < 25; j++) {
        printf ("%3d >>  ", j);
        for (i=j;i<j+4;i++) printf ("%d ", my_seq1[i]); 
        printf (" %8u  \t >>  ",my_h1); // show current kmer and hash
        for (i=j;i<j+4;i++) printf ("%d ", my_seq2[i]); 
        printf ("\t%8u\n",my_h2); // show current kmer and hash
        my_h1 = circ_hash_next ((const uint8_t *) my_seq1 + j + 1, 4, my_seq1[j], my_h1, 5);
        my_h2 = circ_hash_next ((const uint8_t *) my_seq2 + j + 1, 4, my_seq2[j], my_h2, 7);
    }
}
  0 >>  1 2 3 1  327911846       >>  1 2 3 4    1593746901
  1 >>  2 3 1 2  1293003945      >>  2 3 4 5    525513836
  2 >>  3 1 2 3  614849875       >>  3 4 5 2    962242528
  3 >>  1 2 3 1  327911846       >>  4 5 2 3    2639510760
  4 >>  2 3 1 2  1293003945      >>  5 2 3 4    3118376410
  5 >>  3 1 2 3  614849875       >>  2 3 4 5    525513836
  6 >>  1 2 3 1  327911846       >>  3 4 5 2    962242528
  7 >>  2 3 1 1  1292965638      >>  4 5 2 3    2639510760
  8 >>  3 1 1 1  615695110       >>  5 2 3 4    3118376410
  9 >>  1 1 1 1  292045574       >>  2 3 4 5    525513836
 10 >>  1 1 1 1  292045574       >>  3 4 5 1    962206799
 11 >>  1 1 1 1  292045574       >>  4 5 1 1    2634940637
 12 >>  1 1 1 1  292045574       >>  5 1 1 4    2533408602
 13 >>  1 1 1 2  292092073       >>  1 1 4 5    1885336699
 14 >>  1 1 2 1  293549798       >>  1 4 5 2    1341827053
 15 >>  1 2 1 2  327566505       >>  4 5 2 5    2639505508
 16 >>  2 1 2 3  1286135123      >>  5 2 5 2    3117721570
 17 >>  1 2 3 1  327911846       >>  2 5 2 3    441695456
 18 >>  2 3 1 2  1293003945      >>  5 2 3 4    3118376410
 19 >>  3 1 2 1  614840038       >>  2 3 4 3    525508832
 20 >>  1 2 1 2  327566505       >>  3 4 3 2    961602016
 21 >>  2 1 2 1  1286125286      >>  4 3 2 3    2557525224
 22 >>  1 2 1 2  327566505       >>  3 2 3 2    677306848
 23 >>  2 1 2 1  1286125286      >>  2 3 2 3    527482080
 24 >>  1 2 1 2  327566505       >>  3 2 3 2    677306848
Code 
/* Short interlude to play with table indices :D */
#include <stdio.h>
int main() {
    int t1 = 16, t2 = 4, i1, i2, salt; 
    for (salt = 0; salt < 72; salt++) {
        i1 = (salt & ((t1 >> 2) - 1)) << 2;
        i2 = (int)(salt / (t1 >> 2))%t2;
        printf ("%3d %3d %3d| ", salt, i1, i2);
        if (((salt+1)%8)==0) printf ("\n");
    }
}
  0   0   0|   1   4   0|   2   8   0|   3  12   0|   4   0   1|   5   4   1|   6   8   1|   7  12   1| 
  8   0   2|   9   4   2|  10   8   2|  11  12   2|  12   0   3|  13   4   3|  14   8   3|  15  12   3| 
 16   0   0|  17   4   0|  18   8   0|  19  12   0|  20   0   1|  21   4   1|  22   8   1|  23  12   1| 
 24   0   2|  25   4   2|  26   8   2|  27  12   2|  28   0   3|  29   4   3|  30   8   3|  31  12   3| 
 32   0   0|  33   4   0|  34   8   0|  35  12   0|  36   0   1|  37   4   1|  38   8   1|  39  12   1| 
 40   0   2|  41   4   2|  42   8   2|  43  12   2|  44   0   3|  45   4   3|  46   8   3|  47  12   3| 
 48   0   0|  49   4   0|  50   8   0|  51  12   0|  52   0   1|  53   4   1|  54   8   1|  55  12   1| 
 56   0   2|  57   4   2|  58   8   2|  59  12   2|  60   0   3|  61   4   3|  62   8   3|  63  12   3| 
 64   0   0|  65   4   0|  66   8   0|  67  12   0|  68   0   1|  69   4   1|  70   8   1|  71  12   1|

transforming DNA bases into hash values

Above, we have a single hash value RAND[] for each base, but we can create a stream of values.

  1. we define a translation table from DNA to for non-ambiguous data (or some other encoding). Notice that we must know its reverse complement.
  2. hash values inspired by RAND[] are then created. They are “salted” with large primes.

The version currently implemented in biomcmc-lib is, for non-ambiguous DNA:

uint8_t dna_in_2_bits[256][2] = {{0xff}};

void
initialize_dna_to_bit_tables (void) {
    int i;
    for (i = 0; i < 256; i++) dna_in_2_bits[i][0] = dna_in_2_bits[i][1] = 4; // calling function must check if < 4
    dna_in_2_bits['A'][0] = 0;   dna_in_2_bits['A'][1] = 3;  /*  A  <-> T  */
    dna_in_2_bits['C'][0] = 1;   dna_in_2_bits['C'][1] = 2;  /*  C  <-> G  */
    dna_in_2_bits['G'][0] = 2;   dna_in_2_bits['G'][1] = 1;  /*  G  <-> C  */
    dna_in_2_bits['T'][0] = 3;   dna_in_2_bits['T'][1] = 0;  /*  T  <-> A  */
    dna_in_2_bits['U'][0] = 3;   dna_in_2_bits['U'][1] = 0;  /*  U  <-> A  */
}

It just returns a small number, that is concatenated to form the k-mer. I.e. a sequence ACGGT becomes an integer with binary 00 01 10 10 01.

Below is the tentative code, closer to my final implementation (notice that it doesn’t compress homopolymers). I also implemented the reverse-complement rolling hash (implemented using a different algorithm in ntHash).

Code 
//%cflags:-lm
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib/.libs/libbiomcmc.a
#include <biomcmc.h>
const char my_dna_seq[] = 
    "AAACCACCTCCCGGTGGTTT" // first 8 and last 8 are revcompl (AAACC<->GGTTT)
    "AAACCACCTCCCGGTGGTTT"
    "GTTCTTAACATTTCTCGTAC"
    "GTTCTTAACATTTCTCGTAC";

uint32_t rand_hash_list[] = {0x4567, 0x23c6, 0x9869, 0x4873, 0xdc51}; // size = 5
uint32_t prime_salt_list[] = { // size = 64 
  0x343EAF9F, 0x75BD32B,  0x5E1C5A87, 0x343EFDAF, 0x1FDCDBEF, 0x6389CB,   0x1FDCE507, 0x1FDCE3C7, 
  0x1FDCE15F, 0x75BD431,  0x34C8B0F,  0x5397FEF,  0x87413,    0x6389C9,   0x34C8B23,  0x343EFDB5, 
  0x75BD479,  0x5398013,  0x5397FCD,  0x87407,    0x75BD413,  0x343EAF4B, 0x343EB047, 0x6389CF, 
  0x5E1C5A2F, 0x343EFD31, 0x34C8B59,  0x343EFD57, 0x1FDCDC19, 0x5E1C5B0D, 0x34C8B5D,  0x343EB035, 
  0x75BD307,  0x343EAF7B, 0x343EAEF5, 0x75BD433,  0x1FDCDBD5, 0x1FDCE3B7, 0x343EFD19, 0x5398057, 
  0x34C8B51,  0x638971,   0x1FDCE16F, 0x63896F,   0x343EB04D, 0x873FD,    0x63896B,   0x1FDCDC9B, 
  0x5E1C5B1F, 0x343EFCEB, 0x5398007,  0x1FDCE51F, 0x63897B,   0x5E1C5B17, 0x5E1C5ABD, 0xFF85, 
  0x1FDCDC25, 0x873EB,    0x5E1C5AB1, 0x75BD30D,  0x1FDCDCC1, 0xFF8B,     0x5397FC1,  0x5398001};

uint32_t**
new_dna_salted_hash_encoding (uint8_t salt) {
    uint8_t i=255, j;
    uint32_t **shash = (uint32_t**) biomcmc_malloc (2 * sizeof (uint32_t*)); // opposite order as dna_in_2_bits[]
    for (i = 0; i < 2; ++i) shash[i] = (uint32_t*) biomcmc_malloc (256 * sizeof (uint32_t));
    /* notice do{}while() instead of for() since i is always < 256 */
    do { shash[0][i] = shash[1][i] = 4;} while (i--); // anything else is fifth state
    
    shash[0]['A'] = shash[0]['a'] = 0; shash[1]['A'] = shash[1]['a'] = 3;  /*  A  <-> T  */
    shash[0]['C'] = shash[0]['c'] = 1; shash[1]['C'] = shash[1]['c'] = 2;  /*  C  <-> G  */
    shash[0]['G'] = shash[0]['g'] = 2; shash[1]['G'] = shash[1]['g'] = 1;  /*  G  <-> C  */
    shash[0]['T'] = shash[0]['t'] = 3; shash[1]['T'] = shash[1]['t'] = 0;  /*  T  <-> A  */
    shash[0]['U'] = shash[0]['u'] = 3; shash[1]['U'] = shash[1]['u'] = 0;  /*  U  <-> A  */
    
    salt &= 63; // modulus, we only have 64 salts
    /** now we transform the indexes for their equiv. hash values; all have same salt */
    for (j = 0; j < 2; ++j) {
        i = 255; do { shash[j][i] = rand_hash_list[shash[j][i]] + prime_salt_list[salt]; } while (i--);
    }
    return shash;
}

void
del_dna_salted_hash_encoding (uint32_t** shash) {
    if (!shash) return;
    if (shash[1]) free (shash[1]);
    if (shash[0]) free (shash[0]);
    free (shash);
}

#define RoL(val, numbits) ((val) << (numbits)) | ((val) >> (32 - (numbits)))
#define RoR(val, numbits) ((val) >> (numbits)) | ((val) << (32 - (numbits)))
void
roll_hash_add (uint32_t *h, const char dna_base, const uint8_t rol, const uint32_t* shashcode)
{
  *h = RoL(*h, rol);
  *h ^= shashcode[(uint8_t) dna_base]; // XOR h and ki
}

void // kmer_size can't be 0 or 32 
roll_hash_replace_f (uint32_t *h, const char old_base, const char new_base, const uint8_t kmer_size, 
                   const uint8_t rol, const uint32_t* shashcode)
{
    uint8_t remain = (rol * (kmer_size-1)) & 31U; // since x % y = x & (y-1) // this can be calculated outside
    *h ^= RoL(shashcode[(uint8_t) old_base], remain); // remove "leftmost" base 
    *h  = RoL(*h, rol);
    *h ^= shashcode[(uint8_t) new_base];
}
void // kmer_size can't be 0 or 32 
roll_hash_replace_r (uint32_t *h, const char old_base, const char new_base, const uint8_t kmer_size, 
                   const uint8_t rol, const uint32_t* shashcode)
{
    uint8_t remain = (rol * (kmer_size-1)) & 31U; // since x % y = x & (y-1) // this can be calculated outside 
    *h ^= shashcode[(uint8_t) old_base];
    *h  = RoR(*h, rol);
    *h ^= RoL(shashcode[(uint8_t) new_base], remain);  
}
#undef RoL
#undef RoR

int main (){
    uint8_t kmer_size = 5, rol = 5;
    uint32_t h1, h2, h1R, h2R;
    size_t i, j, seqlen = strlen (my_dna_seq);
    uint32_t **shcode1, **shcode2;
    shcode1 = new_dna_salted_hash_encoding (3);
    shcode2 = new_dna_salted_hash_encoding (6);
    h1  = shcode1[0][(uint8_t) my_dna_seq[0]];
    h2  = shcode2[0][(uint8_t) my_dna_seq[0]];
    h1R = shcode1[1][(uint8_t) my_dna_seq[kmer_size-1]];
    h2R = shcode2[1][(uint8_t) my_dna_seq[kmer_size-1]];
    
    for (j = 1; j < kmer_size; ++j) {
        roll_hash_add (&h1, my_dna_seq[j], rol, shcode1[0]); 
        roll_hash_add (&h2, my_dna_seq[j], rol, shcode2[0]);
        roll_hash_add (&h1R, my_dna_seq[kmer_size-1-j], rol, shcode1[1]); 
        roll_hash_add (&h2R, my_dna_seq[kmer_size-1-j], rol, shcode2[1]);
    }

    for (j = (size_t) kmer_size; j < seqlen; ++j) {
       if ((j-kmer_size)%kmer_size == 0) {
            printf ("%3lu >>  ", j - kmer_size);
            for (i=j - (size_t) kmer_size; i < j; i++) printf ("%c ", my_dna_seq[i]); 
            printf (" >> %12u >> %12u || %12u << %12u\n",h1, h1R, h2, h2R); 
        }
        roll_hash_replace_f (&h1, my_dna_seq[j - kmer_size], my_dna_seq[j], kmer_size, rol, shcode1[0]); 
        roll_hash_replace_f (&h2, my_dna_seq[j - kmer_size], my_dna_seq[j], kmer_size, rol, shcode2[0]); 
        roll_hash_replace_r (&h1R, my_dna_seq[j - kmer_size], my_dna_seq[j], kmer_size, rol, shcode1[1]);
        roll_hash_replace_r (&h2R, my_dna_seq[j - kmer_size], my_dna_seq[j], kmer_size, rol, shcode2[1]);
    }
    del_dna_salted_hash_encoding (shcode1);
    del_dna_salted_hash_encoding (shcode2);
}
  0 >>  A A A C C  >>   3728674536 >>   3829420882 ||   2734906157 <<   4185167999
  5 >>  A C C T C  >>   4002146312 >>   2375924754 ||   3000820173 <<   3551637823
 10 >>  C C G G T  >>   3397139481 >>   4002502693 ||   2570072376 <<   3000407344
 15 >>  G G T T T  >>   3829420882 >>   3728674536 ||   4185167999 <<   2734906157
 20 >>  A A A C C  >>   3728674536 >>   3829420882 ||   2734906157 <<   4185167999
 25 >>  A C C T C  >>   4002146312 >>   2375924754 ||   3000820173 <<   3551637823
 30 >>  C C G G T  >>   3397139481 >>   4002502693 ||   2570072376 <<   3000407344
 35 >>  G G T T T  >>   3829420882 >>   3728674536 ||   4185167999 <<   2734906157
 40 >>  G T T C T  >>   2354166194 >>   3031712392 ||   3514117791 <<   2290618189
 45 >>  T A A C A  >>   2373938315 >>   3890239083 ||   3527637390 <<   4191461230
 50 >>  T T T C T  >>   2414983615 >>   3031703787 ||   3520409242 <<   2290626030
 55 >>  C G T A C  >>   2458948238 >>   2353379720 ||   2728681295 <<   3513331349
 60 >>  G T T C T  >>   2354166194 >>   3031712392 ||   3514117791 <<   2290618189
 65 >>  T A A C A  >>   2373938315 >>   3890239083 ||   3527637390 <<   4191461230
 70 >>  T T T C T  >>   2414983615 >>   3031703787 ||   3520409242 <<   2290626030

conclusions

I’ve found a bug in linclust that does not affect MMseqs2, as we see from the above.

]]> C https://leomrtns.github.io/posts/rolling-hash/ Tue, 01 Feb 2022 00:00:00 GMT Quantile functions implemented in biomcmc-lib Leo Martins https://leomrtns.github.io/posts/quantiles/

002 Quantile functions implemented in biomcmc-lib

There are (as of 2019.11.29) two functions implementing the quantile finding in biomcmc-lib: the quickselect and the Wirth algorithm. By “quantile finding” I mean returning the k-smallest element from a vector (e.g. the median). Both algorithms work by modifying the original vector, but in some cases biomcmc-lib works on a copy of it:

// quickselect, copies the vector (quantile specified as fraction over n)
double biomcmc_quantile_double (double *original_vector, int n, double quantile);

// Wirth algorithm, modifies the vector (quantile specified as an integer k < n )
double biomcmc_wirth_algorithm (double *a, int n, int k); 

// copies the vector and returns several quantile values at once using the Wirth algorithm
void   biomcmc_quantile_vector_double (double *original_vector, int n, double *quantile, int n_quantile, double *result);

The algorithms do not interpolate, as in some median-finding algorithms — it finds the k-smallest element, therefore if there is a tie it chooses the … smallest!

Code 
//%cflags:-lm
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib/.libs/libbiomcmc.a
#include <biomcmc.h>

int main (){
    double this, unsorted[] = {1.1, 6.6, 3.3, 8.8, 7.7, 4.4, 4.3, 9.1, 5.5, 9.9};
    int i;
    printf ("before:\n");
    for (i=0; i < 10; i++) printf ("%3.2lf ", unsorted[i]);
    this = biomcmc_wirth_algorithm (unsorted, 10, 1);
    printf ("\nafter finding %3.2lf (Wirth):\n", this);
    for (i=0; i < 10; i++) printf ("%3.2lf ", unsorted[i]);
    this = biomcmc_quantile_double (unsorted, 10, 0.2);
    printf ("\nafter finding %3.2lf (quickselect):\n", this);
    for (i=0; i < 10; i++) printf ("%3.2lf ", unsorted[i]);
    printf ("\n");
}
before:
1.10 6.60 3.30 8.80 7.70 4.40 4.30 9.10 5.50 9.90 
after finding 3.30 (Wirth):
1.10 3.30 4.30 4.40 5.50 7.70

biomcmc-lib commit: 5975331

]]> C https://leomrtns.github.io/posts/quantiles/ Sat, 01 Jan 2022 00:00:00 GMT K-mer in biomcmc-lib Leo Martins https://leomrtns.github.io/posts/kmers/

K-mer in biomcmc-lib (2019.11.29)

This notebook contains a few details of how kmers are implemented in the low-level phylogenetic library biomcmc-lib. This library is opaque to the user, and contains functions that are common to several projects. The functions discussed here relate to the creation of the k-mers (and their hash values) for a given DNA sequence.

Please have in mind that these functions are likely to change or at least change names soon — they are general enough to have allowed me to play with kmers, but are too complex for production-level software. Two missing functionalities are:

  1. inexact (masked) k-mers.
  2. rolling hashes
  3. using 32 bits per hash value (64 bits may be overkill)
  4. homopolymer compression
  5. fine-grained control over k-mer sizes

Some code shown here depends on local libraries, which must be linked at compilation time (//%cflags:-lm etc.)

1.1 Parameters of a kmer set

Since it may be onerous to scan through the DNA sequence several times, I decided to generate kmers for several choices of k in one pass. The library uses 128 bits to store each kmer, and immediately transforms it into its 64 bits hash value.

Furthermore there are 3 possible encodings for each base: 1, 2, and 4 bits per base. The standard used by most kmer algorithms is equivalent to 2 bits per base (A->00, C->01, G->10, T->11). The other two are a compact, GC-content-like one bit per base(A,T->0, and C,G->1), and one common in the phylogenetic likelihood structure, that can accommodate ambiguous characters. For instance, since M represents an amino group (C or A), then we can have A->0001, C->0010, M->0011, and N->1111.

The code below is implemented in biomcmc-lib (2019.11.29), and is responsible for selecting the sets of k-mer sizes depending on an integer mode given by the user. This code is subject to change.

static uint64_t _tbl_mask[] = {0xffffUL, 0xffffffUL, 0xffffffffUL, 0xffffffffffUL, 0xffffffffffffUL, 0xffffffffffffUL, 0xffffffffffffffffUL};
static uint8_t _tbl_shift[] = {      48,         40,           32,             24,               16,                8,                    0};
static uint8_t _tbl_nbyte[] = {       2,          3,            4,              5,                6,                7,                    8};
static uint32_t _tbl_seed[] = {0x9040a6, 0x10bea992,   0x50edd67d,     0xb05a4f09,       0xf07046c5,       0x9c9445ab,           0xb2500f29};

/* i1[] and i2[] (i.e. elements above to be used on first and second 64bits, respectively */
static uint8_t _idx_mode[][2][7] = { // contains list of elements from _tbl above to be used, from 1st and 2nd 64bit blocks
   { {2,6,0,0,0,0,0}, {0,0,0,0,0,0,0} },  
   { {2,6,0,0,0,0,0}, {2,6,0,0,0,0,0} },  
   { {0,2,4,6,0,0,0}, {2,6,0,0,0,0,0} }, 
   { {0,1,2,4,6,0,0}, {0,2,6,0,0,0,0} },
   { {0,1,2,3,4,5,6}, {0,0,0,0,0,0,0} },
   { {0,1,2,3,4,5,6}, {0,1,2,6,0,0,0} }
};   
static uint8_t _n_idx[][2] = { {2,0}, {2,2}, {4,2}, {5,3}, {7,0}, {7,4} }; // how many elems from _idx_mode[] are used
   

kmer_params
new_kmer_params (int mode)
{
  uint8_t  i, j, row, bases_per_byte, _ba_pe_by[] = {2,4,8}; // bases_per_byte is 4 if dense
  kmer_params p = (kmer_params) biomcmc_malloc (sizeof (struct kmer_params_struct));
  p->ref_counter = 1;
  p->hashfunction = &biomcmc_xxh64;
  if (dna_in_4_bits[0][0] == 0xff) initialize_dna_to_bit_tables (); // run once per program
  p->kmer_class_mode = mode;
  switch (mode) { // map each choice to a set of kmers and bitstring encoding (row relates to _idx_mode[] above)
    case 0:  row = 0; p->dense = 1; break;
    case 1:  row = 2; p->dense = 1; break;
    case 2:  row = 3; p->dense = 0; break;
    case 3:
    default: row = 4; p->dense = 1; break;
    case 4:  row = 5; p->dense = 0; break;
    case 5:  row = 1; p->dense = 2; break;
  };
  bases_per_byte = _ba_pe_by[p->dense];

  p->n1 = (uint8_t) _n_idx[row][0]; p->n2 = (uint8_t) _n_idx[row][1];
  for (j=0; j < p->n1; j++) {
    i = _idx_mode[row][0][j];
    p->mask1[j] = _tbl_mask[i];
    p->shift1[j] = _tbl_shift[i];
    p->seed[j] = _tbl_seed[i];
    p->nbytes[j] = _tbl_nbyte[i];
    p->size[j] = _tbl_nbyte[i] * bases_per_byte;
  }
  for (j=0; j < p->n2; j++) {
    i = _idx_mode[row][1][j];
    p->mask2[j] = _tbl_mask[i];
    p->shift2[j] = _tbl_shift[i];
    p->seed[j] = (_tbl_seed[i] >> 2) + 0x420314a1d; // very noise, much random
    p->nbytes[j+p->n1] = _tbl_nbyte[i] + 8;
    p->size[j+p->n1] = (_tbl_nbyte[i] + 8) * bases_per_byte;
  }
  return p;
}

(out of curiosity, Liquid cannot handle the consecutive curly braces (https://github.com/Shopify/liquid/issues/927) thus I need to separate as in { {)

The small program below prints the resulting set of parameters for each choice of mode. Notice that the code must be linked to the libbiomcmc.a local library (some programs, like guenomu still install a global version of this library, but newer programs should use it locally, to avoid version conflicts and since it’s smaller than one MB…)

Code 
//%cflags:-lm
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/biomcmc-lib/lib
//%cflags: -I/usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib
//%cflags: /usr/users/QIB_fr005/deolivl/Academic/Quadram/009.supersptree/build/biomcmc-lib/lib/.libs/libbiomcmc.a
#include <biomcmc.h>

int main (){
    uint8_t i, j;
    for (j = 0; j < 6; j++) {
        kmer_params kp = new_kmer_params (j);
        printf ("<<%d>>\n", j);
        for (i=0;i<kp->n1;i++) printf("\t %3d %3d < 1 >\n", kp->nbytes[i], kp->size[i]);
        for (i=0;i<kp->n2;i++) printf("\t %3d %3d < 2 >\n", kp->nbytes[i + kp->n1], kp->size[i + kp->n1]);
        del_kmer_params (kp);
    }
}
<<0>>
       4  16 < 1 >
       8  32 < 1 >
<<1>>
       2   8 < 1 >
       4  16 < 1 >
       6  24 < 1 >
       8  32 < 1 >
      12  48 < 2 >
      16  64 < 2 >
<<2>>
       2   4 < 1 >
       3   6 < 1 >
       4   8 < 1 >
       6  12 < 1 >
       8  16 < 1 >
      10  20 < 2 >
      12  24 < 2 >
      16  32 < 2 >
<<3>>
       2   8 < 1 >
       3  12 < 1 >
       4  16 < 1 >
       5  20 < 1 >
       6  24 < 1 >
       7  28 < 1 >
       8  32 < 1 >
<<4>>
       2   4 < 1 >
       3   6 < 1 >
       4   8 < 1 >
       5  10 < 1 >
       6  12 < 1 >
       7  14 < 1 >
       8  16 < 1 >
      10  20 < 2 >
      11  22 < 2 >
      12  24 < 2 >
      16  32 < 2 >
<<5>>
       4  32 < 1 >
       8  64 < 1 >
      12  96 < 2 >
      16 128 < 2 >

biomcmc-lib commit: 5975331

]]> C https://leomrtns.github.io/posts/kmers/ Wed, 01 Dec 2021 00:00:00 GMT Welcome to the blog Leo Martins https://leomrtns.github.io/posts/welcome-binfie-notes/ You can find more information about me on the parent site (https://leomrtns.github.io/about/).

This blog will consist mostly of technical notes, but I don’t preclude an occasional rant. Hopefully I’ll move some old posts from other, defunct blogs. You can also find me on a mastodon server https://scicomm.xyz/web/@leomrtns.

Update: in 2025, I moved to Liverpool and started a new job at the University of Liverpool as a Tenure Track Fellow in Health at the Department of Electrical Engineering and Electronics. I am no longer at the Quadram Institute Biosciences (and emails to my old address will bounce).

]]> random https://leomrtns.github.io/posts/welcome-binfie-notes/ Mon, 01 Jan 2018 00:00:00 GMT