Writing

We tried to beat Pfam for BGC retrieval. Pfam won.

A Set Transformer over frozen ESM-2 embeddings lost to plain Pfam-domain Jaccard on biosynthetic gene cluster retrieval. What a negative result says about baselines and evaluation.

Fine-tuning ESM-2 for protein subcellular localization: LoRA matches full fine-tuning with 2.4% of the parameters

Rank-16 LoRA on ESM-2 650M reaches the accuracy of a fully fine-tuned 150M model while training 2.4% of the weights — and a frozen 8M linear probe is already most of the way there.