LLMs for bioinformaticians

What they are useful for, how to work with them and where they can fail

Dr Jamie Soul

Associate Director, Computational Biology Facility, University of Liverpool

18th September 2026

What am I talking about?

Aims

  • Introduce the concepts, the dangers and the opportunities
  • Learn something new
  • Give the panel afterwards something to argue about

Understand

  1. What is an LLM?
  2. LLMs as a bioinformatician’s tool

Use

  1. Prompting as a research skill
  2. Which model should I use?
  3. Agentic AI

Be careful

  1. How it goes wrong
  2. Take homes, and where this is going

1. What is an LLM?

How it works

When we say LLM today, we usually mean a large language model that generates text.

Your text
tokens
the model
next-token odds
pick one
“My favourite programming language is”
Myfavouriteprogramminglanguageis
R46%
Python31%
Rust8%
Julia4%
R
Usually the top one, but chance is a setting
append it, and repeat: “My favourite programming language is R” …

Trained on text and code up to a cut-off date. Facts emerge from the patterns.

It isn’t a database. When it doesn’t know, it may still write something that sounds right.

Instruction tuning

A pretrained model continues text. Answering a question is a behaviour that has to be trained in.

Instruction tuning and preference training turn continue this document into do what I asked.

  • Helpfulness comes from this step
  • So does agreeableness - “You are right to push back on that …”
  • A model trained to please people learns what people approve of, not necessarily what is true.

It may agree with you even when you’re wrong!

Reasoning and tools

Improving the answer

Reasoning

Write out working before answering. More thinking ≠ more knowledge

Tools

Give the model access to information or computation it doesn’t have internally.

A database, web search or your own documents, R/python.

The answer comes out of the tool not the model’s knowledge

Tools have made LLMs much more useful in bioinformatics.

Demo — the same question, with and without tools

R console · ellmer · qwen3:0.6b running locally in Ollama

×

The same question, asked with and without a tool.

An R function sends the prompt to a model running locally on this laptop.

Is this right?

Press play to ask again, this time with a tool

What made the difference

chat$register_tool(tool(
  lookup_gene,
  "Look up the chromosome and exon count for a gene symbol.",
  arguments = list(symbol = type_string("A HGNC gene symbol, e.g. SOX9"))
))

From demos/03-tool-calling.R — hover to copy

2. LLMs as a bioinformatician’s tool

What can we use them for?

Looks like

Understand What does this inherited Perl script do? What does this error mean? Other terms a paper might use

Generate sbatch scripts, a plotting function, documentation for code that already works

Transform R to Python, notes into slides

Review Why is this loop slow?

Use tools Install the software, run it, read the traceback, fix it, run it again

decide is deliberately missing!

They work best when you can tell immediately whether it worked.

LLMs are now integrated in many tools: MultiQC

The LLM doesn’t have to be a separate chat box. It can sit inside the tools you already use.

Seqera (2025) AI Summaries now available in MultiQC

Asking an LLM to annotate cells: impressive, but…

Would you know which clusters it got wrong?

Manual annotation GPT-4 answer Agreement
Adipocyte Adipocytes Full
B cell_memory B cells Partial
Fibroblast Fibroblasts Full
Luminal Epithelial Luminal epithelial cells Full
Lymphatic Endothelial Lymphatic endothelial cells Full
Macrophage Macrophages Full
Mast Cell Mast cells Full
Pericyte Pericytes Full
Smooth Muscle Smooth muscle cells Full
T cell T cells Full
Vascular Endothelial Endothelial cells Partial

One dataset shown. Across ten datasets, most cell types fully or partly matched the manual annotation.

Hou & Ji (2024) Assessing GPT-4 for cell type annotation in single-cell RNA-seq analysis. Nature Methods 21:1462-1465, Fig. 1b, 1e (cropped), CC BY 4.0

Getting an LLM to rewrite scientific software

RustQC — 15 RNA-seq QC tools reimplemented in Rust, built by someone who hadn’t written Rust before.

15 h 34 m → 14 m 54 s on ~186M paired-end reads.

edge* — edgeR → Python in about a week. Code written by Claude and Codex, direction and maths by a human.

Only useful because the outputs were compared against the original tools.

Seqera (2026) Introducing RustQC: 15 RNA-Seq QC tools in one pass, built with AI; Pachter (2026) Differential analysis of genomics count data with edge*, bioRxiv 2026.02.16.706223 · rewrites.bio — principles for AI-assisted rewrites of scientific software

My own work: from GEO free text to perturbMatch

GSE222598 Effect of SF3B4 depletion on gene expression of A549 lung cancer cell

Sample title genotype treatment
A549 cells, si-NC, biol rep1 WT NC siRNA
A549 cells, si-NC, biol rep2 WT NC siRNA
A549 cells, si-NC, biol rep3 WT NC siRNA
A549 cells, si-SF3B4, biol rep1 SF3B4 knockdown SF3B4 siRNA
A549 cells, si-SF3B4, biol rep2 SF3B5 knockdown SF3B4 siRNA
A549 cells, si-SF3B4, biol rep3 SF3B6 knockdown SF3B4 siRNA

The problem

  • Thousands of gene perturbation experiments in GEO
  • No field says which samples are the cases and which the controls
  • Controls can be ctrl, wt, NC siRNA, empty vector …
  • Even the metadata can be wrong: spreadsheet autofill

Fine-tune small local models

  • Qwen3 4B and 30B, fine-tuned on 3,300 curated GEO series, on one GPU
  • Held-out test set, released after 2023: precision 0.925, recall 0.836
  • Small models without fine-tuning get many groups wrong (red)

perturbMatch: give it your differential expression result, and it ranks the gene perturbations that mimic or reverse it.

6,802 signatures 4,453 GEO series 2,907 genes

Soul & Young (2026) bioRxiv 2026.08.08.743502, Fig. 3B · github.com/soulj/perturbMatch

What changes for us?

You

  • ask the question
  • design the analysis
  • define what “correct” means
  • verify the result
  • decide when it’s finished
  • defend the result

LLM

  • write the boilerplate
  • install and configure software
  • write/refactor the code
  • run tools
  • inspect errors
  • iterate

Your name is still on the paper.

3. Prompting as a research skill

Try it: write the prompt

You have an RNA-seq genes x sample counts matrix and a sample sheet: treated vs control, five replicates each.

What are the key points of a prompt to get an LLM to write the differential expression analysis code?

Vague vs specified

Vague

Write me some R code to do differential expression.

You will get something, likely to be inefficient/incorrect.

Specified

Role — bioinformatician working in R and Bioconductor.

Task — differential expression, treated vs control.

Context — counts in counts.rds, sample sheet attached, five replicates per group, batch in the run column.

Constraints — edgeR, not DESeq2. No new dependencies. Do not rename the gene IDs.

Output — one script, and list any packages it needs.

Examples — follow the structure of previous_de.R.

Evaluation — rows in and rows out at each step, and the number of genes at FDR < 0.05.

When the output is bad, check for a missing part.

What you know is important!

Anthropic classified ~400,000 real Claude Code sessions by how expert the user was at the task, not at programming.

Verified success: 15% of novice sessions, 28–33% once the user is intermediate or better.

Most of the gain is the novice intermediate step. After that the curve flattens.

Your biology, your data science, your knowledge of the data.

Hitzig et al. (2026) Agentic coding and persistent returns to expertise. Anthropic, Fig. 5 (left panel). ~400,000 sessions, ~235,000 users, October 2025 to April 2026.

Expertise shows most when it goes wrong

Tell it what you know — which packages we use, which ones are unreliable, what the output should look like, what goes wrong with this kind of dataset.

You are the expert driving the analysis, not Claude.

Hitzig et al. (2026), Fig. 5 (middle and right panels). Sessions with more than three failure signals.

4. Which model should I use?

Do you need an LLM at all?

Macro-average, 12 BioNLP benchmarks, GPT-4-era models

Approach Score
Fine-tuned BERT / BART 0.65
LLM zero-shot 0.38
LLM five-shot 0.48
LLM fine-tuned 0.51

A fine-tuned BERT (~100M parameters) beat LLMs hundreds of times its size on most extraction and classification tasks, and is far cheaper to run.

Chen et al. (2025) Benchmarking large language models for biomedical natural language processing applications and recommendations. Nature Communications 16:3280, Fig. 4 and Table 2. CC BY.

A leaderboard doesn’t necessarily measure your task

100% at competition maths. 29% at finding DE genes.

“Many agents encounter running issues when calling algorithms for batch-effect correction”

Test your own examples

Liu et al. (2026) Benchmarking AI Agents for Addressing Scientific Challenges Across Scales. arXiv:2606.12736, Fig. 1e. SciAgentArena, ~200 tasks.

Running your own models

Reasons to run it yourself

  • Identifiable data stays in-house (or use synthetic data)
  • Same model in five years
  • Big repetitive jobs, no bill per request
  • Repeatable: archivable weights

Downsides

  • Weaker on reasoning and code
  • Someone looks after the hardware

Epoch AI (2026) Open models lag state-of-the-art closed models by 4 months

5. Agentic AI

Demo — Claude Code with a skill

What is an agent?

prompt the model

  tools   memory   files

execution loop   and round again, until it stops

verification a result you can check

The model predicts tokens. The harness is everything else, and you can swap either.

Claude Code Codex Gemini CLI
one company's models
GitHub Copilot Cursor OpenCode
pick the model

Skills to enhance agents

A skill is a folder of Markdown telling the agent how you do something. Human readable, diffable, an open format.

---
name: cbf-dataviz
description: CBF (Computational Biology Facility) house style and helper functions for scientific data visualisation in R/ggplot2. Use whenever producing or reviewing a figure for a CBF project - volcano plots, PCA, heatmaps, box/violin/beeswarm plots, Venn/UpSet, pathway dot plots, circos, MSA, protein domain diagrams - or when choosing a colour palette, picking a ggplot theme, preparing a publication-ready/multi-panel figure, or exporting figures for Inkscape/PowerPoint editing. Trigger on "make a volcano plot", "plot the PCA", "heatmap of these genes", "colour palette for this project", "make this figure publication ready", "tidy up this ggplot", even when CBF is not mentioned.
---

# CBF data visualisation

[...]
You already know ggplot2, what a volcano plot is, and that colour-blind-safe palettes matter.
**Don't re-explain any of that.** This skill is only the things you would otherwise get wrong:
the house defaults, the bundled function API, and the tacit workflow knowledge.

## The defaults - apply without being asked
[...]
- **Theme: `cowplot::theme_cowplot()`.** Use `smplot2::sm_hgrid()` if horizontal gridlines are
  genuinely needed. **Never ship ggplot2's default grey theme** - it reads as amateurish.

cbf-dataviz/SKILL.md, trimmed at […]. Click through for when to load it then the rules it must follow

An SOP that happens to be machine-actionable.

Demo — what did the agent do?

A skill changes the output

Without the skill

With the CBF visualisation skill

The model writes the code. The skill supplies the local rules.

Let the agent run — efficiently

git commit let it run git diff keep, or git checkout git commit

One task per session

A fresh session carries no stale context

What it reads can give it orders

A file or web page can hide instructions

A package name it made up may have been registered by someone else

6. How it goes wrong

What can go wrong

Task Watch out for
Suggest search terms for a literature search You only see the papers it found, so gaps are easy to miss
Find references for a claim Made-up citations: over 140,000 in 2025 papers and preprints. With search, it can still put a real paper on a claim it made up
Explain this code to me You can’t check the explanation, and it sounds as sure when it’s wrong
Write this script The script runs and the numbers are wrong, say from a flipped comparison or samples out of order
Interpret this result A wrong answer can read like a right one

Before you rely on it, ask how you’d know if it were wrong.

Made-up citations: Nature News, 8 and 14 May 2026

Spot the problem

An AI’s fix for a heatmap that wouldn’t draw.

adds random noise to the data.

Other fixes that “work”

  • Edits the test until it passes
  • Hard-codes the expected output
  • Loosens the threshold
  • Silently drops IDs that didn’t map

Look at the code, not just the output!

Example from Mohab Helmy, Prompting for biologists (University of Cambridge), via Ana Mariya Anhel

Do I get the right answer each time?

BioMysteryBench: 99 expert-set problems on real data, with answers that can be checked. Each run five times per model.

Most right answers came from 1 run in 5

Anthropic (2026) Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench, Fig. 3

7. Take homes

Four tips to take away

  1. Split it up. Break the analysis into cheap-to-check steps.
  2. Ground it. Give it your code, data, tools and known answers.
  3. Verify it. Test on your own examples. The risk depends on how easily you’d spot an error.
    • Have a known answer? Keep a small gold-standard test and run it every time.
    • Test the pieces and check the intermediate results.
  4. Record it. Keep your prompts with your code, and declare it when you publish.
    • Model, version and date, what it did, how you checked it. Check the journal/funder rules each time.

Where is this going?

The length of task agents can finish is doubling about every 4 months

Multistep genomics problems: the best model solves 31.5%

Fast progress, but important to remain cynical about outputs!

Kwa, West et al. (2025) Measuring AI Ability to Complete Long Tasks. METR. arXiv:2503.14499 · Li & Ho (2026) GeneBench-Pro. bioRxiv 2026.06.29.735386, Fig. 5A. 129 multistage problems.

Where to go next

  • Ana’s course: prompting, writing, declaring AI use, and what not to hand over
  • The Anthropic, OpenAI and Google prompting guides
  • Try out a local model: Ollama to run it, ellmer to drive it from R
  • Consider trying an agentic LLM: Claude Code, Codex, Gemini CLI or OpenCode
  • Skills repos e.g GPTomics/bioSkills
  • Come and talk to us

Reading: Meng et al. (2026) Large language models in bioinformatics: a comprehensive survey, Front. Genet. 17 · Wang et al. (2025) Large Language Models in Bioinformatics: A Survey, arXiv:2503.04490 · Lu et al. (2024) The AI Scientist, arXiv:2408.06292