The course is oversubscribed and has therefore been closed.

Overview

Foundation models trained on biological sequences, often called biological language models, are changing the way we analyse proteins, DNA and RNA. Built on the same architectures as Large Language Models (LLMs), models such as ESM-2 or ProtT5 learn rich representations of sequences that can be reused for a wide range of tasks, from functional annotation to variant effect and structure prediction.

This one-day course offers a practical introduction to this rapidly evolving field. After a short overview of the main model architectures and of the models available for different molecular data types, participants will work hands-on in Python notebooks to load pre-trained sequence language models with Hugging Face Transformers, extract and explore sequence embeddings, and fine-tune a model on a downstream prediction task. Short lectures alternate with practical exercises run on GPU-enabled notebooks.

By the end of the course, participants will have an overview of the current landscape of biological language models and will be able to judge when and how to apply them to their own research questions.

Audience

This course is designed for PhD students, postdoctoral and other researchers in the life sciences, from both academia and industry, who already have practical experience with deep learning and would like to use pre-trained biological language models in their own analyses.

Learning outcomes

At the end of the course, the participants should be able to:

• Compare encoder (e.g. BERT), decoder (e.g. GPT) and long-context (e.g. Hyena) architectures, and explain which molecular data types (proteins, DNA, RNA) and tasks each is suited to.
• Apply pre-trained biological sequence language models to generate embeddings from sequences using the Hugging Face Transformers library.
• Analyse embeddings using visualisation, clustering and classification to explore relationships between sequences.
• Adapt a pre-trained sequence language model to a downstream prediction task through fine-tuning.
• Evaluate whether a biological language model is appropriate for a given research question, considering data type, model size, computational cost and known limitations.

Prerequisites

Knowledge / competencies

• A good fluency with the Python programming language, including working knowledge of common data analysis libraries such as numpy, pandas and matplotlib.
• Prior knowledge of core deep learning concepts (neural networks, loss functions, training loops, backpropagation) and practical experience of training models with PyTorch are required.
• Familiarity with the Transformer architecture and the attention mechanism is recommended.
• Familiarity with protein sequence data and resources (e.g. FASTA format, UniProt) is recommended.

To get the most out of this course, you should meet the learning outcomes of Diving into Deep Learning - Theory and Applications with PyTorch.
If you are new to Python or machine learning, we recommend first following First Steps with Python in Life Sciences, Data Analysis and Representation in Python and Introduction to Machine Learning with Python.

Technical

You will need access to a computer with a stable internet connection.

The practicals run on GPU-enabled Python notebooks in the cloud. You will receive access instructions before the course; no local installation is required.

Application

The course is oversubscribed and has therefore been closed.

The registration fees for academics are 100 CHF and 500 CHF for for-profit companies.

While participants are registered on a first come, first served basis, exceptions may be made to ensure diversity and equity, which may increase the time before your registration is confirmed.

Applications will close on 27/11/2026 or as soon as the places will be filled up. Cancellation after 27/11/2026 will not be reimbursed. Please note that participation in SIB courses is subject to our general conditions.

You will be informed by email of your registration confirmation. Upon reception of the confirmation email, participants will be asked to confirm attendance by paying the fees within 5 working days.

Venue and Time

This course will be streamed.

The course will start at 9:00 CET and end around 17:00 CET.

Precise information will be provided to the registered participants in due time.

Additional information

Coordination: Diana Marek, SIB Training group.

A Certificate of Attendance will be sent provided you were present at the course, whereas a Certificate of Achievement recommending 0.25 ECTS will be sent provided you passed the exam.

You are welcome to register to the SIB courses mailing list to be informed of all future courses and workshops, as well as all important deadlines using the form here.

SIB abides by the ELIXIR Code of Conduct. Participants of SIB courses are also required to abide by the same code.

For more information, please contact [email protected].