Talks, materials and challenge | Ph. D. school on uncertainty in machine learning

Summer school challenge

Uncertainty Challenge 2026 – challenge presentation_1 Download

The exercises is in the form of a team challenge. You will be divided into teams at the start of the summer school and each team will get their team name and password. The challenge web sites:

General introduction (Lisa Koch)

Abstract:

Calibration (general introduction)
Label-free performance estimation (confidence-based – close ties to calibration)
Calibration and confidence-based performance estimation under distribution shifts

Uncertainty-Aware Visual Odometry: Robust Perception in Noisy Underwater Environments (Olaya Alvarez Tunon)

Abstract:

Visual odometry systems operating in challenging environments face severe noise due to dynamic lightning, turbidity, and degraded geometry. Deep neural networks provide strong high-level representations that enable VO to operate in such conditions, however, reliable deployment requires estimating the uncertainty together with the predictions.

This talk focuses on showing how to explicitly model uncertainty from the data (aleatoric uncertainty) and from the model (epistemic uncertainty), and how learning covariance through dedicated output head can strengthen the performance and reliability of deep learning based VO pipelines.

We will review key advances in multivariate uncertainty prediction for deep networks [1] and general uncertainty quantification via conformal prediction [2]. Building on this, we will examine VO systems that learn metrics-aware covariance to guide optimization and feature weighting [3].

Finally, these ideas will be translated to a practical robotics setup: from front-end correspondence filtering based on predicted uncertainty, to pose-graph optimization weighted by learned covariance, to sensor fusion informed by uncertainty propagation [4]. The goal is to give participants both conceptual intuition and practical strategies for designing VO/SLAM systems that explicitly reason about uncertainty and thus remain robust in noisy, real-world conditions such as underwater environments.

Related papers:

[1] Russell, R. L., & Reale, C. (2021). Multivariate uncertainty in deep learning. IEEE Transactions on Neural Networks and Learning Systems, 33(12), 7937-7943.

[2] Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and trends® in machine learning, 16(4), 494-591.

[3] Qiu, Y., Chen, Y., Zhang, Z., Wang, W. and Scherer, S., 2025, May. MAC-VO: Metrics-Aware Covariance for Learning-Based Stereo Visual Odometry mac-vo. github. io. In 2025 IEEE International Conference on Robotics and Automation (ICRA) (pp. 3803-3814). IEEE.

[4] Thrun, S. (2002). Probabilistic robotics. Communications of the ACM, 45(3), 52-57.

Split Conformal Prediction for Regression (Christian Igel)

Abstract: This tutorial introduces Conformal Prediction (CP) for quantifying the uncertainty of machine learning (ML) models. Conformal Prediction provides prediction sets guaranteed to contain (in expectation) the true outcome with a user-specified probability without strong assumptions about the underlying data distribution. We focus on regression tasks and on Split CP, which leverages a hold-out calibration data set and can be applied to any ML model. We establish the foundational theory of Split CP, proving its marginal coverage guarantee under the assumption of i.i.d. data. Then we present Split Localized Conformal Prediction, an efficient method that approximates conditional coverage by adapting to the local data structure while preserving the rigorous marginal guarantee. The goal of this tutorial is to provide attendees with the basic foundations needed to apply CP in their research and to explore more advanced topics in distribution-free uncertainty quantification.

Lecture notes
To prepare for the lectures, please recall the mighty Hoeffding’s inequality. In addition, please have a look at the probability integral transform. Both are stated in the lecture notes in Appendix B.

Aleatoric and Epistemic Uncertainty in Medical Imaging: From the Decomposition to Clinical Decisions (Christian Baumgartner)

Abstract: A frequently used framework in uncertainty quantification decomposes predictive uncertainty into an aleatoric component (irreducible data ambiguity) and an epistemic one (reducible lack of model knowledge). The first lecture will build this decomposition from parameter and predictive distributions. Using recent classification and segmentation benchmarks, the lecture will show empirically that the theoretically consistent measure is often not the best performing one and that the two components are strongly entangled. Then, the lecture will cover the theory explaining why this may be the case. Lastly, the lecture will discuss how the dichotomy breaks down for interactive large language model agents. The second lecture will cover practical application examples in medical imaging, showing how uncertainty can still be useful despite the shortcomings outlined in the first lecture. We will talk about propagating uncertainty through clinical pipelines to downstream diagnosis, and turning it into calibrated, actionable decisions.

Model selection and uncertainty quantification in Bayesian deep learning (Søren Hauberg)

Abstract: We will discuss the basics of Bayesian deep learning to build a pipeline for model selection and uncertainty quantification. We will gradually transition to discuss why standard approximations fails for overparametrized models and what to do about it.
https://www2.compute.dtu.dk/~sohau/talks/2026_DTU_Uncertainty/part1.html#frame7458
https://www2.compute.dtu.dk/~sohau/talks/2026_DTU_Uncertainty/part2.html#frame2951

Imperfect and uncertain labels in computer vision (Galadrielle Humblot-Renaux)

Abstract: We will focus on uncertainty arising from the data itself and its labels, including:

the types and sources of aleatoric uncertainty
how it manifests in practice across different computer vision tasks and domains
how it can be measured and modelled
how imperfect training labels can affect model performance and post-hoc tasks such as OOD detection
the importance of also taking into account imperfections or uncertainty in the labels used for evaluation

Summer school 2026 – Imperfect and uncertain labels Download

REPEAT for uncertainty estimation in XAI representation learning & SEAL to “protect” learned representations (Robert Jenssen)

Abstract: This talk highlights two different aspects of uncertainty in representation learning. The first part deals with explainable AI (XAI) for representation learning in computer vision and asks: Can we determine if a pixel is certainly important for a learned representation? The second part deals with uncertainty regarding potential misuse of learned representations and proposes a way to “seal” representations with newly proposed matrix-based mutual information approaches.

These are the two most relevant publications:

Wickstrøm, Brüsch, Kampffmeyer, Jenssen: REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability, AAAI 2025. https://ojs.aaai.org/index.php/AAAI/article/view/32900

Wu, Jenssen et al. Mitigating Embedding Leakage via Latent Disruption with Controlled Reconstruction, TMLR 2026. https://openreview.net/forum?id=nZWBrxJyrS