Manifolds of Multimodal Multiscale Science
Project Summary
Scientific measurements are often made at inherently different scales. In biology for instance, emergent features of large-scale systems allow us to abstract away many microscale features despite direct causal relationships between various scales. This project is aimed at improving multimodal and multiscale AI methods which is core to unlocking scientific value from large-scale AI for science systems.
This project will initially focus on developing hyperbolic manifolds to merge existing foundation models. The student is expected to explore scalable training approaches both theoretically and practically to enable multiscale priors in large AI systems in the sciences. The project is expected to enable novel scientific analyses through representation learning and generative modelling.
The project will span multiple domains as an inherently interdisciplinary challenge. The student will be expected to be able to model and evaluate constraints being placed on the model and what affects this may have on the training dynamics of large-scale models. The project is not limited to a single scientific domain and may extend to multiple; this may include but is not limited to medical imaging, genomics, proteins and robotics.
Potential Supervisors
- Dr Micah Bowles (Research Engineer, EIT & Visiting Academic, Department of Physics, University of Oxford)
- Additional Supervisor(s) from the University of Oxford
Skills Recommended
- Strong background in statistics, linear algebra, and computer science
- Experience with deep learning
- Proficiency in Python and a framework like PyTorch/JAX
- Knowledge of multiple scientific fields
Skills to be Developed
- Designing novel neural network architectures
- Multiscale modelling
- Advanced ML for science (ML4Sci)
- Developing next-generation ML multimodal models
University DPhil Courses
Relevant Background Reading
- Wang, Z., Ramasinghe, S., Xu, C., Monteil, J., Bazzani, L., Ajanthan, T. 2024. Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval. arXiv e-prints. doi:10.48550/arXiv.2411.17490
- Maciej Wiatrak, Ramon Viñas Torné, Maria Ntemourtsidou, Adam Dinan, David C. Abelson, Divya Arora, Maria Brbić, Aaron Weimann, R. Andres Floto, 2025. bioRxiv 2025.07.20.665723; doi: https://doi.org/10.1101/2025.07.20.665723
- Pal, A., van Spengler, M., D'Amely di Melendugno, G.~M., Flaborea, A., Galasso, F., Mettes, P. 2024. Compositional Entailment Learning for Hyperbolic Vision-Language Models. arXiv e-prints. doi:10.48550/arXiv.2410.06912
- Mandica, P., Franco, L., Kallidromitis, K., Petryk, S., Galasso, F. 2024. Hyperbolic Learning with Multimodal Large Language Models. arXiv e-prints. doi:10.48550/arXiv.2408.05097
- Mizrahi, D. and 6 colleagues 2023. 4M: Massively Multimodal Masked Modeling. arXiv e-prints. doi:10.48550/arXiv.2312.06647
