Our Research

With the WestAI Service Center, a consortium of excellent scientific institutions in North Rhine-Westphalia has been formed under the leadership of the University of Bonn. The center brings together complementary expertise from various fields of AI application in order to develop new powerful and efficient multimodal AI models for Germany.

Our research focuses on transfer learning and multimodal AI.

At WestAI, we conduct research in the field of artificial intelligence and machine learning for the application of deep learning methods. Deep learning uses deep neural networks and large amounts of data to solve problems. More specifically, we are looking at the sub-area of transfer learning. This method makes it possible to use existing knowledge from an existing AI solution to solve a new problem more efficiently. Artificial neural networks are used, which saves time and resources.

Our research focusses on the transfer learning of multimodal AI models – these are AI models that can process sensor, audio or video data in addition to text, for example. The aim is to advance the state of the art in multimodal AI by investigating the scalability, efficiency, transferability and continuous improvement of learning algorithms.

Our research framework covers the following areas:

Scalable data management and integration

Research into techniques for automatic data pre-processing and generation, especially for heterogeneous and multimodal data types. The consideration of data protection and data sovereignty in real applications plays a central role here.

Large-scale model training

Developing efficient learning methods for training across multiple compute nodes and exploring different model architectures and model training methods for the next generation of accelerated computing.

Efficient transfer learning

Systematic investigation of the conditions of transfer learning from pre-trained models to different domains or application areas, taking into account factors such as model and data scope, architecture and transfer efficiency.

Model compression for low-resource environments

Investigation of methods for compressing pre-trained and ‘transferred’ models (after transfer learning) while maintaining the performance of a model. Research into the use of models on special hardware for energy-efficient execution is also being realised.

Continuous learning

Investigating the ongoing development of large AI models through continuous learning from transferred and compressed models. The scalability and possible applications in federated learning environments are being researched.

Our publications

Reproducible scaling laws for contrastive language-image learning

Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

read more

LAION-5B: An open large-scale dataset for training next generation image-text models

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, Jenia Jitsev. LAION-5B: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 25278-25294.

read more

DataComp: In search of the next generation of multimodal datasets

Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussmann, Richard Vencu, Mehdi Cherti, Ranjay Krishna, Pang Wei Koh, Olga Saukh, Alexander Ratner, Shuran Song, Hannaneh Hajishirzi, Ali Farhadi, Romain Beaumont, Sewoong Oh, Alex Dimakis, Jenia Jitsev, Yair Carmon, Vaishaal Shankar, Ludwig Schmidt; DataComp: In search of the next generation of multimodal datasets. NeurIPS 2023.

read more

Post-Processing Independent Evaluation of Sound Event Detection Systems

Janek Ebbers, Reinhold Haeb-Umbach, Paderborn University, Romain Serizel, and Universit´e de Lorraine, CNRS, Inria, Loria, “Post-Processing independent evaluation of sound event detection systems,” Journal-article, 2023. [Online]. https://dcase.community/documents/workshop2023/proceedings/DCASE2023Workshop_Ebbers_62.pdf

read more

Get in touch with us!

Do you have any questions or are you interested in working with us?

Feel free to email us, and we’ll be happy to help and advise you.

Follow us on LinkedIn

Don’t want to miss WestAI updates, news, and events or want to share them faster with your network?

Follow us on LinkedIn!