Modern AI technologies have great potential of substantially transforming healthcare systems, increasing diagnosis efficiency and reducing treatment cost. Core clinical questions — diagnosis, treatment personalization and understanding disease progression — are all in their nature prediction problems. Therefore, powerful machine learning methods that can leverage large quantities of patient data have potential to be more accurate than human physicians and reduce outcome uncertainty that plagues most of today’s medical interventions. Recent advances have shown that on a number of settings, AI models trained on large-scale annotated datasets of medical images and records can achieve higher diagnosis accuracies than novice doctors.

However, there remain significant challenges before we can extend such technologies to general practice, e.g. medical images and records from different sources vary substantially and most of them do not come with sufficient annotations.

This programme aims to overcome such difficulties by developing new healthcare AI techniques that can take full advantage of both large-scale and small-scale medical images and records (in natural language) from various sources. The proposed work focuses on algorithms for processing text and images, two modalities that cover most of the available patient data. Natural Language Processing algorithms will be used to process at scale patient electronic medical records (EHRs) to extract pertinent patient features and outcomes. Using these automatically extracted meta-data, we will utilize computer vision algorithms to analyse medical images for predicting early diagnosis, personalizing treatment and assessing care efficiency. In particular, this plan comprises the following aspects: ​

• Develop new medical imaging model with better transferability and interpretability, for analyzing sequential medical data for disease screening and diagnosis assistance;

• Develop new methods for training medical AI models from rationale-augmented annotations and interaction with human;

• Develop new methods that allow effective learning from medical images and records obtained from diverse sources, e.g. different hospitals, doctors, and medical instruments.