Singapore’s National University Health System (NUHS) has placed the NVIDIA DGX A100 system at the heart of its newly-launched Endeavour AI production platform.
The first projects to run on the platform are range from predictions on how a patient with a certain condition will fare when admitted to a hospital to analysing magnetic resonance imaging (MRI) images.
With the new platform, NUHS has real-time streaming capabilities to deliver better patient care and treatment, collaborate on biomedical research and transform how illnesses are managed and treated.
Endeavour AI will be integrated with the cluster’s Discovery AI training platform to form a complete training and inference system as part of the group’s digital transformation.
“Advances in healthcare require great compute resources, and NVIDIA DGX A100 delivers easy access to performance needed to aid in a world class hospital,” said Dr Ngiam Kee Yuan, Group Chief Technology Officer of NUHS and Deputy Chief Medical Informatics Officer of National University Hospital (NUH).
NUHS is one of three public healthcare clusters in Singapore that covers 19 hospitals, polyclinics, specialist centres, medical centre, and academic health science institutions.
GPUs are necessary
Discovery AI runs on NVIDIA GPUs to train models using large data sets and CPUs for inferencing.
“As the number, volume and speed at which we are running inferencing increase, GPUs become necessary. Otherwise, we need to expend a lot more CPUs to run the same inferencing at that speed,” said Ngiam.
He cited the example of an AI tool that would run about 100 to 200 inferences per second. For every patient who turns up at NUHS’ hospitals and polytclinics, every time a doctor clicks, saves or free texts, or when new lab test results are out, an AI tool will be running in the background.
All the data are processed by the AI tools. This is done hundreds of times per second throughout the whole cluster at a large volume. If only CPUs are used, NUHS will run out of processing speed very quickly.
“That’s why we strategised, planned and built in NVIDIA DGX A100 from Day One when we deployed Endeavour AI. This is because we will be using it for high speed and large volume inference processed by our AI tools,” said Ngiam.
Real-time processing
Endeavour AI is a software and hardware stack that features streaming data and AI tools running micro services off a Kubernetes backbone for real-time processing. It will start off with dozens of projects initially before scaling up to up to 150 projects.
NUHS produces between 20GB and 30GB of structured data and text daily, or between 1,800 and 2,500 messages per second for one hospital. This translates to about 10,000 to 15,000 messages per second at peak for the entire cluster. The AI tools need to run quickly in the background to absorb all the data on a day-to-day basis.
“We do not want to build one project at a time. We are building a platform that enables multiple projects to run at a time. We have multiple uses for GPUs, largely in training at this point, but certainly we are well underway in operationalising the production use cases. Without the GPU, we cannot do a lot of these things,” said Ngiam.
Improving patient experience
Patients interacting with AI-powered chatbots will experience improvement in appointment making, reduction in waiting time, and enjoy optimised care due to some of the work on patient trajectories.
Radiologists and clinicians benefit from improved accuracy and speed of processing of images, X rays and scans, thanks to the stream capability.
In day-to-day hospital operations, a number of predictions is done automatically without even needing to click a button. Data that streams out of the electronic medical record system are processed and the doctor is alerted if a patient meets a certain set of weights for high risk.
“These are tangible realities and outcomes that we expected when we Endeavour AI went live,” said Ngiam.
Achieving scientific breakthroughs
Even though the NVIDIA DGX A100 has just gone live, NUHS is already looking forward to the next generation of the system to tackle the expected growth in datasets and speed needed to process those data in the next few years.
“We have invested in programmes that look at genomics. When this genomic data hits us, we are not talking about gigabytes but one terabyte of data per day. The amount of compute required to run genomic type processing going forward is going to be exponentially larger. Until then, our next step is to look at how to optimally use our GPUs for the next few years,” said Ngiam.
“NVIDIA DGX A100 lets NUHS consolidate training, inference and analytics into a unified AI infrastructure. It will provide the computing power to help the hospital group achieve operational and scientific breakthroughs in the healthcare sector, benefitting clinicians and patients in Singapore,” said Dennis Ang, Director of Enterprise Business for the SEA and ANZ Region at NVIDIA.
Entelechy Asia 