Intelligent Medical Imaging for Breast Cancer Detection and Diabetic Retinopathy

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging for Breast Cancer Detection and Diabetic Retinopathy Hanung Adi Nugroho Inovasi E-Health and Biomedika untuk Indonesia

Research and Development of Intelligent Medical Imaging Profile

Dr. Ir. Hanung Adi Nugroho

Department of Electrical Engineering and Information Technology Faculty of Engineering, UNIVERSITAS GADJAH MADA Jl. Grafika 2, Kampus UGM, Yogyakarta 55281, Indonesia Telp./ fax. +62-274-552305 Email: adinugroho@ugm.ac.id; adinugroho@ieee.org

Research areas: Biomedical signal and image processing and analysis; computer vision; medical instrumentation; pattern recognition; data mining; statistical data analysis.

Bachelor of Engineering (S.T.) – Teknik Elektro, Universitas Gadjah Mada, Yogyakarta, Indonesia (2001) Master of Engineering (M.E.) – School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Brisbane, Australia (2005) Doctor of Philosophy (Ph.D.) – Electrical and Electronics Engineering Department, Universiti Teknologi PETRONAS, Seri Iskandar, Malaysia (2012)

Intelligent Medical Imaging Research

Medical imaging - Overview

Currently medical imaging is limited to the acquisition of images of the human organs/ body Medical imaging refers to the techniques and processes used to create images of the human body for clinical purposes (medical procedures seeking to reveal, diagnose or examine disease). Medical imaging can be seen as the solution of mathematical inverse problems. This means that cause (the properties of living tissue) is inferred from

Analysis of the images obtained

effect (the observed signal)

is performed clinically by experts adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research

Medical imaging - Technology

a short-lived isotope, such Gamma ray : positron emission tomography (PET) as 18F, is incorporated into a substance used by the body such as glucose which is absorbed by the tumour of interest

X ray : computed tomography (CT) Expose to x-ray radiation, repeated scans must be limited to avoid health effects

Intelligent Medical Imaging Research

Magnetic resonance imaging (MRI) uses powerful magnets to polarise and excite hydrogen nuclei (single proton) in water molecules in human tissue, producing a detectable signal which is spatially encoded resulting in images of the body excellent soft-tissue contrast no known long term effects of exposure to strong static fields health risks associated with tissue heating from exposure to the RF field and the presence of implanted devices in the body, such as pace makers

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research Medical imaging - Technology

Ultrasound : ultrasonography H-F sound, 2-10MHz, safe, 2D moving images

Fundus camera Retinal image

Intelligent Medical Imaging Research Issues, challenge and approach

Issues Issues Approach Approach

Harmful (radiation, contrast agent) • Harmful (radiation, contrast agent)

From From medical medical imaging imaging (image (image

Specialized device – difficult to use - • Specialized device – difficult to use - acquisition with enhancement) to acquisition with enhancement) to highly trained operator needed highly trained operator needed medical image analysis (feature medical image analysis (feature
Expensive (Initial cost, Maintenance) • Expensive (Initial cost, Maintenance) extraction, extraction, classification, classification, pattern pattern
Image Acquisition only, little or no • Image Acquisition only, little or no recognition, measurements) resulting recognition, measurements) resulting analysis for diagnostic purposes, analysis for diagnostic purposes, in in intelligent intelligent imaging imaging (decision (decision subjective subjective support systems) support systems)

Challenge Challenge

To develop intelligent medical imaging system which is objective in analysis that is To develop intelligent medical imaging system which is objective in analysis that is safe to the patients. safe to the patients.

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research Current research in intelligent medical imaging system at DTETI-UGM

Radiology (Breast cancer) Ophthalmology (Diabetic retinopathy)

Diabetic retinopathy Breast cancer Glaucoma

Intelligent Medical Imaging Research in Breast Cancer Intelligent Medical Imaging Research in Breast Cancer adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Breast Cancer Breast cancer is a disease in which malignant (cancer) cells form in the tissues of the breast.

Breast compression
Low-dose X-ray
Just for particular patient

CAD CAD : Computer Aided Diagnosis Intelligent Medical Imaging Research in Breast Cancer

Breast Self Exam Mammograms USG MRI

Limited availability

Elaborate the radiology knowledge into image processing and analysis technology
Assist radiologists to diagnose nodule

Low cost
Short acquisition time
No radiations
High availability
Convenient  more sensitive
Depend on operator
No Radiation  More Detail
Radiologist s experience
Inconsistency of interpretation
Expensive  Limited availability

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Breast Cancer Detection

Research Objective To develop a computer aided diagnosis (CAD) system for classifying breast nodule in ultrasound (US) images to distinguish benign and malignant nodules.

Intelligent Medical Imaging Research in Breast Cancer Diagnosis of Breast Cancer using Ultrasound

A breast ultrasound is a scan that uses penetrating sound waves that do not affect or damage the tissue and cannot be heard by humans.

Normal Abnormal adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer

Methodology

Intelligent Medical Imaging Research in Breast Cancer Scheme of CAD System Computer Aided System USG Image Processing Image Image Image Analysis Display Acquisitions

Diagnosis Radiologists adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer System Diagram Block

Preprocessing Noise and GrayScale (4) USG (1) Marker Segmentation RoI (2) Conversion (3) Images Reduction

(5) Feature Extraction

Moment based

Malignant / Birads based Feature

(8) features

Diagnosis (6) (7) Benign Classification Selection

Geometry Feature • Texture Feature

Texture Features

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer

Nodule Background

Segmented Area Posterior Characteristic Margin Characteristic

Echo Pattern Characteristic

Texture Analysis

Intelligent Medical Imaging Research in Breast Cancer Geometry Features

Geometric feature is constructed by a set of geometrical elements such as points, lines, curves or surfaces = =

1 ∑

. exp ( 2 ) = =

= = 4 .

Intelligent Medical Imaging Research in Breast Cancer Geometry and Moment Based Features

Nodule Moment Based Analysis

Shape characteristics Geometry Analysis

Margin characteristics Background

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Research Roadmap Clinical Integrated Trial Modules

Prototype 2 2018 Margin and Posterior Features

2016 Prototype 1 2015 Echo

Pattern Shape and 2014 Boundary

Intelligent Medical Imaging Research in Breast Cancer Results adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Diagnosis Rules for i-Brids V.1

Benign

Circumscr>Round -
Not Circumscribed Malignant

Unmarked Hypoechoic or Hypoechoic

Circumscribed Malig>Irreg
Not Circumscribed Malignant

Intelligent Medical Imaging Research in Breast Cancer

Segmentation and Image ROI and Filtering Diagnosis

Feature Extraction Capturing adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Performance Analysis of CAD System

100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00%

0.00% Accuracy Sensitivity Specificity PPV NPV

Shape 96.20% 94.70% 97.90% 94.73% 97.91% Margin

80.90% 79.50% 82.50% 78.50% 82.50% Echo

91.23% 95.83% 87.88% 85.19% 96.67%

Intelligent Medical Imaging Research in Breast Cancer Statistical Analysis Diagnosis Statistical Analysis

Malignant Benign

1 Number of Features Agreement

12.5

6.5

2 Number of Features due to Chance

3 Total Number of Subjects

4 Total Number of Agreement

5 Number of Agreement due to chance

6 Kappa

0.68 Kappa statistics are commonly used to indicate the degree of agreement of nominal assessments made by multiple appraisers.

A Kappa 0.68 is in the “substantial” agreement range between radiologists and CAD system.

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Diagnosis Rules for i-Brids V.2

Benign/Malignant

No Posterior Fea>Circumscribed • Enhancement Be
Shadowing Malignant

Unmarked Hypoechoic or Hypoechoic

Malignant

No Posterior Fea>Not Circumscribed • Enhancement Malig
Shadowing Malignant

Intelligent Medical Imaging Research in Breast Cancer

Image Segmentation and ROI and Filtering Diagnosis Feature Extraction

Capturing adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Accuracy of CAD System

98.00% 96.00% 94.00% 92.00% 90.00% 88.00% 86.00% 84.00% 82.00% 80.00% 78.00% 76.00%

Margin Posterior Diagnosis Radiologist 1

89.47% 84.21% 97% Radiologist 2

89.47% 86.84% 97%

Intelligent Medical Imaging Research in Breast Cancer Performance Analysis of CAD System

100% 99% 98% 97% 96% 95% 94% 93% 92%

Sensitivity Specificity PPV NPV Radiologist 1 100% 94.74% 95% 100% Radiologist 2 100% 94.74% 95% 100%

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Statistical Analysis Margin Posterior Diagnosis

No Statistical Analysis

No Circumscribed Indistinct Enhancement Posterior Shadow Malignant Benign

1 Number of Features Agreement

2 Number of Features due to Chance

12.74

6.74

11.6

0.79

3.47

9.5

3 Total Number of Subjects

4 Total Number of Agreement

5 Number of Agreement due to chance

19.47

15.87

6 Cohen's Kappa

1 0.774

1 A Kappa 1 is in the “perfect” agreement range between two radiologist

A Kappa 0.74 is in the “substantial” agreement range between two radiologist

Intelligent Medical Imaging Research in Breast Cancer Video of i-Brids

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Potential Market Total: 6430

Number of Health Care Fasilities in Indonesia

3451 1599

1380 Health Hospitals/Clinics Puskesmas

Laboratories

Breast Cancer :

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Breast Cancer Recognition [1] H. A. Nugroho, N. Faisal, I. Soesanti, and L. Choridah, “Analysis of Computer Aided Diagnosis on Digital Mammogram Images,” in 2014 International Conference on Computer, Control, Informatics and Its Applications Analysis, 2014, pp. 25–29.

[2] A. Nugroho, H. A. Nugroho, and L. Choridah, “Active Contour Bilateral Filter for Breast Lesions Segmentation on Ultrasound Images,” in 2015 International Conference on Science in Information Technology (ICSITech) Active, 2015, pp. 36–40. [3] H. A. Nugroho, Y. Triyani, M. Rahmawaty, , I. Ardiyanto ,and L. Choridah, “Performance Analysis of Filtering Techniques for Speckle Reduction on Breast Ultrasound Images,” in 2016 International Electronics Symposium (IES), 2016, pp. 454–458. [4] M. Rahmawaty, H. A. Nugroho, Y. Triyani, I. Ardiyanto, and I. Soesanti, “Classification of Breast Ultrasound Images based on Texture Analysis,” in iBioMed 2016, 2016, pp. 84–89. [5] Y. Triyani, H. A. Nugroho, M. Rahmawaty, I. Ardiyanto, and L. Choridah, “Performance Analysis of Image Segmentation for Breast Ultrasound Images,” in ICITEE 2016, 2016, no. October, pp. 415–420. [6] H. K. N. Yusufiyah, H. A. Nugroho, T. B. Adji, and A. Nugroho, “Feature Extraction for Classifying Lesion ’ s Shape of Breast Ultrasound Images,” 2nd Int. Conf. Inf. Technol. Comput. Electr. Eng., pp. 105–109, 2015. [7] H. A. Nugroho, H. Khuzaimah, N. Yusufiyah, T. B. Adji, and A. Nugroho, “Zernike Moment Feature Extraction for Classifying Lesion ’ s Shape of Breast Ultrasound Images,” in 7th International Conference on Information Technology and Electrical Engineering (ICITEE), 2015, pp. 458–463. [8] H. A. Nugroho, N. Faisal, I. Soesanti, and L. Choridah, “Identification of Malignant Masses on Digital Mammogram Images based on Texture Feature and Correlation based Feature Selection Hanung,” in 6th International Conference on Information Technology and Electrical Engineering (ICITEE),

2014. [9] H.R. Fajrin, H. A. Nugroho, and I. Soesanti“Ekstraksi Ciri Berbasis Wavelet Dan Glcm Untuk Deteksi Dini Kanker Payudara Pada Citra Mammogram,” in SNST, 2015, pp. 47–52.

[10] M. Sahar, H. A. Nugroho, Tanur, I. Ardiyanto, and L. Choridah “Automated Detection of Breast Cancer Lesions Using Adaptive Thresholding and Morphological Operation,” in International Conference on Information Technology Systems and Innovation (ICITSI), 2016. [11] Tianur, H. A. Nugroho, M. Sahar, R. Indrastuti, and L. Choridah, “Classification of Breast Ultrasound Images based on Posterior Feature,” in International Conference on Information Technology Systems and Innovation (ICITSI), 2016.

Intelligent Medical Imaging Research in Breast Cancer Team Members and Collaborators Department Electrical Engineering and Information Technology Faculty of Engineering Universitas Gadjah Mada Department of Radiology Sardjito Hospital, Yogyakarta

H A Nugroho • I Ardiyanto • M Rahmawaty • Y Triyani • M Sahar • L Choridah • R. Indrastuti • A. Mardhiah • Tianur • A Nugroho • D A Husna • H Khuzaimah • R L Buana

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Diabetic Retinopathy Intelligent Medical Imaging Research in Diabetic Retinopathy

What is diabetic retinopathy TYPE 1 DIABETES: when the pancreas

doesn’t produce insulin

when the pancreas doesn’t produce enough insulin (or the insulin cannot be processed)

GESTATIONAL DIABETES: when the

insulin is less effective during pregnancy

your body needs insulin to transform glucose into energy

Types of diabetes:

TYPE 1 DIABETES:

Intelligent Medical Imaging Research in Diabetic Retinopathy What is diabetic retinopathy

DR is retinopathy (damage to the retina) caused by

complications of diabetes mellitus, which could eventually lead to blindness.

Fact : Diabetic Retinopathy Diabetic

Nearly all patients of type-1 diabetes and 60% of

(DR) Cardiomyopathy patients of type-2 diabetes indicate retinopathy.

DR is the leading cause of the blindness in developing countries among adults aged 20-74 years.

Diabetic Neuropathy Diabetic Nephropathy

Normal vision DR vision adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Diabetic Retinopathy

Diabetes : fact and figures “Worldwide”

2015: 415 million people with diabetes 2040: 642 million people with diabetes

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Diabetic Retinopathy The pathologies of DR new blood vessels

Haemorrhages

Proliferative DR Severe NPDR Intelligent Medical Imaging Research in Diabetic Retinopathy

exudates Micro aneurysms Moderate NPDR Mild NPDR No DR

Issues, challenges and approaches

Issues Diabetes mellitus affect ~10% population (DR is a real concern - epidemic stage?) Needs access to ophthalmologist with fundus camera equipment Low contrast Fundus images requiring Fluorescein angiography - an invasive procedure

Challenges

1. Can we develop a screening & grading system to be made accessible to all diabetes patients?

2. Can we detect DR early even before patient have visual problems?

3. Can we make non-invasive procedure as effective?

Fundus camera technology

Image Processing & Computer Vision

Intelligent Medical Imaging Research in Diabetic Retinopathy Haemorrhages detection

Fundus image Enhancement Enhancement Haemorrhages candidates Detected Haemorrhages Haemorrhages candidates Detected Haemorrhages

Green and V band Histogram Retinal vessels Retinal vessels extraction matching detection elimination

Two-dimensional matched filtering Contrast Opening

Double length Masking enhancement operation filtering operation

Haemorrhages Pre-processing Post-processing candidate detection adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Diabetic Retinopathy

Hard exudates detection

Removal OD and

Detected

Fundus image Filtered image Detected OD Candidate Candidate Hard Exudates Exudates

Hard Exudates Hard Exudates Green channel Complement

Removal OD and

extraction operation

Optic disc (OD) Morphological

[1]

detection operation _{[1] H. A. Nugroho, K. W. Oktoeberza, T. B. Adji, and M. B. Sasongko, "Segmentation of exudates based on high pass filtering in retinal fundus images," in 2015 7th International Conference on Information Technology and Electrical} _{Engineering (ICITEE), 2015, pp. 436-441.} Matched filter

Intelligent Medical Imaging Research in Diabetic Retinopathy Research roadmap

General : to develop a system to assist the ophthalmologists in monitoring and diagnosing diabetic retinopathy disease. First year: to develop algorithms in each module to detect structures and pathologies in DR retinal image. Second year: to integrate the modules and develop an algorithm for screening DR system. Third year: to test the system based on clinical study for monitoring and grading system.

DR monitoring and 2018

Clinical study grading system

Analysis of DR System evaluation pathologies 2017

DR pathologies Classification

DR screening detection system

FAZ detection 2016

Haemorrhages and hard Micro aneurysms exudates detection detection

Macula detection 2015

Optic disc detection adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Diabetic Retinopathy

Recognition 

International Conferences

[1] H. A. Nugroho, D. A. Dharmawan, I. Hidayah, and L. Listyalina, "Automated microaneurysms (MAs) detection in digital colour fundus images using matched filter," in Computer, Control, Informatics and its Applications (IC3INA), 2015 International Conference on, 2015, pp. 104-108.

[2] H. A. Nugroho, L. Listyalina, N. A. Setiawan, S. Wibirama, and D. A. Dharmawan, "Automated segmentation of optic disc area using mathematical morphology and active contour," in Computer, Control, Informatics and its Applications (IC3INA), 2015 International Conference on, 2015, pp. 18-22.

[3] H. A. Nugroho, D. Purnamasari, I. Soesanti, K. W. Oktoeberza, and D. A. Dharmawan, "Detection of foveal avascular zone in colour retinal fundus images," in 2015 International Conference on Science in Information Technology (ICSITech), 2015, pp. 225-230. [4] H. A. Nugroho, K. W. Oktoeberza, T. B. Adji, and M. B. Sasongko, "Segmentation of exudates based on high pass filtering in retinal fundus images," in 2015 7th International Conference on Information Technology and Electrical Engineering

(ICITEE), 2015, pp. 436-441.

[5] H.A. Nugroho, L. Listyalina, and D. A. Dharmawan, "A New Approach for Detection of Retinal Haemorrhages in Colour Fundus Images," presented at the International Seminar on Sensors, Instrumentation, Measurement and Metrology, 2016. [7] I. Ardiyanto, H.A. Nugroho, and R. L. B. Buana, "Maximum Entropy Principle for Exudates Segmentation in Retinal Fundus Images," presented at the International Seminar on Sensors, Instrumentation, Measurement and Metrology, 2016. [8] H.A. Nugroho, W.KZ. Oktoeberza, I. Ardiyanto, R.L.B. Buana, and M. B. Sasongko, "Automated Segmentation of Hard Exudates Based on Matched Filtering," presented at the International Seminar on Sensors, Instrumentation,

Measurement and Metrology, 2016.

Intelligent Medical Imaging Research in Diabetic Retinopathy Recognition



Journals

[1] H. A. Nugroho, K. W. Oktoeberza, T. B. Adji, and F. Najamuddin, "Detection of Exudates on Color Fundus Images using Texture Based Feature Extraction," International Journal of Technology, vol. 6, p. 04, 2015. [2] H.A. Nugroho, D.A. Dharmawan, and L. Listyalina, "Automated Segmentation of Foveal Avascular Zone (FAZ) in Digital Colour Retinal Fundus Images," International journal of biomedical engineering and technology, 2016.

adinugroho@ugm.ac.id adinugroho@ugm.ac.id Intelligent Medical Imaging Research in Diabetic Retinopathy Team members and Collaborator

Rapid Assessment Diabetic Retinopathy and Intelligent System Research Groups Department of Electrical Engineering and Information Technology, Faculty of Engineering Universitas Gadjah Mada, Indonesia

(Hanung Adi Nugroho, Noor Akhmad Setiawan, Teguh Bharata Adji, Indriana Hidayah,

Igi Ardiyanto, Ratna Lestari Budiani Buana, Dhimas Arief Dharmawan, Latifah Listyalina, Dewi Purnamasari, Widhia Oktoeberza KZ)

Department of Ophthalmology, Sardjito Hospital, Yogyakarta, Indonesia (dr. Muhammad Bayu Sasongko, dr. Kartika Dhani)

THANK YOU adinugroho@ugm.ac.id adinugroho@ugm.ac.id