MIMBCD-UI BreastScreening Prototype

MIMBCD-UI is a Research Project that deals with the use of a recently proposed technique in literature: Deep Convolutional Neural Networks (CNNs). These deep networks will incorporate information from several different modes by a User Interface (UI) implemented based on our Prototype Cornerstone repository. The Prototype Cornerstone repository is an Open Source Project with a goal to deliver an example of web based medical imaging platform for the breast cancer diagnosis. We also have several demos to see in our YouTube Channel, please follow us.

Research

Like the fact that these projects are research projects, we underline and ask for the support of our noble scientific community to help us across the path. With this project, we hope to provide valuable information regarding our research topics and theories. Also, you can follow and support our project on ResearchGate.

Citing

We kindly ask scientific works and studies that make use of the repository to cite it in their associated publications. Similarly, we ask open-source and closed-source works that make use of the repository to warn us about this use.

You can cite our work using the following BibTeX entry:

@mastersthesis{calisto2017mimbcdui,
  doi = {10.13140/RG.2.2.15187.02084},
  url = {http://rgdoi.net/10.13140/RG.2.2.15187.02084},
  author = {Francisco Maria Calisto},
  title = {Medical Imaging Multimodality Breast Cancer Diagnosis User Interface},
  school = {Instituto Superior T\'{e}cnico},
  year = 2017,
  address = {Avenida Rovisco Pais 1, 1049-001 Lisboa - Portugal (EU)},
  month = 10,
  note = {A Medical Imaging Tool for a Multimodality use of Breast Cancer Diagnosis on an User Interface.}
}

Recent Publications

We have several publications published across several research conferences and journals. In this section, we provide a short list of our recent publications. To see a more complete list, please follow our publications.

Francisco Maria Calisto, Carlos Santiago, Nuno Nunes, Jacinto C. Nascimento,

Introduction of Human-Centric AI Assistant to Aid Radiologists for Multimodal Breast Image Classification,

International Journal of Human-Computer Studies,

Volume 150, 2021, 102607, ISSN 1071-5819,

https://doi.org/10.1016/j.ijhcs.2021.102607.

(https://www.sciencedirect.com/science/article/pii/S1071581921000252)

Abstract: In this research, we take an HCI perspective on the opportunities provided by AI techniques in medical imaging, focusing on workflow efficiency and quality, preventing errors and variability of diagnosis in Breast Cancer. Starting from a holistic understanding of the clinical context, we developed BreastScreening to support Multimodality and integrate AI techniques (using a deep neural network to support automatic and reliable classification) in the medical diagnosis workflow. This was assessed by using a significant number of clinical settings and radiologists. Here we present: i) user study findings of 45 physicians comprising nine clinical institutions; ii) list of design recommendations for visualization to support breast screening radiomics; iii) evaluation results of a proof-of-concept BreastScreening prototype for two conditions Current (without AI assistant) and AI-Assisted; and iv) evidence from the impact of a Multimodality and AI-Assisted strategy in diagnosing and severity classification of lesions. The above strategies will allow us to conclude about the behaviour of clinicians when an AI module is present in a diagnostic system. This behaviour will have a direct impact in the clinicians workflow that is thoroughly addressed herein. Our results show a high level of acceptance of AI techniques from radiologists and point to a significant reduction of cognitive workload and improvement in diagnosis execution.

Keywords: Human-computer interaction; Artificial intelligence; Healthcare; Medical imaging; Breast cancer

Instructions

First of all, you will need NodeJS installed locally on your machine. This project needs both npm and http-server dependencies to install and run the core project. If you do not have those installed please follow the INSTALL instructions.

DICOM Server

The following assumes you will be using a git version control for this repository, storing thanks to GitHub. First, Download and Install git. Our system needs to be integrated with WADO-URI servers, DICOMWeb servers or any HTTP based server that returns a DICOM P10 instances. We suggest you to use an Orthanc server, since it is a simple and powerful standalone DICOM server by providing a RESTful API.

You can download a latest version or you can use our own sample of an Orthanc version with our examples of patient images. The instructions to use our solution are as follows.

Clone

1.1.1. Clone the DICOM Server repository:

git clone git@github.com:MIMBCD-UI/dicom-server.git

1.1.2. Go inside the project folder:

cd dicom-server/

Install

1.2.1. Install the local dependencies:

npm install

1.2.2. You can now Run the project, just follow the next section.

Run

1.3.1. Inside the project folder:

cd dicom-server

1.3.2. Start the DICOM Server (Orthanc) for MacOS:

npm start

cd ..
cd dicom-server/orthancAndPluginsOSX.stable/
./startOrthanc.command

NOTE: If you are not using MacOS, for instance, if you are using Windows or Debian you have a documentation for that. Just follow the Windows or Debian documentations. You also have several other options.

1.3.3. Open the link:

localhost:8042

NOTE: If you need some help see the Demo.

Main Server

Our main server uses NodeJS and has several dependencies. For the following steps you must have already installed both NodeJS and npm in your machine.

Clone

2.1.1. Clone the project repository:

git clone git@github.com:MIMBCD-UI/prototype-breast-screening.git

2.1.2. Go inside the project folder:

cd prototype-breast-screening/

Install

2.2.1. Install the local dependencies:

npm install

2.2.2. You can now Run the project, just follow the next section.

Run

2.3.1. Inside the project folder:

cd prototype-breast-screening/

2.3.2. Create the main conficuration file:

vim config/env.json

p.s.: we use vim but you can use yours.

2.3.3. Fill the env.json file with:

{
  "environment": ""
}

2.3.4. Put the desire deployment environment:

{
  "environment": "local"
}

Note: by default the local environment should be the one to test in your machine.

2.3.5. Save the file;

2.3.6. If you have already run the DICOM Server on a previous section, please jump to the 2.3.7. point, otherwise do:

npm run dicom-server

2.3.7. Run the code:

npm run build:prod

2.3.8. Start the project:

npm start

2.3.9. Open the link:

localhost:8086/src/public/index.html

Allow-Control-Allow-Origin

Access-Control-Allow-Origin is a CORS (Cross-Origin Resource Sharing) header. If you want to know How does Access-Control-Allow-Origin header work? follow the link.

Google Chrome

To deal with the CORS issue it is necessary to open Google Chrome with --disable-web-security flag on:

open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir

Or install the CORS plugin for Google Chrome.

About

For more information about the MIMBCD-UI project just follow the link. Pieces of information about details of this repository are also in the Wiki of the project. This prototype was developed using several libraries and dependencies. Despite that all libraries had their importance and supported the development, one of it was of chief importance. The CornerstoneJS library and secondary libraries, respectively, are supporting this prototype. We Acknowledge all people involved in the path.

Information

The current information will summarize important items of this repository. In this section, we address all fundamental items that were crucial to the current information.

The following list, represents the set of related repositories for the presented one:

License

The prototype-breast-screening repository is distributed under the terms of both Academic License and Commercial License, for academic and commercial purpose, respectively. For more information regarding the License of the hereby repository, just follow both ACADEMIC and COMMERCIAL files.

Copyright

When source code, documentation and other content is contributed to one of our repositories, the copyrights in those contributions remain owned by the original copyright holders. The copyrights are not assigned to the organization itself. Instead, they are licensed for distribution as part of the project. Whether a project uses new incremental solutions, the original copyright holders retain their copyrights.

Intellectual Property

The content of the present repository has obtained the patent right of World Intellectual Property Organization (WIPO) invention. Moreover, the hereby invention for this repository is under protection of the patent number WO2022071818A1 with the application number PCT/PT2021/050029. The title of the invention is “Computational Method and System for Improved Identification of Breast Lesions”, registered under the WO patent office.

Team

Our team brings everything together sharing ideas and the same purpose, developing even better work. In this section, we will nominate the full list of important people for this repository, as well as respective links.

Authors

Francisco Maria Calisto [ Website ResearchGate GitHub Twitter LinkedIn ]

Companions

Bruno Oliveira
Luís Ribeiro Gomes
Carlos Santiago
Jacinto C. Nascimento
Pedro Miraldo
Nuno Nunes

Acknowledgments

This work was partially supported by national funds through FCT and IST through the UID/EEA/50009/2013 project, BL89/2017-IST-ID grant. We thank Dr. Clara Aleluia and her radiology team of HFF for valuable insights and helping using the Assistant on their daily basis. From IPO-Lisboa, we would like to thank the medical imaging teams of Dr. José Carlos Marques and Dr. José Venâncio. From IPO-Coimbra, we would like to thank the radiology department director and the all team of Dr. Idílio Gomes. Also, we would like to provide our acknowledgments to Dr. Emília Vieira and Dr. Cátia Pedro from Hospital Santa Maria. Furthermore, we want to thank all team from the radiology department of HB for participation. Last but not least, a great thanks to Dr. Cristina Ribeiro da Fonseca, who among others is giving us crucial information for the BreastScreening project.

A special thanks to Chris Hafey, the propelling person of CornerstoneJS, who also developed the cornerstoneDemo. Not forgetting the three supporters of the CornerstoneJS library, Aloïs Dreyfus, Danny Brown and Erik Ziegler. We also would like to give a special thanks to Erik Ziegler who support several issues during this path.

Supporting

Our organization is a non-profit organization. However, we have many needs across our activity. From infrastructure to service needs, we need some time and contribution, as well as help, to support our team and projects.