Assertive and Reactive Prototype

Welcome to the results repository for our paper “Assertiveness-based Agent Communication for a Personalized Medicine on Medical Imaging Diagnosis” (10.1145/3544548.3580682) in proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ‘23) presented during the “AI in Health” track. In this repository, we aim to develop a prototype of the assertive and reactive trial for the Assertiveness-based BreastScreening-AI condition. The prototype is based on an early developed version of the BreastScreening prototype. To test and validate the prototype, initial User Research studies with the prototype are linked (10.13140/RG.2.2.22989.92645/1). A number of advantages over the introduction of our novel proposed type of interactive techniques appear to be evident. Furthermore, we develop the prototype the MIDA project as well. The MIDA project is a research project that deals with the use of a recently proposed technique in literature: Deep Convolutional Neural Networks (CNNs). These deep networks will incorporate information from several different modes by a User Interface (UI) implemented based on our later prototypes. We also have several channels and demos to see in our YouTube Channel, please follow us!

Citing

We kindly ask scientific works and studies that make use of the repository to cite it in their associated publications. Similarly, we ask open-source and closed-source works that make use of the repository to warn us about this use.

You can cite our work using the following BibTeX entry:

@inproceedings{10.1145/3544548.3580682,
author = {Calisto, Francisco Maria and Fernandes, Jo\~{a}o and Morais, Margarida and Santiago, Carlos and Abrantes, Jo\~{a}o Maria and Nunes, Nuno and Nascimento, Jacinto C.},
title = {Assertiveness-Based Agent Communication for a Personalized Medicine on Medical Imaging Diagnosis},
year = {2023},
isbn = {9781450394215},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3544548.3580682},
doi = {10.1145/3544548.3580682},
abstract = {Intelligent agents are showing increasing promise for clinical decision-making in a variety of healthcare settings. While a substantial body of work has contributed to the best strategies to convey these agents’ decisions to clinicians, few have considered the impact of personalizing and customizing these communications on the clinicians’ performance and receptiveness. This raises the question of how intelligent agents should adapt their tone in accordance with their target audience. We designed two approaches to communicate the decisions of an intelligent agent for breast cancer diagnosis with different tones: a suggestive (non-assertive) tone and an imposing (assertive) one. We used an intelligent agent to inform about: (1) number of detected findings; (2) cancer severity on each breast and per medical imaging modality; (3) visual scale representing severity estimates; (4) the sensitivity and specificity of the agent; and (5) clinical arguments of the patient, such as pathological co-variables. Our results demonstrate that assertiveness plays an important role in how this communication is perceived and its benefits. We show that personalizing assertiveness according to the professional experience of each clinician can reduce medical errors and increase satisfaction, bringing a novel perspective to the design of adaptive communication between intelligent agents and clinicians.},
booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems},
articleno = {13},
numpages = {20},
keywords = {Clinical Decision Support System, Healthcare, Breast Cancer},
location = {Hamburg, Germany},
series = {CHI '23}
}

Instructions

First of all, you will need NodeJS installed locally on your machine. This project needs both npm and http-server dependencies to install and run the core project. If you do not have those installed please follow the INSTALL instructions.

DICOM Server

The following assumes you will be using a git version control for this repository, storing thanks to GitHub. First, Download and Install git. Our system needs to be integrated with WADO-URI servers, DICOMWeb servers or any HTTP based server that returns a DICOM P10 instances. We suggest you to use an Orthanc server, since it is a simple and powerful standalone DICOM server by providing a RESTful API.

You can download a latest version or you can use our own sample of an Orthanc version with our examples of patient images. The instructions to use our solution are as follows.

Clone

1.1.1. Clone the DICOM Server repository:

git clone git@github.com:MIMBCD-UI/dicom-server.git

1.1.2. Go inside the project folder:

cd dicom-server/

Install

1.2.1. Install the local dependencies:

npm install

1.2.2. You can now Run the project, just follow the next section.

Run

1.3.1. Inside the project folder:

cd dicom-server

1.3.2. Start the DICOM Server (Orthanc) for MacOS:

npm run start:multi

cd ..
cd dicom-server/orthancAndPluginsOSX.stable/
./startOrthanc.command

NOTE: If you are not using MacOS, for instance, if you are using Windows or Debian you have a documentation for that. Just follow the Windows or Debian documentations. You also have several other options.

1.3.3. Open the link:

localhost:8248

NOTE: If you need some help see the Demo.

Main Server

Our main server uses NodeJS and has several dependencies. For the following steps you must have already installed both NodeJS and npm in your machine.

Clone

2.1.1. Clone the project repository:

git clone git@github.com:MIMBCD-UI/prototype-assertive-reactive.git

2.1.2. Go inside the project folder:

cd prototype-assertive-reactive/

Install

2.2.1. Install the local dependencies:

npm install

2.2.2. You can now Run the project, just follow the next section.

Run

2.3.1. Inside the project folder:

cd prototype-assertive-reactive/

2.3.2. If you have already run the DICOM Server on a previous section, please jump to the 2.3.3. point, otherwise do:

npm run dicom-server

2.3.3. Run the code:

npm run build:local

2.3.4. Start the project:

npm run start:local

2.3.5. Open the link:

localhost:8286/src/public/index.html

Allow-Control-Allow-Origin

Access-Control-Allow-Origin is a CORS (Cross-Origin Resource Sharing) header. If you want to know How does Access-Control-Allow-Origin header work? follow the link.

Google Chrome

To deal with the CORS issue it is necessary to open Google Chrome with --disable-web-security flag on:

open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir

Or install the CORS plugin for Google Chrome.

Information

The current information will summarize the essential parts of this repository. In addition to related resources, we address all significant parts that were crucial to the present information.

The following list, represents the set of related repositories for the presented one:

Dataset Resources

To publish our datasets we used a well known platform called Kaggle. To access these datasets just follow the uta4-sm-vs-mm-sheets dataset, as an example. Here, you will find all of our published datasets and any associated information, such as descriptions and download links. We are also working on several other User Research studies, while this repository is being an important asset to them.

About

For more information about the MIDA project just follow the link. Pieces of information about details of this repository are also in a wiki. This prototype was developed using several libraries and dependencies. Despite that all libraries had their importance and supported the development, one of it was of chief importance. The CornerstoneJS library and secondary libraries, respectively, are supporting this prototype. We Acknowledge all people involved in the path.

License

The prototype-assertive-reactive repository is distributed under the terms of both Academic License for academic purpose and Commercial License for commercial purpose, as well as under the CC-BY-SA-4.0 copyright. The content of the present repository has obtained the patent right of World Intellectual Property Organization (WIPO) invention. Moreover, the hereby invention for this repository is under protection of the patent number WO2022071818A1 with the application number PCT/PT2021/050029. The title of the invention is “Computational Method and System for Improved Identification of Breast Lesions”, registered under the WO patent office.

See ACADEMIC and COMMERCIAL for details. For more information about the MIMBCD-UI Project just follow the link.

Acknowledgments

A special thanks to Chris Hafey, the propelling person of CornerstoneJS, who also developed the cornerstoneDemo. Not forgetting the three supporters of the CornerstoneJS library, Aloïs Dreyfus, Danny Brown and Erik Ziegler. We also would like to give a special thanks to Erik Ziegler who support several issues during this path.

Team

Our team brings everything together sharing ideas and the same purpose, developing even better work. In this section, we will nominate the full list of important people for this repository, as well as respective links.

Authors

Francisco Maria Calisto [ Academic Website ResearchGate GitHub Twitter LinkedIn ]
João Fernandes [ ResearchGate ]
Margarida Morais [ ResearchGate ]
Carlos Santiago [ ResearchGate ]
João Maria Abrantes [ ResearchGate ]
Nuno Nunes [ ResearchGate ]
Jacinto C. Nascimento [ ResearchGate ]

Promoters

Hugo Lencastre
Nádia Mourão
Miguel Bastos
Pedro Diogo
João Bernardo
Madalena Pedreira
Mauro Machado

Companions

Bruno Dias
Bruno Oliveira
Luís Ribeiro Gomes
Pedro Miraldo

Supporting

Our organization is a non-profit organization. However, we have many needs across our activity. From infrastructure to service needs, we need some time and contribution, as well as help, to support our team and projects.