Questions and answers from the "Tomorrow" session
Eric Sutherland
Senior Health Economist, OEC
Question 1: You said AI is not accountable, that it is just maths. But what about when general AI acquires consciousness?
Artificial general intelligence (AGI) refers to systems that are not task-specific but rather are able to understand and carry out a wide range of tasks. AGI does not equal or imply consciousness – so far there is no evidence to suggest that AI systems will be able to acquire consciousness in the future. On the other hand, accountability is a feature that belongs to decision-makers. So, in a healthcare setting, whoever is allowed to make a decision is simultaneously accorded accountability. As healthcare decisions should be made by healthcare professionals who are trained in making those decisions, they will be the ones who will be accountable. To give an example, there are decision-support systems being deployed today in clinical settings (e.g. when evaluating an ECG or prescribing medication); these give a recommendation to the doctor, but it is the latter who takes the decision and is therefore accountable.
Question 2: Will AI replace doctors with robots?
In its current state, AI cannot replace doctors. Depending on how it progresses, however, we cannot rule out the possibility of robot doctors, but this is up to us as societies to decide which professions we want to replace by automation and to what extent. AI solutions are best thought of as augmenting human capabilities.
Question 3: Would you agree that “artificial intelligence” is the wrong term and we should use “algorithm” instead?
Although both terms cannot necessarily be used interchangeably, algorithms can be used to replace the term AI when talking about decision-support systems in clinical settings. Algorithms are a good analogy for the AI that has been in development since the 1940s. More recent advancements with generative AI are more complicated than simply referring to them as “algorithms”. I prefer “augmented intelligence” as it emphasises the ability of this tool to help humans to take decisions.
Question 4: There is no question that AI is a powerful tool but how can it be ensured that the data that is being used is accurate and real?
Data quality is one of the main challenges in developing powerful AI. Without high-quality data, it is not possible to develop high-performing AI. On the other hand, AI that is not performing sufficiently well would not/should not be cleared for clinical use and therefore data quality is more an issue for algorithm development and less so for algorithm deployment. This is where a data governance programme – which includes a defined approach for AI – is an essential component of an overall AI-in-health strategy.
Question 5: Humans remain responsible, but how can we be confident in the assessments made by AI? We can’t reassess the data ourselves – what is the level of quality control on AI?
This question primarily regards AI as a decision-support tool. Any algorithm that should be employed clinically goes through a quality assessment and it is decided if its accuracy across a range of parameters (sensitivity, specificity, positive and negative predictive rates) is sufficient for it to be deployed in a clinical setting. When physicians use it as a decision-support tool, they have to consider if the output of the algorithm is consistent/coherent with their expertise and they have to decide whether to accept or reject the decision/suggestion made by the algorithm. The two instances that are central to confidence and carry the accountability in this process are 1) regulatory bodies like the FDA and EMA that approve the tool for use in clinical settings; and 2) the healthcare professionals making the decisions based on their medical expertise and judgement.
Question 6: Is it possible to validate AI according to 21 CFR and/or GxP requirements?
Validating an algorithm refers to demonstrating its accuracy in a specific setting for a specific clinical task. Different countries will have different regulations for giving clearances to algorithms and the level of evidence required for clearance. This is an ongoing topic of discussion about how and when AI solutions would be clinically validated and how that validation would be managed as the AI solutions continuously improve.
Question 7: There is a huge risk of doctors always relying on the diagnostic suggestions of AI in the future. How can we mitigate this risk?
The current literature shows mixed results concerning overreliance on AI (and other decision-support systems), with generally low rates of overreliance being reported. In general, as a prevention strategy, you could say that the better doctors are informed about the strengths and shortcomings of a support system, the less over reliant on it they will become. A critical aspect of this is that doctors need to be educated on how to responsibly integrate AI into their decision-making processes, which includes effectively challenging recommendations. The risk is that doctors have less opportunity to develop their diagnostic skills due to the increased role given to AI. It will always be important for health professionals to be the face of care for patients and have the knowledge and skills to deliver excellent care, which will be increasingly augmented with tools such as AI.
Question 8: If AI has potential advantages and we as humans fail to increase collaboration, could we not use AI to improve collaboration and togetherness instead?
Using AI for one purpose would not exclude other purposes. Having said that, AI is a tool used by humans. If humans do not want to collaborate, AI cannot make them do it. However, AI as a tool may make it easier for collaboration to occur (e.g. by identifying areas of common interest or overcoming communication barriers).
Question 9: With regard to blood/organs donors, how can we assure security and anonymity of data? Who owns it? Who can profit from it? Patients, corporation, states?
Data governance is crucial, and the degrees of anonymity and security required change depending on the nature of the data. There is no regulatory/legal consensus on the definition of data ownership. In principle, individuals should have the right to opt-out as well as the right to be forgotten. Anyone can benefit from data sharing and the more the data are shared, the more societies can benefit from it. There are increasing number of “privacy enhancing technologies” that are able to improve the security and anonymity of data while optimising its useability.
Question 10: This sounds a bit utopian. Free flow of data across borders with likeminded people for common benefits – great. What about sharing with those who do not play nice?
The misuse of data by malicious actors is a serious risk. However, there is a middle ground between locking up data and opening it up to unlimited access for everyone. It is, for example, technically possible to make data available for use without sharing individual data points and we advocate for making data available for use and using appropriate technology to monitor and prevent misuse. Governance of the data ecosystem would include screening for those who “do not play nice” and establish clear penalties in those cases.
Question 11: Data must be provided with patients' consent – how can we build up trust in AI so that patients share the data required to make a prevention-based health system possible?
AI policy is data policy. The approach to obtaining consent for the use of personal data – specifically in the development of AI solutions, but also more generally in the realm of digital health – is a current challenge where legacy approaches to consent have had mixed results. This is an area where collaboration within and across Europe will be beneficial to unlocking the life-saving potential of personal data for individuals and communities while protecting the privacy of the individual. In particular, as we shift from the legacy of facility-centric health systems to truly people-centric health systems, it will be necessary to embed a consent architecture such that consent is obtained and shared broadly. Equally, those consent-based models will need to determine when data use in the public interest overrides individual consent as well as the safeguards for individual data protection that are essential in those situations. To foster trust with the public, we need to co-design this consent architecture for a person-centric system with the public. And once those designs are in place, communication with the general public is key to ensuring their freedom to choose while ensuring safety and equality for the wider public.
Fiona AdsheadChair of the Sustainable Healthcare Coalition |
Kirsty ReidDirector Science Policy, European Federation of Pharmaceutical Industries and Associations |
Question 12: Global transport has a big impact on the environment – what more can be done to have a local ecosystem for pharmaceuticals and still promote accessibility?
There are many initiatives underway to decrease the impacts of transportation of raw materials and final medicinal products. These come under Scope 3 of emissions, and the highest emission concerns fall within this scope. Companies are committing to decreasing emissions and working with suppliers in this regard. Some specific case studies can be found at https://www.efpia.eu/media/sydk5acr/white-paper-on-climate-change.pdf.
Question 13: What is the responsibility and corresponding actions taken by industry concerning medicines/substances excreted by patients into the environment?
Multiple initiatives under the Eco-Pharmaco-Stewardship, an initiative led by the European Federation of Pharmaceutical Industries and Associations (EFPIA), the Association of the European Self-Care Industry (AESGP) and Medicines for Europe has contributed over the last couple of years to improving scientific understanding, finding new ways of detecting trace amounts of pharmaceuticals in the environment, understanding their impact, prioritising APIs posing a potential risk to the environment and also further reducing discharges from manufacturing plants. As an industry, we are striving to continually enhance our processes to deliver life-saving treatments in ways that are also protective of the environment. Details of action taken can be found at https://www.efpia.eu/media/636524/efpia-eps-brochure_care-for-people-our-environment.pdf.
Question 14. How can we develop greener healthcare processes and ensure the health of patients is not compromised at the same time?
Healthcare organisations and clinicians should always put the needs of patients first – green solutions should only be adopted when it is clinically appropriate.
Greening healthcare systems can bring a range of mutual benefits by delivering care more effectively and efficiently through reducing energy costs, improving efficiency, and promoting early intervention and prevention.
In order to achieve these benefits, we need to integrate environmental factors into healthcare decision-making, and consider how by being “sustainable by design” we can both improve health outcomes, reduce cost and reduce environmental impact.
Harmonised standard methods of measurement will be critical to achieving this goal. From our own work on care pathway impact assessment in the Sustainable Healthcare Coalition, we know how important it is to take a systems approach to impact measurement across the care pathway (https://shcoalition.org/sustainable-care-pathways/), and how critical it is to work in collaboration to produce new assessment methods, such as those on clinical trials (https://shcoalition.org/clinical-trials/).
Question 15: The “green” production of active substances is generally more expensive, especially in terms of equipment. Won't this make drugs more expensive?
Several different factors affect the cost of medicines, including the amount of R&D, cost of components and therapeutic type. Concerns over cost should not hold back the development and investment into “green” production opportunities.
Mark Skylar-Scott
Assistant Professor of Bioengineering, Stanford University
Question 16: To make this a reality in due time, what steps should policy makers start taking at the European level?
Expanding government funding for basic and applied scientific research remains the ultimate mechanism for supporting the bold and long-term thinking necessary for technologies such as organ biomanufacturing. In light of the 3D bioprinting and stem cell advances presented, the USA and China are both making large investments into organ bioprinting, and the EU should have a horse in this race. Beyond funding, policy makers should foster regular and in-depth communication between scientists, engineers, regulators, lawmakers and patient advocacy groups is crucial to ensuring that organ biomanufacturing technology is developed with a focus on equal access, ethics, safety and patient wellbeing.
Question 17. Embedded 3D printing looks promising; why are our countries not making the effort to fund research in this field as a priority?
Embedded 3D printing is undoubtedly one of the most exciting advances in the past decade, allowing us to write soft and living cells into complex 3D shapes. The good news is that the EU already has several leading scientists in embedded 3D bioprinting and other forms of 3D bioprinting. However, funding always remains rate limiting for technological progress in the basic and applied sciences. Funding constraints are especially acute when manufacturing large scale organs; whole organs require a lot of cells and biomaterials and these ingredients do not come cheap!
Question 18: Thanks for the presentation. Do you think this would apply to autologous organs only or for allogeneic organs too if autologous would take too long or not work?
Most clinical trials using induced pluripotent stem cells are currently opting for banked (donor) allogeneic cells to ease regulatory, procedural and economic burdens associated with autologous therapies. In our organ manufacturing programme, the question of autologous versus allogeneic has little impact in the scientific or engineering aspects, and we have opted for a banked cell line for our initial demonstrations. However, as scientists, clinicians, and regulators gain more experience in deriving, characterising, regulating and delivering cell therapies, I believe that the benefits of not requiring lifelong immunosuppression and rejection-free transplantation will, in the long run, point us towards autologous approaches.
François-Xavier Lery
Head of the Pharmaceutical and Consumer Care Section, EDQM, Council of Europe
Question 19: Many talked about collaboration with stakeholders and patients. How do we make sure that the patient’s voice is heard during decision making?
EDQM involves relevant stakeholders including patient by while conducting public and targeted consultations on elaborated draft standards (e.g. recommendations/resolutions addressed to governments or guidelines addressed to healthcare professionals/authorities ) before these standards are adopted by member states through intergovernmental committees and then by the Committee of Ministers when relevant.
Vanja Nikolac-Markić
Head of the Substances of Human Origin (SoHO) Division, EDQM
Question 20: What is the main cause of organ donor shortages in Europe? What can we do to improve the situation?
The demand for organ transplants is increasing worldwide, with more patients eligible for these often life-saving treatments and a rise in successful outcomes. However, the demand far outstrips the number of organs donated/available for transplantation, and this has resulted in an organ shortage.
The procurement of organs from a deceased person depends first and foremost on the willingness of that person to donate their organs after their death, and on the ability of the health system to carry out this process in accordance with the highest professional standards. It is therefore crucial to raise public awareness of organ donation as a responsibility towards society, and a decision to be discussed and taken during life. Every year, the EDQM sponsors European Day for Organ, Tissue and Cell Donation (EDD) in a host country, which organises events to raise public awareness of the need for organ, tissue and cell donation, promote the principle of voluntary and non-remunerated donation, and honour donors and healthcare professionals. More about EDD.It is also necessary to adapt the organisation and technical capacity of healthcare systems to the specific needs of the donation process, which takes place in the most sensitive context of end-of-life care. All ethical topics that may arise (in terms of end-of-life care and the donation process) should be transparently resolved to ensure public trust.
Question 21: What concrete action could the EDQM take to support the development of these artificial organs - waiting for victims of car accidents is not ideal!
Although the results are encouraging, the development of artificial organs is still in the early stages of research. The problems linked to artificial organs are completely different from those associated with human organ transplantation. Therefore, for the time being at least, this is outside the EDQM’s work programme.
Alexandre Méjat
Deputy Director and Head of the International Scientific Networks division, AFM Téléthon
Question 22: Are CAR-T cells already used for cancer treatment? How can such a costly gene therapy be made available to vast number of patients?
Yes, CAR-T cell technology is already being used for lymphomas and certain childhood acute leukaemias, but it involves collecting the patient's own lymphocytes, freezing them and sending them to the USA or Europe (a production unit in France will soon be operational). Reducing the cost of such therapy and making it accessible to a large number of patients will require lymphocyte modification to be carried out in the hospital itself, and therefore more hospitals would need to have the means to produce these cells on-site.
Question 23: Gene therapy is very promising. What are the three main challenges of this type of therapy and how can we tackle them?
To make gene therapy accessible to as many people as possible, it is necessary to:
- better understand the immune response induced by these treatments, in order to control their consequences;
- increase production capacity, reducing the cost of these innovative therapies indirectly;
- implement genetic diagnostic programs, including newborn screening, to reduce diagnostic delay and ensure the patients are able to access treatment as early as possible.
Question 24: Can gene therapy (zolgensma) help to treat type 2 spinal muscular atrophy (SMA) in 2-3 year-olds?
Gene therapy for SMA is most effective when administered early. Maximum efficacy is considered to be before 6 months, although some countries allow injections up to 2 years. Beyond that, the effects may be very limited.
Question 25: How can we solve the issue of impurities in gene therapy?
Eliminating impurities from gene therapy preparations involves not only removing elements that are not gene therapy vectors, but also separating empty (inactive) particles from complete (active) ones. Various gradient or filtration separation methods can be used to purify preparations. However, increasing the production capacity of gene therapy vectors while raising their level of purity is still the subject of very active research.
Question 26: Is there any hope that Amyotrophic Lateral Sclerosis (ALS)/Charcot disease can be treated in the same way as SMA?
In recent years, gene therapy approaches have been developed for demyelinating (CMT 4J), intermediate (CMT X1) or axonal (CMT 2D) forms of Charcot-Marie-Tooth disease, making it possible to target Schwann cells with a therapeutic gene and thus reduce peripheral nerve damage.
Similarly, in ALS, the idea is to produce a viral vector that will deliver the therapeutic SOD1 or FUS gene to the diseased cells, then inject it into the brain and spinal cord to penetrate the motor neurons.
Question 27: What proportion of newborn babies should be screened for type 1 SMA?
According to the most recent newborn screening programs, SMA prevalence is about 1/15 000 (varying between 1/7000 to 1/30 000), and type 1 SMA makes up about 50% of cases. Pilot programmes usually screen about 200 000 babies before being extended to generalised national programmes.