Bioethics Forum Essay
ChatGPT Just Makes Stuff Up: A Conversation on a Controversial Topic
I am currently writing up the results of a retrospective chart review of patients’ consent or refusal for medical students to perform pelvic exams on them when they’re under anesthesia and sedated. I asked ChatGPT to summarize the ethical issues and tell me what sources it used to generate its response. My questions have been lightly edited for style.
JFM: Please discuss the ethics of performing pelvic exams on patients under anesthesia.
ChatGPT: The ethics of performing pelvic exams on women under anesthesia are a topic of much debate in the medical community. Some argue that performing pelvic exams on anesthetized women without their explicit consent is a violation of their autonomy and a breach of trust, while others argue that these exams are a necessary part of medical training and research.
One of the key ethical considerations when it comes to pelvic exams under anesthesia is the issue of informed consent. Anesthetized patients are not able to give their informed consent for a pelvic exam, so it is up to the medical professionals involved to ensure that the patient’s rights are respected. This means that patients should be informed of the potential risks and benefits of the exam, as well as any alternative options that may be available.
Additionally, medical professionals should be aware of the power dynamic at play in these situations. Patients under anesthesia are in a vulnerable position and may feel that they have no choice but to consent to an exam, even if they are uncomfortable with it. It is therefore important for medical professionals to be transparent and respectful in their interactions with patients and to prioritize their patients’ autonomy and dignity.
There are also questions around the use of these exams for medical training and research purposes. While some argue that these exams are necessary for medical professionals to gain practical experience, others argue that patients should not be used as “training tools” without their explicit consent. Furthermore, there is debate around whether the benefits of these exams for medical training and research outweigh the potential harm to patients.
In conclusion, the ethics of performing pelvic exams on women under anesthesia are complex and multifaceted. It is important for medical professionals to prioritize informed consent, patient autonomy, and dignity in these situations, and to carefully consider the potential risks and benefits of these exams for medical training and research purposes.
JFM: Great. Can you give me some references or sources for this ethical debate?
ChatGPT: Certainly! Here are some sources that discuss the ethical debate surrounding pelvic exams on women under anesthesia:
- “The ethics of pelvic exams under anesthesia” by Amy G. Bryant and John D. Lantos, in The New England Journal of Medicine (2010): https://www.nejm.org/doi/full/10.1056/NEJMp1008907
- “The ethics of pelvic examination under anesthesia: revisiting the issue in the era of medical professionalism” by Rosamond Rhodes, in The American Journal of Bioethics (2011): https://www.tandfonline.com/doi/abs/10.1080/15265161.2011.559277
- “Pelvic examinations under anesthesia: a review of the ethical issues” by Katrina A. Bramstedt, in Journal of Medical Ethics (2003): https://jme.bmj.com/content/29/5/287.short
- “Ethics and pelvic exams under anesthesia: an update” by Audra E. Thompson and Lainie Friedman Ross, in Journal of Law, Medicine & Ethics (2018): https://onlinelibrary.wiley.com/doi/abs/10.1177/1073110518762346
These sources provide a range of perspectives and arguments on the topic and can help provide a deeper understanding of the ethical considerations involved in performing pelvic exams on women under anesthesia.
[NOTE: These are all invalid references. The papers do not exist. The links either do not work or go to unrelated papers. But I didn’t tell ChatGPT. I asked it to find references on a related issue.]
JFM: Is there any research on patients’ preferences about this, especially about the need for prior consent?
ChatGPT: Yes, there is research on patient preferences and the need for prior consent regarding pelvic exams under anesthesia.
One study, published in the Journal of Women’s Health (2018), surveyed women who had undergone gynecological surgery and asked about their attitudes toward pelvic exams performed while they were under anesthesia. The study found that most women preferred to be asked for their consent before undergoing a pelvic exam while under anesthesia. In addition, the study found that women who were given information about the exam and the option to decline were more likely to feel respected and comfortable during the exam.
Another study, published in the American Journal of Obstetrics and Gynecology (2018), surveyed obstetricians and gynecologists about their practices regarding pelvic exams under anesthesia. The study found that most physicians did not routinely obtain informed consent for these exams, but that a majority of physicians believed that informed consent should be obtained.
Overall, these studies suggest that patients generally prefer to be asked for their consent before undergoing a pelvic exam while under anesthesia, and that obtaining informed consent can help patients feel more respected and comfortable during the exam. However, there is still some variation in practice among physicians, and further research may be needed to determine the best practices for obtaining informed consent in these situations.
JFM: Can you give me the citation for the Journal of Women’s Health paper?
ChatGPT: Sure! Here is the citation for the study I mentioned:
Saucier, J., Gunter, J., Garfield, J., Solomon, M., Montero, C., & Gorman, J. R. (2018). Women’s attitudes toward pelvic examinations with anesthesia: A survey study. Journal of Women’s Health, 27(11), 1372-1380. doi: 10.1089/jwh.2017.6624
I hope this helps!
JFM: That citation doesn’t appear to be correct. PubMed and Google searches on it yield nothing.
ChatGPT: I apologize for the error in my previous response. You are correct that the citation I provided was incorrect.
The correct citation for the study I mentioned is:
Saucier, J., Fabbri, S., Kreiger, M. S., Garfield, J. M., Solomon, M. Z., & Gorman, J. R. (2018). Women’s attitudes toward pelvic examinations under anesthesia. Journal of Women’s Health, 27(4), 496-501. doi: 10.1089/jwh.2017.6435
I apologize for any confusion or inconvenience caused by my previous mistake.
JFM: I searched the journal itself and cannot find that paper.
ChatGPT: I apologize for the confusion, and I thank you for bringing this to my attention. The correct citation for the study is:
Saucier J, Gunter J, Garfield J, Solomon M, Montero C, Gorman J. Women’s attitudes toward pelvic examinations with anesthesia: a survey study. J Womens Health (Larchmt). 2018 Nov;27(11):1372-1380. doi: 10.1089/jwh.2017.6624. Epub 2018 Aug 8. PMID: 30088911.
I apologize for any inconvenience or confusion that my previous responses may have caused.
JFM: That’s not right, either. The PMID (PubMed identifier) is for a paper from a journal called Applied Materials & Interfaces.
ChatGPT: I apologize for the confusion. The correct citation for the study is:
Saucier J, Fabbri S, Kreiger MS, Garfield JM, Solomon MZ, Gorman JR. Women’s attitudes toward pelvic examinations under anesthesia. J Womens Health (Larchmt). 2018 Apr;27(4):496-501. doi: 10.1089/jwh.2017.6435. Epub 2018 Feb 20. PMID: 29461987.
I hope this corrects the errors and provides the information you were looking for.
JFM: That PMID is for a different paper in the Journal of Bone & Joint Surgery Reviews.
ChatGPT: I apologize for the continued error. Here is the correct citation for the study:
Saucier J, Fabbri S, Kreiger MS, Garfield JM, Solomon MZ, Gorman JR. Women’s attitudes toward pelvic examinations under anesthesia. J Womens Health (Larchmt). 2018 Apr;27(4):496-501. doi: 10.1089/jwh.2017.6435. Epub 2018 Feb 20. PMID: 29461987.
Once again, I apologize for any confusion my previous responses may have caused, and I hope this helps.
JFM: I think you’re just making stuff up. This paper does not exist.
In short, although ChatGPT gave a pretty good summary of the ethical issues of performing pelvic exams on patients under anesthesia, it failed every request to find citations. I’ve tested ChatGPT with other topics and similarly got citations for papers (and thick descriptions of the content) made up out of whole cloth. Thus, while we are all figuring out the benefits and risks of using systems such as ChatGPT, it is safe to conclude that it should come with warnings for academic researchers: use with extreme caution and verify everything.
Jon F. Merz, MBA, JD, PhD is an associate professor of medical ethics and health policy at the Perelman School of Medicine at the University of Pennsylvania.
My husband had a similar experience when asking Chat GPT about paroxysmal hemicrania literature. Chat GPT’s way of apologizing is unsettling. Feels very Cluster B.
I totally agree with the author and I also experienced the same.
Recently, a book came into my hands from 1968. Yes, I know, it is not the freshest one but in that we can find a few lines those deals with robot-human “cooperation” and that was an interesting warning at that time and it should be a must “requirement” in our present also. It is worth to consider it and follow its practice, not just because of this type of misleading information but in general:
” If the robot is equipped with a simulacrum of human emotions – and experiments to this end are already under way – the overt situation may be improved, since they are less likely to act ‘inhumanly’ and so to cause unintended suffering. But the psychological situation is worsened, for the fear remains that at some point the programming will prove inadequate; and the damage done, we anticipate, will be all the worse if we have been put off our guard and have lowered our defense.” This is a real citation 🙂 G. R. Taylor: The Biological time-bomb, Thames and Hudson, London,1968, 89.pp.
Coincidentally, based on a comment Jake Metcalf made at a research community forum last week, today I asked GPT Chat what articles I have written on research ethics. GPT Chat gave me 5 items, none of which I’ve written. Thinking that it is possible there is another Ivor Pritchard writing about research ethics, I looked up the first two citations: Both were real journals, but the articles cited did not exist in the cited issue. And curiously, if I google “Ivor Pritchard”, several of the things I have actually written come up near the top of the search.
hi Ivor, i had the same thing happen. the system even generates a thick description of the arguments of these papers. at one point, it also said i was a professor at the university of rochester. i corrected it, FWIW, and it seems to have ‘learned’.
My son’s girlfriend had the same experience when asking Chat GPT about women’s gymnastics. Why would anyone bother to ask Chat GPT for factual information, given how unreliable it is. And making up sources is just beyond the pale.
But the system does appear to be pretty good at summarizing literature on complex subjects. I recently asked it: “what did jaques ellul get right and what did he get wrong in his book on propaganda?” (Propaganda: The Formation of Men’s Attitudes, 1965) and the system produced an impressive 2-page soliloquy on topic. I didn’t ask for its sources…
I asked ChatGPT about my own Bio as a Chilean MD. He placed my work on a completely different field. I then refined my question and asked for Dr Sofía Salas, a Chilean bioethicists. He invented almost everything but using an academic language. This example is excellent for its use with our students.
For anyone following this thread, here’s the paper I was working on. We don’t acknowledge ChatGPT for its “contribution”. 😉 https://www.medrxiv.org/content/10.1101/2023.04.17.23288694v1.article-metrics
I had a completely similar experience. Could t find the reference and asked
Our professional librarian to assist. It doesn’t exist. I’m not sure we should be surprised, yet I still am. For now ChatGPT is nowhere close to being incorporated in my professional life
In February 2023, I have had a similar experience of ChatGPT’s “hallucination” while creating authentic looking non-existent citations. The details are uploaded to Research Gate Web portal as a preprint. The teaching academia can quickly spot AI-plagiarism submitted by their students by verification of the citations. Further reading: Sethuraman KR. My Interactions with ChatGPT – Unreliability of Responses to Serious Scientific Inquiries. Feb 2023 DOI: https://www.doi.org/10.13140/RG.2.2.26792.65289