preprints_ui: fd8w5_v1
Data license: ODbL (database) & original licenses (content) · Data source: Open Science Framework
| id | title | description | date_created | date_modified | date_published | original_publication_date | publication_doi | provider | is_published | reviews_state | version | is_latest_version | preprint_doi | license | tags_list | tags_data | contributors_list | contributors_data | first_author | subjects_list | subjects_data | download_url | has_coi | conflict_of_interest_statement | has_data_links | has_prereg_links | prereg_links | prereg_link_info | last_updated |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| fd8w5_v1 | Simple Prompting Enhances ChatGPT’s Diagnostic Accuracy in Psychiatric Cases | Despite the centrality of the diagnostic assessment in psychiatry, the diagnostic agreement among mental health practitioners often varies from poor to moderate. The potential of Large Language Models (LLMs; such as ChatGPT), among other approaches, has been studied to be used as standardized tools to support clinicians’ decision-making. The current work investigates the diagnostic accuracy of ChatGPT 3.5 across different case presentation styles (i.e., vignette and outline) and prompting techniques. A total of 46 psychiatric cases with an accompanying diagnosis were used. Two trained clinical psychologists evaluated the accuracy of the generated diagnosis against the reference diagnosis. A robust statistical approach was then used to investigate the effect of case format and prompt type on the average diagnostic accuracy. The results showed a moderate agreement between the ratings of the two clinical psychologists (kappa = 0.687). Moreover, a statistically significant main effect of prompting technique on ChatGPT diagnostic accuracy emerged (p = 0.009). The highest accuracy was achieved when ChatGPT 3.5 was simply instructed to provide and justify a single diagnosis for each case as compared to when it was asked to provide a diagnosis likelihood (p < 0.001) or when it was asked to act as a clinical psychologist (p < 0.001). The results of the current work reinforce the potential to use ChatGPT as a supporting tool for the diagnostic step in psychiatry and provide a general indication in order to ensure good performance when using it. | 2024-09-11T11:08:13.263715 | 2024-09-11T14:50:20.932619 | 2024-09-11T14:50:00.265663 | newaddictionsx | 1 | accepted | 1 | 1 | https://doi.org/10.31219/osf.io/fd8w5 | CC-By Attribution 4.0 International | ChatGPT; clinical diagnosis; clinical psychology; diagnostic reliability; large language models; psychiatric diagnosis | ["ChatGPT", "clinical diagnosis", "clinical psychology", "diagnostic reliability", "large language models", "psychiatric diagnosis"] | Seraphina Fong; Alessandro Carollo; Martina Dal Maso; Giovanni Martinotti; Debora Luciani; Yasser Saeed Khan; Luca Pellegrini; Ornella Corazza; Gianluca Esposito | [{"id": "hzy3r", "name": "Seraphina Fong", "index": 0, "orcid": "0000-0002-8256-746X", "bibliographic": true}, {"id": "76j29", "name": "Alessandro Carollo", "index": 1, "orcid": "0000-0002-2737-0218", "bibliographic": true}, {"id": "w6esc", "name": "Martina Dal Maso", "index": 2, "orcid": null, "bibliographic": true}, {"id": "ugawh", "name": "Giovanni Martinotti", "index": 3, "orcid": null, "bibliographic": true}, {"id": "ux7nv", "name": "Debora Luciani", "index": 4, "orcid": null, "bibliographic": true}, {"id": "razxg", "name": "Yasser Saeed Khan", "index": 5, "orcid": null, "bibliographic": true}, {"id": "d6phm", "name": "Luca Pellegrini", "index": 6, "orcid": null, "bibliographic": true}, {"id": "g3e6u", "name": "Ornella Corazza", "index": 7, "orcid": null, "bibliographic": true}, {"id": "35h7k", "name": "Gianluca Esposito", "index": 8, "orcid": "0000-0002-9442-0254", "bibliographic": true}] | Seraphina Fong | Mental Disorders; Other Psychiatry and Psychology; Clinical Psychology; Psychiatry and Psychology; Medicine and Health Sciences; Social and Behavioral Sciences; Psychology | [{"id": "584240d954be81056ceca85b", "text": "Mental Disorders"}, {"id": "584240d954be81056ceca928", "text": "Other Psychiatry and Psychology"}, {"id": "584240d954be81056ceca93d", "text": "Clinical Psychology"}, {"id": "584240d954be81056cecaa2b", "text": "Psychiatry and Psychology"}, {"id": "584240da54be81056cecaaaa", "text": "Medicine and Health Sciences"}, {"id": "584240da54be81056cecac48", "text": "Social and Behavioral Sciences"}, {"id": "584240da54be81056cecac68", "text": "Psychology"}] | https://osf.io/download/66e17a70b4e35b0d2c45a5e3 | 0 | no | not_applicable | [] | 2025-04-09T21:06:22.707544 |