As AI capabilities advance in complex medical scenarios that doctors face on a daily basis, the technology remains controversial in medical communities.
It really needs to be pinned to the top of every single discussion around chatgbt:
It does not give answers because it knows. It gives answers because it thinks it looks right.
Remember back in school when you didn’t study for a test and went through picking answers that “looked right” because you vaguely remember hearing the words in Answer B during class at some point?
It will never have wisdom and intuition from experience, and that’s critically important for doctors.
“Looks right” in a human context means the one that matches a person’s actual experience and intuition. “Looks right” in an LLM context means the series of words have been seen together often in the training data (as I understand it, anyway - I am not an expert).
Doctors are most certainly not choosing treatment based on what words they’ve seen together.
