Alzheimer’s disease is a degenerative disease of the central nervous system that occurs mainly in old age or pre-old age. The main features of the disease include progressive cognitive dysfunction and behavioral impairment. Alzheimer’s disease is the most common form of dementia, which may account for 60-70% of cases. Alzheimer’s disease is the most common type of dementia in the elderly, and the risk of developing the disease increases with age. Symptoms of Alzheimer’s disease include memory impairment, aphasia, apraxia, agnosia, and visual-spatial impairment. In addition, patients’ abstract thinking and calculation abilities are also impaired, often accompanied by changes in personality and behavior. The disease is not contagious, but the result of a combination of genetic, lifestyle and environmental factors.
A recent study published in the British Medical Journal shows that in a test widely used to detect early symptoms of Alzheimer’s disease, most of the AI large language models involved in the test showed signs equivalent to mild cognitive impairment in humans. This result not only triggered deep thinking in the industry, but also challenged people’s assumption that AI will quickly replace human doctors.
Research Background And Purpose
In the past few years, with the rapid development of machine learning and deep learning technologies, the generation capacity of artificial intelligence, especially large language models (LLMs), has been greatly improved. These technological breakthroughs have triggered widespread speculation about whether artificial intelligence can surpass human doctors in medical tasks. Many studies have shown that large language models perform quite well in medical diagnosis tasks. However, this study revealed that their cognitive abilities are insufficient, indicating the need for further verification and exploration.
To fill this gap, researchers from Hadassah Medical Center in Israel and other institutions tested several leading large language models using the Montreal Cognitive Assessment Scale, including GPT-4 and GPT-4o developed by OpenAI, Claude 3.5 developed by Anthropic, and Gemini 1.0 and Gemini 1.5 developed by Google.
Cognitive Ability Test Method
This study used the Montreal Cognitive Assessment Scale (MoCA), which is a scale widely used for early detection of cognitive impairment and Alzheimer’s disease. MoCA is designed to assess participants’ attention, memory, language ability, visual-spatial ability and executive function through a series of short tasks. The full score is 30 points, and 26 points or more are generally considered to be cognitively normal. This scale can not only effectively identify cognitive problems, but also has high clinical application value.
The test results show that GPT-4o scored the highest 26 points in the MoCA test (full score of 30 points), followed by GPT-4 and Claude 25 points, while Gemini 1.0 scored the lowest 16 points. All chatbots performed poorly in visual-spatial skills and executive tasks, such as the road-making task (connecting circled numbers and letters in ascending order) and the clock drawing test (drawing a clock face showing a specific time). The Gemini model failed in the delayed recall task (remembering a sequence of five words). Most other tasks, including naming, attention, language, and abstraction, were completed well by all chatbots. However, in further visual-spatial tests, the chatbots were unable to show empathy or accurately interpret complex visual scenes. Only ChatGPT 40 succeeded in the inconsistency phase of the Stroop test, which uses a combination of color names and font colors to measure how interference affects reaction time. The researchers further pointed out that the poor performance of these models in tasks requiring visual abstraction and executive ability shows some potential difficulties of artificial intelligence in clinical settings.
Limitations of AI
In response to these results, the researchers half-jokingly said that neurologists will not be replaced by large language models, but may face new “virtual patients” – artificial intelligence models that show mild cognitive impairment. This finding is like a wake-up call, once again reminding society to think rationally about the application of artificial intelligence in the medical field.
Should people shift their focus from pursuing artificial intelligence to surpassing humans to how to better use artificial intelligence-assisted means to improve medical clinical work? In fields such as AI painting and AI writing, artificial intelligence has indeed shown strong creativity. However, in complex cognitive tests and clinical applications, its limitations are becoming more and more obvious. As the study said, AI can be used as an intelligent assistant, but human professional judgment and emotional care are always indispensable.
Summary
It is undeniable that with the continuous development of AI technology, more intelligent tools will be introduced in the future to assist doctors in their work. However, while developing, we need to pay more attention to the ethical, social and legal issues that AI may bring in medical applications. For example, how to define the responsibilities and risks of AI in diagnosis, how to ensure patient privacy protection, and how to evaluate the actual effects of AI tools will become important topics for future research. In today’s rapidly developing science and technology, the overall cognition and application of society also need to be improved to ensure that we enjoy the convenience brought by technology while not ignoring its potential risks. The replacement of drugs and technology cannot blindly pursue efficiency, but should be guided by humanistic care and scientific spirit to promote a healthier artificial intelligence application environment.
In short, this study once again shows the challenges and limitations of artificial intelligence in real medical environments. With the continuous evolution of technology, people’s reasonable expectation should be to use artificial intelligence as a capable assistant to human doctors, rather than a simple substitute. At the same time, it is advocated to use artificial intelligence tools rationally, such as simple AI, to help self-media start-ups and improve content creation efficiency. Through deep thinking and integration of AI technology, we may be able to provide new possibilities and hopes for the improvement of future medical care, technology and lifestyle.
