Between August 2021 and June 2022, researchers from the Centre for Eye Research Australia placed a self-imaging fundus camera in two general practice clinics in Victoria. An AI model read each photograph for glaucoma. They write that general practitioners "typically lack both the equipment and training necessary to diagnose glaucoma" [1]. Patients were approached while they waited to see their doctor.
The model was first trained in 2016 on colour fundus photographs from a Chinese population. In its original validation it reached a sensitivity of 95.6 percent and a specificity of 92.0 percent. However, Jan and colleagues found that it did considerably less well in the two clinics.
- One in three people could not be photographed. Of 414 adults aged 50 or over who consented, 277 had an image captured from at least one eye. The people who could not be imaged were older, 71.4 years against 64.8, and more often reported a cataract diagnosis, 40.9 percent against 19.1 percent.
- Sensitivity was 65.0 percent and specificity was 94.6 percent, counted per person. Counted per image, as in the original validation, sensitivity was 71.1 percent. The reference standard was two therapeutically endorsed optometrists grading the same photographs, with an ophthalmologist adjudicating. It included no optical coherence tomography (OCT) and no visual field test.
- Most flagged patients did not attend. Among 161 people who had seen an eye care provider in the previous twelve months with no glaucoma diagnosis, the model flagged 18. Sixteen could be contacted. Six attended. One reported a glaucoma diagnosis at follow-up three months later.
The authors wrote that "too many patients in our study did not adhere to referral" [1].
Data as a table
| Group | Stage | People |
|---|---|---|
| Everyone who consented | Consented | 414 |
| Everyone who consented | Image captured, at least one eye | 277 |
| Subgroup: eye care in 12 months, no glaucoma diagnosis | People in this subgroup | 161 |
| Subgroup: eye care in 12 months, no glaucoma diagnosis | Flagged as referable by the AI | 18 |
| Subgroup: eye care in 12 months, no glaucoma diagnosis | Contactable | 16 |
| Subgroup: eye care in 12 months, no glaucoma diagnosis | Attended the referral | 6 |
| Subgroup: eye care in 12 months, no glaucoma diagnosis | Reported a glaucoma diagnosis at follow-up | 1 |
| Measure | Original validation | Clinic | Exact 95% interval, clinic | Interval as printed |
|---|---|---|---|---|
| Sensitivity | 95.6 | 65.0 | 40.8 to 84.6 | 59.4 to 70.6 |
| Specificity | 92.0 | 94.6 | 91.0 to 97.0 | 91.9 to 97.2 |
The title calls it a pragmatic trial. The authors describe the design as prospective observational, with no comparator arm and no trial registration. They declare that one senior author is chief medical officer of Eyetelligence, an Australian retinal AI company. It is still the paper I would hand a colleague first, because it reports what product literature leaves out. It counts who could not be imaged, what the sensitivity was in a real clinic, and how many flagged patients turned up.
In my own practice, the referrals I remember are the ones that came back marked "did not attend".
AI is sold to optometrists on its accuracy. It is reasonable to expect an accurate tool to improve care in a primary care practice. However, this cannot be assumed, and a tool can also be in wide use without any evidence that it helps patients. This series looks at what happens after the accuracy figure. Both points apply to the tool I sell, and I come back to that at the end.
Two intervals in the paper do not reproduce
The sensitivity of 65.0 percent is printed with a 95 percent confidence interval (CI) of 59.4 to 70.6, and the specificity with 91.9 to 97.2. The paper's own table shows 15 people with referable glaucoma on manual grading and 5 more whose photographs the graders could not grade. The published figures reproduce from that table only when those 5 are counted as referable, which makes the sensitivity 13 of 20. The methods say that such participants were excluded, so the table and the methods do not agree.
An exact interval on 13 of 20 runs from 40.8 to 84.6 percent by my calculation. On 243 of 257, the specificity interval is 91.0 to 97.0. The published interval has the width that a denominator of 277 would give. Only the authors can settle it. Until they do, and whichever count is right, the true sensitivity in those clinics could plausibly be anywhere from about 40 to 88 percent.
What none of this has shown
No trial followed vision over time. I found seven randomised trials of an AI tool in ophthalmology or optometry published between January 2023 and September 2026 [2, 3, 4, 5, 6, 7, 8]. None had a vision outcome as its primary endpoint. Three measured a process of care, three measured accuracy or agreement, and one measured what patients knew and how anxious they were. The autorefraction trial measured corrected acuity on the day as a secondary outcome, and it was non-inferior.
One more trial was published on 12 October 2026, after the search window closed. Cleland and colleagues randomised 2,378 people at diabetes clinics in 24 hospitals in northern Tanzania. One group had AI screening and was told the result face to face before leaving. The other had standard screening and was told the result by telephone a few weeks later. Among the people whom masked graders found to have referable retinopathy, 35.9 percent attended the eye clinic within 8 weeks after AI screening, against 18.2 percent [9]. Its outcome was attendance, a process of care, so it does not change the count of vision outcomes.
Data as a table
| Trial | Participants | Setting | Primary endpoint | Result, AI arm vs comparator | Endpoint group |
|---|---|---|---|---|---|
| ACCESS | 164 young people | Two paediatric diabetes sites, Baltimore | Eye examination completed within 6 months | 100 vs 22 percent | Process of care |
| B-PRODUCTIVE | 993 patients, 105 clinic days | Retina clinics, Bangladesh, randomised by clinic day | Completed encounters per specialist hour | 1.59 vs 1.14 | Process of care |
| AI-OCT for macular oedema | 276 patients | Multicentre, Hong Kong screening programme | False-positive referral rate | 24.1 vs 69.1 percent | Process of care |
| Retina4IRD | 300 randomised, 295 analysed | Multicentre, inherited retinal disease | Top-5 genetic category accuracy | 88.5 vs 67.3 percent | Accuracy or agreement |
| EyeFM copilot | 668 participants | Single centre, 16 ophthalmologists, China | Correct diagnosis, and correct referral | 92.2 vs 75.4 percent, and 92.2 vs 80.5 percent | Accuracy or agreement |
| Deep-learning autorefraction | 282 adults | Crossover, Beijing Tongren Hospital | Agreement with subjective refraction, spherical equivalent | Mean difference 0.07 dioptres, r = 0.96 | Accuracy or agreement |
| Cataract education chatbot | 64 patients | Single-centre pilot, Brussels | Knowledge, anxiety and satisfaction | No significant difference detected | Patient-reported |
The wider AI trial literature to November 2023 looks much the same. A scoping review of 86 randomised trials of AI in clinical practice found that 70, or 81 percent, had positive primary endpoints. In 46 of the 86, or 54 percent, the endpoint was diagnostic yield or performance. Its full text records that all three ophthalmology trials reported a reduction in operational time [10].
The count comes from the evidence map behind this series. The map was built from PubMed, ClinicalTrials.gov, the Australian New Zealand Clinical Trials Registry and the regulator lists between 27 and 30 September 2026. It is not a registered systematic search, so a trial indexed under terms I did not use would be missing.
Two more gaps run through this literature. Attendance after a positive result is the step the software cannot take for the patient. It was 16 of 25 young people in a United States diabetes clinic trial [2], 6 of the 16 who could be reached in the glaucoma clinics, and just over one in three people with referable retinopathy in the AI group in Tanzania.
Outside diabetic retinopathy the evidence is also thin. The only OCT model I found tested in community optometry, in the United Kingdom, could process images from only half the participants and was no more accurate than the optometrists [11].
When the clinician and the AI disagree
Two studies measured what an AI reading does to the clinician's own decision.
Explaining a glaucoma AI's recommendation did not measurably improve optometrists' accuracy. One kind of explanation made them more likely to follow the AI when it was wrong. Gomez and colleagues put 87 optometrists in front of a glaucoma referral model built from routine eye care data. Accuracy was 51 percent unaided (95 percent CI 46 to 56). It was 58 percent with the AI recommendation and no explanation, 60 percent with a feature-importance explanation and 62 percent with a scorecard explanation. The three AI-supported conditions did not differ significantly from one another, so the study found no gain from the explanations.
Post-hoc explanations "increased over-reliance on incorrect AI recommendations" [12]. The authors note that the effect was small and rested on few cases.
The investigators chose the 20 cases so that the AI was right on 16 of them. The 80 percent describes the case set and not the model in a clinic.
In a second study, Kashiwagi and colleagues asked 45 trainee and expert ophthalmologists to grade 120 fundus photographs. At least a week later they read the same photographs again with a simulated AI. They were told it was highly accurate, and its answers were set to be wrong 30 percent of the time. Accuracy rose from 48.4 to 59.6 percent overall. It was 63.9 percent when the AI was right and 47.9 percent when it was wrong [13].
However, the paper tested the readings made with a wrong AI against the first-session readings and found no significant difference. The 47.9 percent covers a subset of the photographs and the 48.4 percent covers all 120, so those two figures are not a matched pair. The gain was larger for trainees, 14.2 percentage points against 8.6, and the effect of a wrong AI was less pronounced in experts. The standard deviations run from 20 to 27 percentage points, and a repeat reading at least a week apart carries its own learning effect.
Data as a table
| 87 optometrists | Accuracy, percent | Published 95% interval |
|---|---|---|
| Optometrists alone | 51 | 46 to 56 |
| With AI, no explanation | 58 | 56 to 60 |
| With AI, feature importance | 60 | 55 to 64 |
| With AI, scorecard | 62 | 57 to 66 |
| AI alone, on the 20 study cases | 80 | not reported |
| 45 ophthalmologists | Photographs | Accuracy, percent |
|---|---|---|
| First reading, unaided | All 120 | 48.4 |
| Second reading, with AI | All 120 | 59.6 |
| Second reading, with AI | Photographs where the AI was right | 63.9 |
| Second reading, with AI | Photographs where the AI was wrong | 47.9 |
Conflict of interest, funding and how this was written
I own OptoGuide Pty Ltd, which sells OptoGuide, a clinical decision support product for optometrists. Its clinical recommendations come from fixed rules, and no AI model runs when a clinician uses it.
No one funded this issue. I have no financial relationship with Eyetelligence, the Centre for Eye Research Australia, or any maker of an AI system named here. I checked every figure against the paper or the register it is cited to, and the errors are mine. This series will cover the field without promoting OptoGuide, and where a finding bears on rule-based decision support I will say so.
Before I believed OptoGuide changed a patient's outcome, I would want to see referral decisions made with and without it, in clinics I do not run, and followed through to what happened to each patient.
Issue 3 follows the diabetic retinopathy systems out of their pivotal trials: what held, what moved, who could be photographed and who came back after a positive screen. Issue 4 maps where AI now sits in an eye examination, what each tool does and which tools are on the Australian register. The issue after that covers the rules for Australian optometrists using AI: the guidance from the Australian Health Practitioner Regulation Agency (Ahpra) and the Code of conduct, the TGA's treatment of software as a medical device, the privacy obligations, including the automated decision-making transparency requirement that starts on 10 December 2026, and where liability sits when a tool is wrong.