A pattern matcher has no idea which pixels you care about. Trained on photographs of light-skinned patients, it learns which visual features travel with which disease, and the tone of the skin around a lesion is one of the loudest features in the frame [3]. That is why the GPT-4 result indicts the feature the model is using rather than its average accuracy: the mole itself was untouched and only the background moved [6]. The border check that dermatology actually teaches lost to the surrounding pigment [7].
The gap between what is pitched and what gets done sits in the deployment story. The case for these tools is reach, with lifesaving screening in remote areas and underresourced communities where dermatologists are scarce [13]. Performance runs the other way, strongest on the tone that dominates the shared university and hospital image libraries these models are built from [11][12]. Melanoma on pigmented skin is already found later and survived less often [8].
For whoever owns the rollout, the useful grid has two axes: whether a clinician sees the output before the patient acts on it, and which direction the error runs. Consumer app, false positive, and a harmless dark spot buys panic and an appointment [9]. Consumer app, false negative, and a melanoma reads as reassurance with no second reader anywhere in the loop, which is the case when someone asks a general chatbot instead of a clinic [9][10]. Clinician tool, false positive, and someone gets a biopsy they did not need. Clinician tool, false negative, and the miss is the same, except a human can be warned in advance that background tone is a known failure mode. The clinician-assist cell is the only one of the four with a brake, but app stores are stocking the consumer version instead [1].
What this account does not give a buyer is a size for the gap. There are no accuracy percentages, no per-tone sample sizes, and no stated share of darker-skin images in the training libraries [14]. It does hand over a test that costs an afternoon. Take the images your tool currently gets right, darken the surrounding skin without altering the lesion, and count how many answers move [4][6]. If they move, you have measured your own exposure without waiting for a vendor benchmark or a peer-reviewed table.
The recommendation comes with its price attached. Hold autonomous consumer triage until you can report performance by skin tone, and ship the clinician-assist version with the background-tone failure named in the interface rather than buried in release notes. The cost of that call lands on the patients the tool was pitched to help first, who continue waiting because no dermatologist is available [13]. Training-set composition is a spec line. A product shipped without it delivers its worst performance to its stated beneficiaries [2][12].
