This work highlights the distinctive way computers recognise objects and draws a contrast between objects and their physical attributes and how these are assigned values by a computer vision system.
An installation “Of Machines Learning to See Lemon” was presented at the London Design Festival 2018 (image above). As part of our process to investigate machine learning image classification algorithms we created a website leveraging the public MobileNet model (a speed optimized version of the ImageNet project, a dataset of over 15 million high-resolution images collected from the web and labelled by humans.) This website uses a mobile, or computer camera to return both what it thinks is in the image, and the percentage probability that this thing is a lemon. This investigative project can be seen here: learningtoseelemon.netlify.com
Computers see things very differently to humans. We use the term ‘see’ here to indicate the way digital technologies using computer vision increasingly perform some of the functions of the human visual system but in a distinctive way. For example, an automated image recognition algorithm, although it is rooted in the search for the neural ‘algorithm’ of human visual object recognition works on the basis of what it has been previously exposed to. It can only ‘see’ based on what it has been programmed to see. The ability of a computer to correctly identify a lemon thus depends on the number, quality, and accuracy of examples in the data class ‘lemon’ it has processed. The elision of this difference in visual processing conceals where training images come from, who has generated them, and in what context. It is also increasingly clear that computational image recognition is a process subject to the cultures, biases and prejudices of the particular situations in which it is developed. This work was developed in the context of an exhibition at London College of Communication in September 2018. The exhibition explored the twin contemporary technological characteristics of complexity and speed. The image above is illustrative of the way computer vision technologies work to recognise objects by assigning values to attributes. In this case attributes of colour and shape. As the algorithm gets closer to its intended target, a lemon, these attributes become more recognisable to the human eye. We have chosen lemons as the example for this work so as to evoke the long history of lemons in European still life painting and thereby highlight the very different values at play between human and computational ‘seeing’. The work is also intended to explore the fragility of computer vision systems, particularly when they may mis-recognise an object. This problem is currently addressed using a data first approach, which maintains that more data automatically equals greater accuracy without questioning the possible bias of image provenance.
McClymont, Alistair and Fass, John
"Of Machines Learning to See Lemon,"
Available at: https://digitalcommons.wpi.edu/oa/vol8/iss2/8