Researchers have developed a software capable of recognising sketches with an accuracy of 74.9 per cent leaving behind real human counterparts who only managed to identify sketches with a success rate of 73.1 per cent.
Dubbed Sketch-a-Net, the program is a ‘deep neural network’ based software designed to emulate the processing of the human brain. Designers made the software such that it could recognise the order of the strokes – something that was previously ignored but is important for understanding drawings on touchscreens.
Researchers say that their algorithm could become more relevant as more and more devices start getting touchscreens and could pave way for new ways to interact with computers.
Touch panels have been around ever since designing began on computers and touchscreens could understand what you are drawing enabling you to retrieve a specific image by drawing it with your fingers, which is more natural than keyword searches for finding items such as furniture or fashion accessories.
Researchers say that their algorithm could also aid police forensics when an artist’s impression of a criminal needs to be matched to a mugshot or CCTV database.
The research, which was accepted at the British Machine Vision Conference, also showed that the program performed better at determining finer details in sketches. For example, it was able to successfully distinguish the specific bird variants ‘seagull’, ‘flying-bird’, ‘standing-bird’ and ‘pigeon’ with 42.5 per cent accuracy compared to humans that only achieved 24.8 per cent.
Sketches are very intuitive to humans and have been used as a communication tool for thousands of years but recognising free-hand sketches is challenging because they are abstract, varied and consist of black and white lines rather than coloured pixels like a photo. Solving sketch recognition will lead to a greater scientific understanding of visual perception.
Timothy Hospedales, co-author of the study and Lecturer in the School of Electronic Engineering and Computer Science, QMUL, said: “It’s exciting that our computer program can solve the task even better than humans can. Sketches are an interesting area to study because they have been used since pre-historic times for communication and now, with the increase in use of touchscreens, they are becoming a much more common communication tool again. This could really have a huge impact for areas such as police forensics, touchscreen use and image retrieval, and ultimately will help us get to the bottom of visual understanding.”
