A computer, which technique is based on a novel brain-computer interface, is able to produce entirely new information, such as fictional images that were never before seen.
Researchers at the University of Helsinki have developed a technique in which a computer models visual perception by monitoring human brain signals. In a way, it is as if the computer tries to imagine what a human is thinking about. As a result of this imagining, the computer is able to produce entirely new information, such as fictional images that were never before seen.
The technique is based on a novel brain-computer interface. Previously, similar brain-computer interfaces have been able to perform one-way communication from brain to computer, such as spell individual letters or move a cursor.
As far as is known, the new study is the first where both the computer’s presentation of the information and brain signals were modelled simultaneously using methods. Images that matched the visual characteristics that participants were focusing on were generated through interaction between human brain responses and a generative neural network.
The study was published in the Scientific Reports journal in September. Scientific Reports is an online multidisciplinary, open-access journal from the publishers of Nature.
Neuroadaptive generative modelling
The researchers call this method neuroadaptive generative modelling. A total of 31 volunteers participated in a study that evaluated the effectiveness of the technique. Participants were shown hundreds of -generated images of diverse-looking people while their EEG was recorded.
The subjects were asked to concentrate on certain features, such as faces that looked old or were smiling. While looking at a rapidly presented series of face images, the EEGs of the subjects were fed to a neural network, which inferred whether any image was detected by the brain as matching what the subjects were looking for.
Based on this information, the neural network adapted its estimation as to what kind of faces people were thinking of. Finally, the images generated by the computer were evaluated by the participants and they nearly perfectly matched with the features the participants were thinking of. The accuracy of the experiment was 83 per cent.
“The technique combines natural human responses with the computer’s ability to create new information. In the experiment, the participants were only asked to look at the computer-generated images. The computer, in turn, modelled the images displayed and the human reaction toward the images by using human brain responses. From this, the computer can create an entirely new image that matches the user’s intention,” says Tuukka Ruotsalo, Academy of Finland Research Fellow at the University of Helsinki, Finland and Associate Professor at the University of Copenhagen, Denmark. […]