In this profile series, we interview innovators on the front-lines – those who have dedicated their life’s work to improving the human condition through technology advancements. This time, meet Damian Borth.
Damian Borth, chair in the Artificial Intelligence & Machine Learning department at the University of St. Gallen (HSG) in Switzerland, and past director of the Deep Learning Competence Center at the German Research Center for Artificial Intelligence (DFKI). He is also a founding co-director of Sociovestix Labs, a social enterprise in the area of financial data science. Damian’s background is in research where he focuses on large-scale multimedia opinion mining applying and in particular
Damian talks about his realization in
What has your journey been like in
I spent two years in Taiwan, went to the University of Kaiserslautern, Germany for my PhD while having a stopover at Columbia University, and did my post-doctoral at UC Berkeley and the International Computer Science Institute in Berkeley. In Berkeley, I spent my time on
What made you become a
I was actually a “non-believer” in
What did you do in Berkeley?
I continued the work we have started at Columbia in sentiment analysis for pictures. It could classify objects like e.g. animals such as a dog or a cat. We attached adjectives to the noun and made the analysis differentiate between a scary dog or a cute dog. The vocabulary was roughly 2,000 adjectives noun pairs (ANP). By conditioning the noun with an adjective, we were able to move a very objective judgement to a subjective assessment. Doing so we were able to derive a link from this mid-level representation to a higher level of sentiment representation. The positive image of a cute dog or a laughing baby could flip to a negative sentiment when it saw a dark street or a bloody accident. This mid-level representation proved to be also very successful beyond sentiment analysis and was applied to aesthetics and emotion detection. It created a bridge between the objective world and the subjective world of visual content. In Berkeley I was also part of the team creating the YFCC100m dataset the largest curated image dataset at that time. Having such a dataset with 100 million creative common images and videos from Flickr helps if you want to train a very deep neural network architecture.
Did you continue your sentiment analysis work with DFKI?
We call it Multimedia Opinion Mining (MOM), because we want it to consider different modalities such as video and audio. Currently we’re extending
Can you elaborate on the disaster response case? How can your work help these first responders?
We were analyzing data collected from a wildfire case at Fort McMurray. When we looked at the data, initially we saw that the area around the fire, in particular the vegetation and already burned area was a strong indicator for the direction of the fire spread. Once the wind changed the fire changed its course as well which caused more damage. This analysis would have predicted that change of how the fire develops much earlier. Such information is very valuable to the first responders and their work on the ground. Another case we’re currently working on is with flooding. We started a benchmark challenge to foster collaboration to build up a community with MediaEval Satellite Task. In the first year 16 teams from around the world have been participating. The teams submit their neural networks results and we compare the performance on the test data set to figure out which one provides the best predictions. This way we know very quickly which approaches work and which not.
0 Comments