Novel methods revolutionize digital storytelling
MeMAD project provides novel methods for efficient re-use and re-purpose of multilingual audiovisual content. These methodologies revolutionize video management and digital storytelling in broadcasting and media production. We go far beyond the state-of-the-art automatic video description methods by making the machine learn from the human. The resulting description is thus not only a time-aligned semantic extraction of objects but makes use of the audio and recognizes action sequences.
The project addresses the challenge of improving the discoverability and findability of audiovisual data, by developing novel methods for accessing and using the content. As a bonus, the methods developed for this automatic analysis, have a tremendous effect on the costs of the multimedia production processes: tasks that earlier have required hundreds of hours of human labour can be carried out with just hours of machine work.
The MeMAD project makes audiovisual content smarter and more appealing to users by interpreting the content and by providing supporting links to other media assets and external information sources. The media description and linking techniques developed by MeMAD use new and emerging technologies, such as deep and recurrent neural networks, machine learning, artificial intelligence and big data analysis. In other words, technologies capable of learning from humans, to make media smarter and more accessible for everybody.
Combining Automatic Efficiency with Human Accuracy
The state of the art in both fiction production and factual storytelling strongly relies on human interpretation. Generally, adding captions, descriptions, and links to related content is a slow and expensive process. By contrast, automatic description is cost-effective and produces consistent output. However, automatic methods are often unable to capture and describe the essential contents from a human perspective: the generated descriptions are correct, but not necessarily informative. Therefore, a new approach is needed in order to develop automatic methods to generate descriptions of longer action sequences in a fluent narrative.
In MeMAD project, the human description helps the machine to become a better analyst and describer. By combining the best available knowledge in machine processing, machine learning and human editing of verbal description, MeMAD project aims to industrialise the re-use and re-purposing of existing media and to revolutionise the process of digital storytelling. Does it mean that machines will steal our jobs? No, but they will enable us to do something extraordinary that we could not afford to do without them.