The MeMAD project is nearing completion. It is time to start looking at what we have accomplished. In the beginning of this year, we held a series of three webinars, in which we presented the MeMAD results and insight collected during the project from three different viewpoints: accessibility, research and media production. In addition, we…
Yle, the Finnish Broadcasting Company, has published a blog post about the experiments our project has done with the machine-translated subtitles. Four video examples used in our consumer evaluations are also embedded in the article, shedding extra light on the two previous blog posts here in the MeMAD blog. Read more (in Finnish): https://yle.fi/aihe/artikkeli/2021/01/14/korean-kriisi-ja-zombeja-tornadossa-ylen-kokemuksia-automaattisista …
The beginning of 2021 will bring a lot of interesting activities to tune in online! The MeMAD project will host three webinars to present the final results and findings of the project. The MeMAD presence will begin by co-hosting an event on Accessibility with the Goethe-Institut Finnland as a part of Night of Science today!…
Earlier this year, we published some early findings from focus groups where viewers discussed their reactions to machine-translated subtitles. In the past few months, we have conducted another two focus groups and a questionnaire study on the same topic. One of the focus groups consisted of Finnish-speakers who watched two Swedish-language clips subtitled in Finnish, and…
End-to-end named entity recognition for spoken Finnish Named entity recognition (NER) is a natural language processing task in which the system tries to find named entities and classify them in predefined categories, such as person, location, organization, date and product. NER is an integral part of other large natural language processing tasks, such as information…
MeMAD project partners Lingsoft and University of Helsinki have received funding in the first call for pilot projects of the European Language Grid (ELG). The project funding stage was fairly competitive: Ten projects out of 110 applications were funded, and two out of these ten are tightly linked with the MeMAD project! These two pilot projects will…
MeMAD project partners have received recognition for their accomplishments. Three project innovations were recognized by the European Commission’s Innovation Radar. Innovation Radar is a European Commission initiative that champions potential innovations and innovators in EU-funded research and innovation framework programmes. The innovations are categorised in four different levels of market maturity. Exploring: Innovations are actively exploring…
One of MeMAD’s objectives has been to produce fully automated subtitles that are created from source language speech with automatic speech recognition, automatic segmentation and machine translation. We have previously conducted user tests where professional translators at the Finnish public broadcaster Yle post-edit machine-translated subtitles. The purpose of these tests is to see how automated subtitles…
At the end of July, MeMAD featured in a volume of ground-breaking research in the field of multimedia accessibility with the publication of Innovation in Audio Description Research (Routledge), an edited collection of projects pitched at the cutting edge of audiovisual research. In Comparing human and automated approaches to visual storytelling (Braun, Starr & Laaksonen),…
Maaseudun Tulevaisuus published two articles about machine translation and the MeMAD project, interviewing our researcher Maarit Koponen from the University of Helsinki Read more (in Finnish): Machine can translate and interpret even voice in the future: “It can be fairly useful for a tourist soon”, says a researcher. Do you know how Google Translate works…
MeMAD translation models made in Helsinki are now integrated in Hugging Face transformers, the world’s most popular library for the workwith deep neural NLP models. They can be used by anyone and integrated in research, development and real-world applications making machine translation accessible without any dependence on commercial services and restricted terms and conditions. Language…
Imagine that a tourist calls an emergency service speaking a foreign language. How to find a person that speaks the right language? Or you have tons of multilingual television broadcasts in need of automatic translation or subtitling. Most current automatic speech recognition (ASR) and other language technology tools assume that the source language is known…
INA hosted our 6th in plenary meeting in Paris in the beginning of February. Together, we hashed out the final stages of development of our multimodal solutions and the final evaluation. We have made our open source contributions available at the MeMAD GitHub repository. If you haven’t checked them, be sure to follow the development. …
In his book, “Weaving the Web”, Sir Tim Berners-Lee shared his vision about the future of his creation: “In an extreme view, the world can be seen as only connections, nothing else. We think of a dictionary as the repository of meaning, but it defines words only in terms of other words. I liked the…
At the end of the second year of the MeMAD project, different aspects of the prototype were chosen for evaluation, based on product maturity. There were four tracks of evaluation: video editing assistance, searching, intralingual subtitling with the help of automatic speech recognition (ASR) and interlingual subtitling with the help of machine translation (MT). This…
One important part of the MeMAD project is working with potential users of the tools developed, as well as presenting the research and development work to professional communities and wider public audiences. In November and early December of 2019, we carried out user evaluation with professional translators to find out what they thought about using…
Imagine that someone has recorded a series of hundreds of meetings and your task is to find out who spoke where and when. Sounds like a lot of transcription work, especially if the speaker turns are short, spontaneous and overlapping and sometimes hard to understand even after careful listening. The automatic approach to this task…
The European Broadcasting Union EBU’s annual Metadata Network MDN Workshop 2019 was organised in June 2019 in Geneva. It gathered 88 experts from different media companies, research institutes and vendor companies to share latest experiences and ideas related to metadata in the media business. Thematically related to MeMAD, many broadcasters are already working on automatic…
The 2019 edition of MediaEval – a benchmark initiative offering challenges in multimedia retrieval, access and exploration – was hosted by EURECOM last October 2019. It directly followed the 27th ACM International Conference on Multimedia which also took place in Nice on the French Riviera. MeMAD reaffirmed its attachment to fostering multimodal research with audiovisual…
In September, the MeMAD project gathered in Surrey for the fifth plenary meeting. As we’re about half-way through the project, we drilled down to the most central use cases and continued planning of the evaluation process in the final 18 months of MeMAD. With all of the pieces of the puzzle coming together, it was…
MeMAD project MeMAD project aims at designing cutting-edge audiovisual media indexing methods. This work will meet the need for automatic procedures required for the management of large audiovisual collections, as well as media accessibility for the visually and hearing impaired. INA contributed to the MeMAD platform with a software module dedicated to audio analysis: music…
We’ve reached the mid-way point of the MeMAD project, and during that time we have developed plenty of exciting state-of-the-art and competition-winning stuff to show off: If you haven’t already, we encourage you to check out the MeMAD Github repository! Some of the cool stuff you can find there include: Audio tagging Deep captioning Face…
Try placing hands over your ears when following a conversation. The speech of other people immediately becomes muffled and hard to understand. You might pick some of the words, but you miss many others. Keeping up with the conversation quickly starts to feel tiring and frustrating as you try to guess the missing bits. You…
Typical machine translation is done on the sentence level. This means that sentences are translated in isolation without considering the flow in the surrounding context. From a human perspective, this is less than satisfactory. A natural text needs to be well connected, and the story it tells should be coherent and fluent. Any human translator…
MeMAD project will have a strong presence in the Media for All conference in Stockholm next week. Sabine Braun and Kim Starr from University of Surrey will give a presentation titled “Comparing human and automated approaches to video description” on Wednesday, and Yle will have a large participation including Christoffer Forssell in the Wednesday morning…
During the last week of April, the MeMAD project team was back in Helsinki for the fourth plenary meeting. Before the plenary kicked off, members from Eurecom, Limecraft, Lingsoft, Aalto and INA got together on Wednesday to flesh out the principles for data handling and the use of RDF, based on an EBUCore-compatible model created…
MeMAD work done on datasets and data licensing for research was presented at two conferences during late 2018. The Baltic audiovisual archives community gathered at the annual BAAC conference in Tallinn. Read the best parts in our separate blog post by Lauri Saarikoski. Same topics were discussed also at the Helsinki Digital Humanities Summit in…
MeMAD draft use cases were circulated for comments and feedback at the annual EBU professional workshop for metadata developers network MDN. The first day of 2018 workshop was dedicated for demonstrations and projects on AI and media industry. Read more at https://tech.ebu.ch/mdn2018.
What is audio tagging? If you are a frequent social media user, then you must be familiar with the word ‘tag’. This type of tagging normally happens when you find a funny meme or video that perfectly depicts your friends or matches their need. This process contains analyzing the content and assigning it to relevant…
During the first year of our machine translation efforts in MeMAD, we focused on participating in two shared tasks on multimodal MT: the caption translation task at WMT 2018 and the speech translation task at IWSLT 2018. Starting with the good news, we can note that the MeMAD team won with a convincing victory over…
“Wow, this is the future”, I thought to myself. “How do I get to be a part of this project”, my thoughts continued. The demo I attended in August led me to apply for this job a month later. Now I have been the Yle Industry Partner for MeMAD project for almost two months. Normally…
MeMAD research work – especially the parts analysing video and audio – depends on data and media because data intensive methods such as data mining are being used. MeMAD research teams of course use the standard open datasets such as TRECVID, but we have also worked to gain access and get rights to also use…
Exploring a city as vibrant as Berlin, an urban landscape punctuated by music from booming sound systems and pulsating with imagery projected on every wall and façade during the current Festival of Lights, sharpens the mind to the relevance of accessible media in all our lives. This dynamic geographical space, with its iconic audiovisual heritage,…
MeMAD project will have presence in IBC2018, the world’s most influential media conference by consortium members INA, Limecraft and Yle. Remember to look us up to find out more about the MeMAD project!
The H2020 MeMAD project ambitions to develop a set of technologies that will describe comprehensively videos, ranging from visual objects, people and motions that can be seen, to music, sounds and speech that can be hear and background knowledge and common sense that enable to understand the stories being told in those videos. In this…
Lingsoft will present MeMAD project at the exhibition of Digitality and logic of Internet on May 21st in Turku, Finland. The event gathers together research around digitality and digital huamities. Especially logic of Internet will be discussed in many of the presentations.
MeMAD project was introduced at Audiovisual translation – New trends in translation technology on May 17th in Tallinn, Estonia. Dr. Tiina Lindh-Knuutila from Lingsoft was invited to present how language technologies can enhance audiovisual translation process. Read more about the presentation here.
The MeMAD project launched with a vibrant discussion and exchange of ideas with its stakeholders at the beginning of 2018. We wanted to organise a kick-off event with all the central stakeholders as they play a key role in shaping the MeMAD project and its outcomes. In this post, we review our stakeholders’ wishlist about…
MeMAD researcher Maija Hirvonen (University of Helsinki) analyzes what happens to our ability to read when videos are taking over in our everyday communication. Read more here (only in Finnish).
Press release, free for publication 4 March 2018 Lingsoft is part of an EU project that is set to revolutionize the management of large video collections. The project aims to create a next-generation media experience by combining the efficiency of artificial intelligence with human accuracy. MeMAD, a three-year EU research project that began in 2018,…
Pressmeddelande, fri för publicering 4.4.2018 Lingsoft är med i ett EU-projekt, som revolutionerar hanteringen av stora videosamlingar. Projektets syfte är att skapa nästa generations medieupplevelse genom att kombinera effektiviteten i artificiell intelligens med precisionen i mänskligt arbete. Det tre år långa EU-forskningsprojektet som inleddes i början av 2018, MeMAD, söker metoder för att nyttja…
Tiedote, vapaa julkaistavaksi 27.3.2018 Lingsoft on mukana EU-hankkeessa, joka mullistaa suurten videokokoelmien käsittelyn. Hankkeen tarkoituksena on rakentaa seuraavan sukupolven mediakokemusta yhdistämällä tekoälyn tuoma tehokkuus ihmistyön tarkkuuteen. Vuoden 2018 alussa käynnistynyt kolmivuotinen EU-tutkimushanke MeMAD etsii menetelmiä tekoälyn hyödyntämiseen media-alalla, erityisesti videoaineiston kuvailussa, kääntämisessä ja tekstittämisessä. Lingsoft keskittyy hankkeessa erityisesti suomen- ja ruotsinkielisen puheentunnistuksen jatkokehittämiseen, puhujantunnistukseen ja aineistojen asiasanoitukseen.…
MeMAD researcher Maija Hirvonen (University of Helsinki) was interviewed by a Finnish newspaper Turun Sanomat about what happens to our ability to read when videos are taking over in our everyday communication. Read more here (only in Finnish).
Transforming audiovisual information into words will revolutionize handling of huge media archives when artificial intelligence is taught how to describe videos. Read more here (only in Finnish).
Uusi menetelmä yhdistää tekstikuvailuun sekä videolla kuuluvan puheen että kuvan sisällön ja tuottaa tekstin suoraan halutulle kielelle. Tammikuussa 2018 alkaneessa EU:n rahoittamassa projektissa kehitetään Aalto-yliopiston uusimpiin kuvan- ja äänenkäsittelyn tutkimustuloksiin perustuva menetelmä audiovisuaalisen aineiston kuvailuun ja luokitteluun. Projektin yhtenä opetusaineistona käytetään Yleltä saatuja videomuotoisia arkistomateriaaleja. “Menetelmän avulla saadaan automaattisesti tekstikuvaus videon tapahtumista ja puheesta. Vertailemalla…
Yle’s newsletter about how methods developed in EU project can revolutionize handling of audiovisual archives. Read more here (only in Finnish).
In future, will artificial intelligence watch videos and listen to podcasts for you? Will it serve you the best clips or translate videos to languages that you understand? Will machines propose movie editors raw versions of different cuts or wade through all the media and preprocess it for journalists? Read the whole article here (only…
WELCOME TO THE MEMAD KICK-OFF SEMINAR – METHODS FOR MANAGING AUDIOVISUAL DATA: COMBINING AUTOMATIC EFFICIENCY WITH HUMAN ACCURACY The materials for the seminar can be found here.
The seminar presents MeMAD (Methods for managing audiovisual data: combining automatic efficiency with human accuracy), a new three year research and innovation project to develop efficient and accurate automatic language-based methods for managing, accessing and publishing video content. These methods will benefit in particular the creative industries, such as television, cinema and on-demand services, as they tackle with the increasing audiovisual big data.