The MeMAD Webinars are online (to stay)

The MeMAD project is nearing completion. It is time to start looking at what we have accomplished.  In the beginning of this year, we held a series of three webinars, in which we presented the MeMAD results and insight collected during the project from three different viewpoints: accessibility, research and media production. In addition, we…

Read more

MeMAD featured at Yle’s blog on accessibility

Yle, the Finnish Broadcasting Company, has published a blog post about the experiments our project has done with the machine-translated subtitles. Four video examples used in our consumer evaluations are also embedded in the article, shedding extra light on the two previous blog posts here in the MeMAD blog. Read more (in Finnish): https://yle.fi/aihe/artikkeli/2021/01/14/korean-kriisi-ja-zombeja-tornadossa-ylen-kokemuksia-automaattisista  …

Read more

Viewer perspectives on machine-translated subtitles – Part II

Earlier this year, we published some early findings from focus groups where viewers discussed their reactions to machine-translated subtitles. In the past few months, we have conducted another two focus groups and a questionnaire study on the same topic. One of the focus groups consisted of Finnish-speakers who watched two Swedish-language clips subtitled in Finnish, and…

Read more

End-to-end named entity recognition for spoken Finnish

End-to-end named entity recognition for spoken Finnish Named entity recognition (NER) is a natural language processing task in which the system tries to find named entities and classify them in predefined categories, such as person, location, organization, date and product. NER is an integral part of other large natural language processing tasks, such as information…

Read more

MeMAD partners’ pilot projects in European Language Grid

MeMAD project partners Lingsoft and University of Helsinki have received funding in the first call for pilot projects of the European Language Grid (ELG). The project funding stage was fairly competitive: Ten projects out of 110 applications were funded, and two out of these ten are tightly linked with the MeMAD project! These two pilot projects will…

Read more

Innovation Radar recognizes MeMAD project research

  MeMAD project partners have received recognition for their accomplishments. Three project innovations were recognized by the European Commission’s Innovation Radar. Innovation Radar is a European Commission initiative that  champions potential innovations and innovators in EU-funded research and innovation framework programmes. The innovations are categorised in four different levels of market maturity.  Exploring: Innovations are actively exploring…

Read more

Viewer perspectives on machine-translated subtitles

One of MeMAD’s objectives has been to produce fully automated subtitles that are created from source language speech with automatic speech recognition, automatic segmentation and machine translation. We have previously conducted user tests where professional translators at the Finnish public broadcaster Yle post-edit machine-translated subtitles. The purpose of these tests is to see how automated subtitles…

Read more
Book cover - Innovation in Audio Description Research. Pale blue cover featuring a network of yellow and dark blue dots. Routledge.

Opening a New Chapter on Audio Description

At the end of July, MeMAD featured in a volume of ground-breaking research in the field of multimedia accessibility with the publication of Innovation in Audio Description Research (Routledge), an edited collection of projects pitched at the cutting edge of audiovisual research. In Comparing human and automated approaches to visual storytelling (Braun, Starr & Laaksonen),…

Read more

Huge contribution to the community – over 1000 freely available translation models

MeMAD translation models made in Helsinki are now integrated in Hugging Face transformers, the world’s most popular library for the workwith deep neural NLP models. They can be used by anyone and integrated in research, development and real-world applications making machine translation accessible without any dependence on commercial services and restricted terms and conditions. Language…

Read more

Deep learning for spoken language identification

Imagine that a tourist calls an emergency service speaking a foreign language. How to find a person that speaks the right language?  Or you have tons of multilingual television broadcasts in need of automatic translation or subtitling. Most current automatic speech recognition (ASR) and other language technology tools assume that the source language is known…

Read more

The MeMAD Knowledge Graph

In his book, “Weaving the Web”, Sir Tim Berners-Lee shared his vision about the future of his creation: “In an extreme view, the world can be seen as only connections, nothing else. We think of a dictionary as the repository of meaning, but it defines words only in terms of other words. I liked the…

Read more

Automatic metadata high up in the mountains

The European Broadcasting Union EBU’s annual Metadata Network MDN Workshop 2019 was organised in June 2019 in Geneva. It gathered 88 experts from different media companies, research institutes and vendor companies to share latest experiences and ideas related to metadata in the media business.  Thematically related to MeMAD, many broadcasters are already working on automatic…

Read more
Group picture of MediaEval participants.

MeMAD at MediaEval 2019 organized at EURECOM

The 2019 edition of MediaEval – a benchmark initiative offering challenges in multimedia retrieval, access and exploration – was hosted by EURECOM last October 2019. It directly followed the 27th ACM International Conference on Multimedia which also took place in Nice on the French Riviera. MeMAD reaffirmed its attachment to fostering multimodal research with audiovisual…

Read more
The members of the MeMad consortium posing in front of the University of Surrey

Putting the user in the center

In September, the MeMAD project gathered in Surrey for the fifth plenary meeting. As we’re about half-way through the project, we drilled down to the most central use cases and continued planning of the evaluation process in the final 18 months of MeMAD. With all of the pieces of the puzzle coming together, it was…

Read more
A photo showing the Conversation Assistant application in use.

Speech recognition for the deaf and hard of hearing

Try placing hands over your ears when following a conversation. The speech of other people immediately becomes muffled and hard to understand. You might pick some of the words, but you miss many others. Keeping up with the conversation quickly starts to feel tiring and frustrating as you try to guess the missing bits. You…

Read more
Three document icons on a black background.

Document-level Machine Translation

Typical machine translation is done on the sentence level. This means that sentences are translated in isolation without considering the flow in the surrounding context. From a human perspective, this is less than satisfactory. A natural text needs to be well connected, and the story it tells should be coherent and fluent. Any human translator…

Read more

MeMAD in Media for All conference

MeMAD project will have a strong presence in the Media for All conference in Stockholm next week. Sabine Braun and Kim Starr from University of Surrey will give a presentation titled “Comparing human and automated approaches to video description” on Wednesday, and Yle will have a large participation including Christoffer Forssell in the Wednesday morning…

Read more

Working together to bring value

During the last week of April, the MeMAD project team was back in Helsinki for the fourth plenary meeting. Before the plenary kicked off, members from Eurecom, Limecraft, Lingsoft, Aalto and INA got together on Wednesday to flesh out the principles for data handling and the use of RDF, based on an EBUCore-compatible model  created…

Read more

MeMAD use cases for validation at EBU MDN 2018

MeMAD draft use cases were circulated for comments and feedback at the annual EBU professional workshop for metadata developers network MDN. The first day of 2018 workshop was dedicated for demonstrations and projects on AI and media industry. Read more at https://tech.ebu.ch/mdn2018.

Read more

Tag the sound

What is audio tagging? If you are a frequent social media user, then you must be familiar with the word ‘tag’. This type of tagging normally happens when you find a funny meme or video that perfectly depicts your friends or matches their need. This process contains analyzing the content and assigning it to relevant…

Read more
A conference presenter standing by his research poster representing MeMAD.

To be or not to be… multimodal in MT

During the first year of our machine translation efforts in MeMAD, we focused on participating in two shared tasks on multimodal MT: the caption translation task at WMT 2018 and the speech translation task at IWSLT 2018. Starting with the good news, we can note that the MeMAD team won with a convincing victory over…

Read more

Unlocking copyrighted media archives for research

MeMAD research work – especially the parts analysing video and audio – depends on data and media because data intensive methods such as data mining are being used. MeMAD research teams of course use the standard open datasets such as TRECVID, but we have also worked to gain access and get rights to also use…

Read more

IMT: How to transform videos into words?

The H2020 MeMAD project ambitions to develop a set of technologies that will describe comprehensively videos, ranging from visual objects, people and motions that can be seen, to music, sounds and speech that can be hear and background knowledge and common sense that enable to understand the stories being told in those videos. In this…

Read more

Research network Digin interested about MeMAD

Lingsoft will present MeMAD project at the exhibition of Digitality and logic of Internet on May 21st in Turku, Finland. The event gathers together research around digitality and digital huamities. Especially logic of Internet will be discussed in many of the presentations.

Read more

MeMAD introduced in Tallinn

MeMAD project was introduced at Audiovisual translation – New trends in translation technology on May 17th in Tallinn, Estonia. Dr. Tiina Lindh-Knuutila from Lingsoft was invited to present how language technologies can enhance audiovisual translation process. Read more about the presentation here.

Read more

Lingsoft: Artificiell intelligens revolutionerar hanteringen av audiovisuellt material

  Pressmeddelande, fri för publicering 4.4.2018 Lingsoft är med i ett EU-projekt, som revolutionerar hanteringen av stora videosamlingar. Projektets syfte är att skapa nästa generations medieupplevelse genom att kombinera effektiviteten i artificiell intelligens med precisionen i mänskligt arbete. Det tre år långa EU-forskningsprojektet som inleddes i början av 2018, MeMAD, söker metoder för att nyttja…

Read more

Lingsoft: Tekoäly mullistaa audiovisuaalisen aineiston hallinnan

Tiedote, vapaa julkaistavaksi 27.3.2018 Lingsoft on mukana EU-hankkeessa, joka mullistaa suurten videokokoelmien käsittelyn. Hankkeen tarkoituksena on rakentaa seuraavan sukupolven mediakokemusta yhdistämällä tekoälyn tuoma tehokkuus ihmistyön tarkkuuteen. Vuoden 2018 alussa käynnistynyt kolmivuotinen EU-tutkimushanke MeMAD etsii menetelmiä tekoälyn hyödyntämiseen media-alalla, erityisesti videoaineiston kuvailussa, kääntämisessä ja tekstittämisessä. Lingsoft keskittyy hankkeessa erityisesti suomen- ja ruotsinkielisen puheentunnistuksen jatkokehittämiseen, puhujantunnistukseen ja aineistojen asiasanoitukseen.…

Read more

Aalto-yliopisto: Tutkijat opettavat tekoälyn kuvailemaan videoita – EU-hankkeessa kehitettävä menetelmä voi mullistaa suurten videokokoelmien käsittelyn

Uusi menetelmä yhdistää tekstikuvailuun sekä videolla kuuluvan puheen että kuvan sisällön ja tuottaa tekstin suoraan halutulle kielelle. Tammikuussa 2018 alkaneessa EU:n rahoittamassa projektissa kehitetään Aalto-yliopiston uusimpiin kuvan- ja äänenkäsittelyn tutkimustuloksiin perustuva menetelmä audiovisuaalisen aineiston kuvailuun ja luokitteluun. Projektin yhtenä opetusaineistona käytetään Yleltä saatuja videomuotoisia arkistomateriaaleja. “Menetelmän avulla saadaan automaattisesti tekstikuvaus videon tapahtumista ja puheesta. Vertailemalla…

Read more

Welcome to the MeMAD Kick-off Seminar – Methods for managing audiovisual data: combining automatic efficiency with human accuracy

The seminar presents MeMAD (Methods for managing audiovisual data: combining automatic efficiency with human accuracy), a new three year research and innovation project to develop efficient and accurate automatic language-based methods for managing, accessing and publishing video content. These methods will benefit in particular the creative industries, such as television, cinema and on-demand services, as they tackle with the increasing audiovisual big data. 

Read more