Methods for Managing Audiovisual Data: Combining Automatic Efficiency with Human Accuracy

An EU funded Horizon2020 research project (2018-2020)

In the MeMAD project, an international consortium of research and business partners develops methods of accessing audiovisual content through language (video description, subtitling, and linked data). The aim is to combine human and machine translation efforts to create an improved model of information extraction, retrieval, and search. The model is capable of creating descriptions that are not only time-aligned semantic extractions of objects but also make use of the audio and recognize action sequences. Such methodology can revolutionize video management and digital storytelling in broadcasting and media production, and help dealing with the growing amount of audiovisual big data. In addition, MeMAD investigates whether these methods can be adapted to semi-automatic audio description and subtitling to improve accessibility.

Source of image: The National Archives of Finland, R. H. Rehbinderin arkisto, Johan Albert Ehrenström 1821-1826 (Ba:9) (113).

Assistant Professor Simo Määttä, M.A. Merja Nivala, and Professor Emeritus Juhani Härmä

Funding: Finnish Cultural Foundation

In this project, a selection of the correspondence between Johan Albrecht Ehrenström and his colleagues Carl Johan Walleen and Robert Henrik Rehbinder is translated from French and Swedish into Finnish. These statesmen where influential public figures at the beginning of the Russian rule in Finland, during the first decades of the 19th century. Mr. Ehrenström lived and worked in Helsinki, whereas Mr. Walleen and Mr. Rehbinder lived and worked in Saint Petersburg. The letters are stored in the National Archives of Finland.

The correspondence deals mainly with the construction of a new administrative center of Helsinki, capital of Finland since 1812, as well as the relationship between the Czar and the Grand Duchy of Finland and other current and private affairs. Most letters are written in French, which was the lingua franca of the elites in the early 19th century. French was also the language of communication between the Swedish-speaking administration of the Grand Duchy of Finland and the Czar’s Russian-speaking administration. Many letters are bilingual, for Swedish is used as well. It is important to translate these letters into Finnish, so that Finnish researchers can use them more widely.

The project starts by a thematic classification of Ehrenström’s letters. Subsequently, a selection will be transcribed, translated into Finnish, and published. The project is coordinated by Assistant Professor Simo Määttä. M.A. Merja Nivala is responsible for cataloguing, transcribing, and translating the letters. Counselling is provided by Professor Emeritus Juhani Härmä, who has analysed letters belonging to this collection in his research.

Mul­ti­mo­dal Trans­la­tion with the Blind

A postdoctoral research project at University of Helsinki, funded by the Academy of Finland

Principal investigator: Maija Hirvonen

Project duration: 2017-2019

The MUTABLE project studies the interactive translation process by analyzing team audio description in which sighted and blind describers work on an audio description face-to-face. Specific topics of interest in this project consist of the interface between cognition and interaction, the use of multimodal resources while translating, and the organisation of work and expertise in perceptually asymmetrical teams. The main research data contain video recordings of authentic audio description processes in Austria, Finland, and Germany.

The project highlights the role of interaction and users in the translation process. The results will increase knowledge of translating as a multimodal and interactive process and of interaction between sighted and non-sighted persons. In addition, the results will help to identify visually-impaired persons’ expertise in relation to translation and multimodal communication. This new knowledge can be applied to foster inclusion in the society as well as diversity in the workplace.

Read more