Who are you?
My name is Tuomo Hiippala and I am Assistant Professor of English Language and Digital Humanities in the Department of Languages at the University of Helsinki since 2018.
What is your research topic?
My research focuses on multimodality, that is, the way human communication relies on appropriate combinations of expressive resources. The phenomenon of multimodality is increasingly acknowledged as an inherent feature of human communication. To exemplify, face-to-face interaction involves constant coordination of spoken language, gestures, gaze and posture, whereas page-based documents regularly use combinations written language, photographs, diagrams and layout to communicate with their reader.
Just which expressive resources are combined and how depends largely on the communicative situation in question. Humans engage in a wide range of communicative situations every day and negotiate them largely without problems. I am interested in the underlying principles of multimodal communication that help humans navigate this diversity. This understanding, however, is not likely to be achieved without empirical research, which is currently slowed down by the lack of large corpora with rich annotations. For this reason, I am especially interested in developing computational approaches that would enable conducting empirical research on multimodality at scale.
How is your research related to Kielipankki?
I have published two multimodal corpora that are distributed through Kielipankki: one related to my doctoral dissertation (GeM-HTB) and another created as a part of my recent research project (AI2D-RST). I find the service to be extremely useful for long-term storage and easy access to the data, and plan to continue sharing multimodal corpora created in future projects.
Publications related to the resources:
Tuomo Hiippala (2016) Helsingin kaupungin matkailuesitteiden multimodaalinen korpus. Terra 128(2): 75-85.
Tuomo Hiippala, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Serafina Orekhova, Aino Tuomainen, Matthew Stone, John A. Bateman (2020) AI2D-RST: A multimodal corpus of 1000 primary school science diagrams.
The FIN-CLARIN consortium consists of a group of Finnish universities along with CSC – IT Center for Science and the Institute for the Languages of Finland (Kotus). FIN-CLARIN helps the researchers in Finland to use, to refine, to preserve and to share their language resources. The Language Bank of Finland is the collection of services that provides the language materials and tools for the research community.