ETS is an organization that develops large-scale assessments for a range of different fields, including English. Probably their best-known assessment system internationally is the computerized Test of English as a Foreign Language or TOEFL iBT (Internet-Based Test).
Interestingly for DigiTala, ETS has long developed automated scoring procedures for both speaking and writing. Of particular interest for us is the SpeechRater engine that they have developed for the scoring of spoken English as a foreign language. Thanks to some DigiTala project members’ contacts with ETS researchers, we proposed collaboration to them during the project application stage and they agreed to become our partners.
As the first concrete step in DigiTala – ETS collaboration, four of us (Raili Hildén, Mikko Kuronen, Ari Huhta and Aku Rouhe) visited ETS at Princeton in late February 2020. Our hosts were Dr. Keelan Evanini and Dr. Klaus Zechner from the team that designs automated speech analysis and scoring procedures for English. We had several highly informative meetings with Klaus and Keelan as well as with their colleagues representing several other lines of work that underlie the TOEFL iBT and other ETS assessment systems ranging from conceptual design and test task development to psychometric analysis. The DigiTala project members in Finland attended most of the meetings virtually.
Despite the very intensive days of meetings with the ETS staff and (virtually) colleagues at home, the DigiTala visitors managed to find some time to stroll on the beautiful ETS campus and to visit downtown Princeton where the famous Princeton University is located.
ETS focuses on English whereas DigiTala covers Swedish and Finnish. There are, thus, clear differences in what the two groups of researchers do. However, we believe that we can learn a lot from the two decades of work on automated scoring of English carried out at ETS. Conversely, our research on automated speech recognition and scoring of our national languages is likely to be useful for ETS if they expand their repertoire of assessed languages in the future. One area of common interest is the development of feedback based on automated analysis of speech, which is under way at ETS and which is part of the deliverables of the DigiTala project.
Fun fact about Princeton: Princeton was the provisional capital of the United States for a few months in 1783, and Nassau Hall of the Princeton University served as its seat of government.