• Facebook
  • LinkedIn
  • Blogs
  • Inloggen
  • Home
  • Schrijfsels
    • Blogs
    • Populaire
    • Wetenschappelijk
    • Drenten
  • In de media
    • AV
    • Web
    • Krant
    • Lezingen
    • AV-Lezingen
    • Televisie
  • Werk
    • TST & AI
      • Spraakherkenning
      • AI
      • Showcases
      • Subtitles
      • LIPS
      • NTU
      • UvN
      • Maastricht
      • Diplomaten
      • TTS
    • Affiliaties
      • UTwente
      • UU
      • Telecats
      • Vorige wergevers
    • Netwerken
      • NOTaS
      • CLST
      • Levende Herinneringen
    • Infrastructuur
      • CLARIAH
      • CLARIN-NL
      • CLARIN-EU
      • DARIAH
      • CHAT
      • LISTEN
      • Verteld Verleden
    • Projecten
      • Voltooid
        • TTNWW
        • Radio Oranje
        • Buchenwald
        • Bosnian Memories
        • Croatian Memories
      • Lopend
      • Demonstratie
        • Gemeenten
        • Oratie
        • Preek
        • Klimaatverandering
        • Café Weltschmerz
        • Debat Gemist
    • Programming
    • Software
    • Over mij
    • LOT 2023
      • Background
      • Course
      • Assignments
      • AV-files
  • Persoonlijk
    • Arjan
      • Arjan
      • Ouders
      • Zus & Broer
      • Neven & Nicht
      • Grootouders
    • Brigitte
      • Brigitte
      • Ouders
      • Broer & Zus
      • Nicht & Neef
      • Grootouders
      • Operatie
    • Drentsche Patrijshonden
      • Pepper
      • Moos
    • Huizen
      • Samen
        • Italië
        • Burg. Reiger (2010 - ...)
        • Steve Biko (2009 - 2010)
        • Baarnseweg (2007 - 2009)
        • Poortstraat (1994 - 2007)
      • Arjan
        • Poortstraat (1977 -1994)
        • Sweelincklaan (1972 -1977)
        • Soestdijkseweg (1965 -1972)
        • Biltzigt (1958 -1965)
      • Brigitte
        • Thijssenlaan (1985 - 1994)
        • Braamstraat (1981 - 1985)
        • Voorstraat (1980 -1981)
        • Hopakker (1980 -1980)
        • Kemperstraat (1979 -1979)
        • Verwerstraat (1968 - 1979)
        • Oude Raadhuisstraat (1961 - 1968)
        • Drostlaan (1960 - 1961)
    • Rolanda
      • Levensverhaal
      • Rolanda 85
      • Afnemende Gezondheid
      • Begrafenis
      • Foto's
        • 85 Jaar
        • Sta-op stoel
  • Interessant
    • Zipf's Law
    • Conversatie Regels
    • Toon PDF
  • Extra Activiteit
  • WebServices
    • LabelMaker
    • IetsAnders

blogs

A CLARIN Transcription Portal for Interview Data

A CLARIN Transcription Portal for Interview Data

06 maart 2020

Christoph Draxler, Henk van den Heuvel, Arjan van Hessen, Silvia Calamai, Louise Corti, Stefania Scagliola Abstract In this paper we present a first version of a transcription portal for audio...

Cross disciplinary overtures with interview data

Cross disciplinary overtures with interview data

29 november 2018

Integrating digital practices and tools in the scholarly workflow Summary As increasingly sophisticated new technologies come on stream, there is one type of data that begs to be explored by the...

A Transcription Portal for Oral History Research and Beyond

A Transcription Portal for Oral History Research and Beyond

28 november 2018

Background and Introduction Over the past 2 years a number of researchers from various backgrounds have been working on the exploitation of digital techniques and tools for working with oral...

Speech Recognition and Scholarly Research: Usability and Sustainability

Speech Recognition and Scholarly Research: Usability and Sustainability

05 oktober 2018

Roeland Ordelman and Arjan van HessenNetherlands Institute for Sound and Vision and University of Twente, The Netherlands Objectives In spite of significant efforts and progress, automatic speech...

Evaluation of the OH-portal

Evaluation of the OH-portal

04 oktober 2018

During the successful and enjoyable workshop in Arezzo (May 2017), it became clear that, if done properly, automatic transcription of interviews could be useful to get a quick overview of what was...

In response to your inquiry Automatic e-mail answer suggestion in a Dutch Contact Centre

In response to your inquiry Automatic e-mail answer suggestion in a Dutch Contact Centre

09 oktober 2015

M.R. Boedeltje and A.J. van Hessen Telecats and Telecats/University of Twente Abstract In the past years, the number of service requests through e-mail has shown an explosive growth. Equal to...

Utterance generation for transaction dialogues

Utterance generation for transaction dialogues

19 juli 2015

Joris Hulstijn and Arjan van Hessen University of Twente The Netherlands { joris | hessen }@cs.utwente.nl   1. Transaction Dialogues Obligations Transactions (ticket reservation,...

Croatian Memories: an interview collection with personal narratives on war and trauma

Croatian Memories: an interview collection with personal narratives on war and trauma

17 juli 2015

  Arjan van Hessen1, Franciska de Jong1, 2, and Stef Scagliola2 1 Human Media Interaction Group, Universiteit Twente, The Netherlands {a.j.vanhessen, f.m.g.dejong}@utwente.nl 2 Erasmus...

Evaluation of the OH-portal

During the successful and enjoyable workshop in Arezzo (May 2017), it became clear that, if done properly, automatic transcription of interviews could be useful to get a quick overview of what was said in the interviews. The participants in Arezzo were aware of the imperfections of automatic speech recognition (ASR) and knew that the recognition results will decrease when audio quality is low, when there is background noise, or the speech spoken in (heavy) dialect.

What was explicitly asked in Arezzo, was to keep the portal to be developed, as simple as possible by making no or just a few demands on the audio input, having clear instructions and using as little technical jargon as possible.

After the request for making the portal was approved in autumn 2017, the team of Christoph Draxler at the LMU in München started to build the first version of the OH-portal. An upgraded beta version (1.0.0) of the portal was presented to participants of the 2018 workshop in München.

The portal

Afbeelding1Fig. 1: Screenshot of the portal

 

The idea behind the OH-portal is simple. You go to the website (https://www.phonetik.uni-muenchen.de/apps/oh-portal), select one or more sound files, upload them and download the automatically generated transcription. Currently, audio-files must be formatted as wav files, but in the near future the portal itself will transform a range of submitted audio-files into the correct wav format. What is already possible is that it does not matter with which sample frequency the files are recorded or whether they are mono or stereo. In case of stereo, the portal asks the user whether he or she wants to process both audio-channels separately or together (i.e. added to one signal). If the user chooses for separately, both channels are done one after the other so that when you have recorded the interview with 2 speakers each on a single channel, you can better separate the different speakers, determine turn-taking and get even a better recognition result.

Workflow

Within the portal, the  knop1 button to select the wav files recognised from a user’s own computer. Then click on the knop1 button whereafter a selection window will open (see below) where you can set the different options.

Afbeelding2Fig. 2: verify-window. Select the language and eventually the ASR-engine for that language. Moreover,
select the various option you want to do after the recognition such as “manual transcription/correction and thereafter word alignment.

 

At this moment, the choices made in the "Verify Files window", are valid for all the selected files: so, you cannot select different languages or recognisers when you select more than one audio-file.
Once the choices have been made, you can start the process via the button.

The audio-files are uploaded and then processed. As said, if a stereo file is included, you will be asked how you want to process the stereo file.

Manual Transcription

It is widely recognised that speech recognition hardly ever works flawlessly. Depending on the quality of the recordings, the way of speaking and the use of words/jargon by the various speakers, their accents and the presence of background noise, speech recognition will be more or less successful. With good recordings, clear, coherent speech, an error rate of less than 10% is possible for the four languages in the current portal (En, Nl, It, and De).

But even with very good recognition, something can go wrong. The Manual Transcription button offers the possibility to make corrections in the recognition results. However, by editing the recognized text, the connection between the recognized word and the time of the spoken words in the audio-file is broken. After the automatically obtained transcription has been corrected manually, you can restore this connection by choosing Word alignment. The ASR-engine will redo the job, but this time it knows exactly what was said. The result is now a perfect transcription where from every word it is exactly known when it was pronounced. This offers the possibility to automatically generate subtitles and make a karaoke version where the pronounced word is highlighted when played.

Webservice

During the first day of the 2018 workshop, the way automatic speech recognition works, the choices made and the problems that appeared when building the portal, were explained to the audience. The current OH-portal is a web service that "collects" the audio files and then, depending on the choices made, forwards the files to the different speech recognizers (WebASR in Sheffield, LST-NL in Nijmegen, LST-En in Enschede, EML-D/It in Germany).

Each recognizer then returns its output in a particular output format. So, to get a uniform output result, the results must be re-written by the OH-portal to one of the selected standards. When additional languages are added in the near future, this rewriting process has to be done over and over again.

Commercial versus Open Source

There are many more recognizers available (and also for more languages than the current 4), but they are almost all (semi-) commercial. It is very easy to connect the excellent working Google recognizer and in the beta versions of the OH-portal this was done. But there is a price to pay.

Paying with money is usually not a problem because it is nearly almost just a few euros per hour of speech. But almost always the audio data used is stored on the discs of the commercial parties for extra training, testing or something else. And that is often a problem because in many interviews the content is sensitive and likely subject to the GDPR. Even in our situation with a (reliable) portal - where all user’s data are removed 24 hours after they have been processed - it may be a problem because collection-owners may expressly state that the data cannot leave the "building" without permission, or have not yet put in place GDPR-compliant processing agreements.
As a safety measure, for use of the OH-portal during the workshop it was therefore decided to remove the commercial recognizers as an option of choice; and open collections were used as far as possible for testing purposes.

At the first evaluation on Wednesday afternoon, however, participants questioned whether it might be useful to restore this “commercial” option, and to explicitly indicate that the recognizers X, Y and Z are "commercial and that they will probably keep your data on their disks”. It is then up to the users to decide whether or not to use these recognizers. This is something we will consider for the next version(s).

Hands-on

On Thursday morning, following a short demonstration of how the portal could be used, participants were invited to upload a short sound fragment (own sound file or one available via workshop portal) and recognize, edit, align and finally download the results.

In most cases, this worked well but the systems of the LMU were unable to operate with 20 users in parallel, so error messages appeared, and some participants had to wait a very long time to get the results of a fragment of 5 minutes.

Scalability

The biggest problems were solved overnight by the team of Christoph, but scalability is certainly something to look at for the next version. Fortunately, most participants were very pleased with the simplicity of the portal. The only thing that turned out to be tricky was extracting the audio from video interviews and / or converting special formats (eg *.wma or *.mp3) into the proscribed *.wav format.

Transformation

Technically this transformation is a piece of cake, but where users do not know how to do it and do not have the right software on their computer, this may be a barrier. The future option to do it in the portal was therefore greeted with enthusiasm.

Most participants were more than satisfied with the recognition results and did understood that automatic speech recognition of sound fragments that were barely audible was a difficult, if not an impossible ask.
Participants asked whether additional output formats could be included so that they could import the results of the entire process directly into their own systems (Zwangsarbeit Archiv, ELAN), and whether XML files, marked up in, say TEI (Text coding Initiative), could be exported to be exploited in onward tools. Technically this is no problem, but we cannot support all formats of all OH-projects. The portal builders have indicated that in the short term they will look at potentially interesting export formats and will add these to the current export formats.

Conclusion

In general, the participants were satisfied with the opportunities presented by the OH portal. Everyone could, after some help with converting the sound files, process their files, correct the automatic transcription manually, re-align it and download the final results. The fact that the load of the services was too high due to the simultaneous use of 20+ participants, which caused the systems to fail, was actually the only thing that went wrong during the hands-on session. For the builders of the portal however, it was a useful wake-up call ?.

In the coming months the scaling problem will be solved and several other recognizers (both commercial and non-commercial) will be added. During the CLARIN conference days in Pisa we will see which other CLARIN participants have a recognizer available and would like to participate in this OH-portal.

Finally, given that participants at Munich were testing other stand-alone CLARIN and non CLAIN speech processing and analysis tools, the idea of extending the idea of a TChain to an “AChain“ (annotation and analysis) might be useful, thereby offering a more seamless journey from audio recording to an annotated (knowledge-rich) interview.

Kaldi

At the moment Kaldi is the most popular platform for Deep Neural Network (DNN) based speech recognition. The Dutch and English recognizers are already working with Kaldi and both in Germany and Italy scholars are working on a Kaldi-based recognizer for their own language. Because it would be a shame to invent the wheel several times, it was agreed to investigate to what extent we can join forces and work together on Kaldi-based recognizers.

Arjan van Hessen

In response to your inquiry Automatic e-mail answer suggestion in a Dutch Contact Centre

logo Telecats h

M.R. Boedeltje and A.J. van Hessen

Telecats and Telecats/University of Twente

Abstract

In the past years, the number of service requests through e-mail has shown an explosive growth. Equal to most telephony services handled by call centres, 80% of the incoming e-mails is about 20% of the subjects, making it worthwhile to compose standard answers for at least the 20% most popular questions. Personal answering (each e-mail is answered by a human agent) is simply too expensive to do without the use of predefined answers. By using IR and text classification techniques combined with Natural Language Processing, the process of finding the correct answer for a request can be (partly) automated. In this paper we will describe an e-mail answer suggestion system using IR based classification and NLP techniques. A practical study using an e-mail corpus of 17,000 incoming e-mails (collected and categorized in a Dutch contact centre), has shown that this approach is able to present the correct answer within a ranked list of 5 possible suggestions, for almost 88% of all incoming e-mails. Furthermore, we will show that this approach can be used as well
for spoken content by combining the categorization techniques with the recognition result of the answer on the famous question: ”How can we help you?”.

1 Introduction

With the ongoing acceptance of e-mail as a fast, cheap and reliable way of communication, companies receive an increasing number of service requests via e-mail. To handle these e-mails, call centres are ”transformed” into so-called contact centres handling both e-mail and telephone calls. Since most service requests cover a relatively small set of problems or questions (80% of the mails is about 20% of the subjects), many of these requests may be answered using a relatively small set of standard answers. Handling great amounts of e-mail in a contact centre is a very labour-intensive task, requiring a serious investment of time and money. Automating the answering process could therefore account for serious cost reduction and a decrease in response time. Due to the difficulty of automatically selecting the correct answer and thus the risk of sending the incorrect answer to a customer, most companies are reluctant to incorporate such an automatic e-mail answering system in their contact centres. However, automatically suggesting one or more relevant answers to incoming messages provides a good alternative. If we manage to suggest the correct answer in for instance a top-x of relevant answers, the agent only needs to select the correct answer out of these x answers, instead of
formulating the answer manually or searching the correct answer from all possible answers. Such a system would improve the efficiency in a contact centre and reduce the time spent on answering e-mail. However, one has to tune the number of suggested answers. Suggesting a small number of answers decreases the time an agent has to spend on browsing through the suggestions if the right answer is in these suggestions. However, if the right answer is not present in the suggestions, an agent has to spend extra time on manually searching the correct suggestion. Increasing the number of suggestions increases the chance that right answer is in the
suggestions, but also increases the time an agent has to browse these suggestions. Ideally, no more than 5 suggestions are given to the agents.

full paper

 

Pagina 3 van 4

  • 1
  • 2
  • 3
  • 4
  • Laatste aanpassing website: donderdag 26 januari 2023, 17:29:46.