Resource: TC-STAR 2006 Evaluation Package - ASR Mandarin Chinese

Reference TC-STAR 2006 Evaluation Package - ASR Mandarin Chinese
Date of Submission Jan. 24, 2014, 4:31 p.m.
Status accepted
ISLRN 699-787-763-020-2
Resource Type Primary Text
Media Type Audio
Source
Language Chinese
Description

TC-STAR is a European integrated project focusing on Speech-to-Speech Translation (SST). To encourage significant breakthrough in all SST technologies, annual open competitive evaluations are organized. Automatic Speech Recognition (ASR), Spoken Language Translation (SLT) and Text-To-Speech (TTS) are evaluated independently and within an end-to-end system.

The second TC-STAR evaluation campaign took place in March 2006.
Three core technologies were evaluated during the campaign:
• Automatic Speech Recognition (ASR),
• Spoken Language Translation (SLT),
• Text to Speech (TTS).

Each evaluation package includes resources, protocols, scoring tools, results of the official campaign, etc., that were used or produced during the second evaluation campaign. The aim of these evaluation packages is to enable external players to evaluate their own system and compare their results with those obtained during the campaign itself.

The speech databases made within the TC-STAR project were validated by SPEX, in the Netherlands, to assess their compliance with the TC-STAR format and content specifications.

This package includes the material used for the TC-STAR 2006 Automatic Speech Recognition (ASR) second evaluation campaign for Chinese. The same packages are available for both English (ELRA-E0011) and Spanish (ELRA-E0012) for ASR and for SLT in 3 directions, English-to-Spanish (ELRA-E0014), Spanish-to-English (ELRA-E0015), Chinese-to-English (ELRA-E0016).

To be able to chain the components, ASR, SLT and TTS evaluation tasks were designed to use common sets of raw data and conditions. Three evaluation tasks, common to ASR, SLT and TTS, were selected: EPPS (European Parliament Plenary Sessions) task, CORTES (Spanish Parliament Sessions) task and VOA (Voice of America) task. The CORTES data were used in addition to the EPPS data to evaluate ASR in Spanish and SLT from Spanish into English.

This package was used within the VOA task and consists of 2 data sets:
- Development data set: consists of 3 hours of audio recordings from the broadcast news of Mandarin Voice of America between 1 and 11 December 1998 which corresponds more or less to 42,000 Chinese characters.
- Test data set: consists of 3 hours of audio recordings from news broadcast between 23 and 25 December 1998 and corresponds to 44,000 Chinese characters.

Version 1.0
Distributor ELRA