Good news! However, only phone conversations?
"added its first Indic language sets: Monsoon en-IN and Monsoon hi-IN. The leaderboard previously covered only European languages.
The data comes from unscripted, dual-channel phone conversations recorded by 4,888 speakers across hundreds of Indian districts, each tagged with 12 attributes including age, occupation, device model, and native district.
Each language has a public split for self-scoring and a private, held-out split so models can’t be tuned to the test data;
Hindi references use a lattice of accepted spelling variants rather than one fixed transcript, since no normalizer can resolve Hindi’s spelling variations.
Prior research cited here found ASR error rates ranging from about 4% to 44% across different districts, a gap that standard aggregate word-error-rate scores don’t show. The results show that South Asian languages require different assessment tools, but show promise as ASR extends to billions of speakers on the subcontinent."
The Open ASR Leaderboard Adds Its First Global South Language "Voice Arena and Hugging Face partner to launch open ASR evaluation for Hindi and Indian English"
No comments:
Post a Comment