Named Entity Recognition and transliteration in Bengali

Authors: Ekbal, Asif; Naskar, Sudip Kumar; Bandyopadhyay, Sivaji

Source: Lingvisticae Investigationes, Volume 30, Number 1, 2007 , pp. 95-114(20)

Publisher: John Benjamins Publishing Company

Buy & download fulltext article:

Price: $37.41 plus tax (Refund Policy)

Abstract:

The paper reports about the development of a Named Entity Recognition (NER) system in Bengali using a tagged Bengali news corpus and the subsequent transliteration of the recognized Bengali Named Entities (NEs) into English. Three different models of the NER have been developed. A semi-supervised learning method has been adopted to develop the first two models, one without linguistic features (Model A) and the other with linguistic features (Model B). The third one (Model C) is based on statistical Hidden Markov Model. A modified joint-source channel model has been used along with a number of alternatives to generate the English transliterations of Bengali NEs and vice-versa. The transliteration models learn the mappings from the bilingual training sets optionally guided by linguistic knowledge in the form of conjuncts and diphthongs in Bengali and their representations in English. The NER system has demonstrated the highest average Recall, Precision and F-Score values of 89.62%, 78.67% and 83.79% respectively in Model C. Evaluation of the proposed transliteration models demonstrated that the modified joint source-channel model performs best in terms of evaluation metrics for person and location names for both Bengali to English (B2E) transliteration and English to Bengali transliteration (E2B). The use of the linguistic knowledge during training of the transliteration models improves performance.

Keywords: NAMED ENTITY RECOGNITION; HMM BASED APPROACH; NAMED ENTITY TRANSLITERATION; MODIFIED JOINT SOURCE-CHANNEL MODEL; BENGALI

Document Type: Research article

DOI: http://dx.doi.org/10.1075/li.30.1.07ekb

Publication date: 2007-01-01