Locality Effects and Ambiguity Avoidance in Natural Language GenerationMichael White - Department of Linguistics, The Ohio State University
Locality Effects and Ambiguity Avoidance in Natural Language Generation
Department of Linguistics
The Ohio State University
Joint work with Rajakrishnan Rajkumar, Manjuan Duan, Ethan Hill, Marten van Schijndel and William Schuler
Comprehension and corpus studies have found that the tendency to minimize dependency length, or dependency locality (Gibson 2000, inter alia), has a strong influence on constituent ordering choices. In this talk, I’ll begin by examining dependency locality in the context of discriminative realization ranking, showing that adding a global feature for capturing a dependency length minimization preference to an otherwise comprehensive realization ranking model yields statistically significant improvements in BLEU scores, significantly reduces the number of heavy/light ordering errors and better matches the distributional characteristics of sentence orderings in English news text. Next, complementing this realization ranking study, I’ll present the results of a recent corpus study that goes beyond Temperley’s (2007) study of locality effects in written English by taking into account lexical and syntactic surprisal as competing control factors, where we find that dependency length remains a significant predictor of the corpus sentence for a wide variety of syntactic constructions, and moreover that embedding depth and embedding difference (Wu et al., 2010) together help to improve the prediction accuracy in cases of anti-locality. After that, I’ll turn to the question of whether statistical parsers can be used for self-monitoring in order to avoid remaining ordering errors in surface realization, in particular those involving “vicious” ambiguities (van Deemter, 2004), i.e. those where the intended interpretation fails to be considerably more likely than alternative ones. Using parse accuracy in a simple reranking strategy for self-monitoring, we find that BLEU scores cannot be improved with any of the parsers we tested, since these parsers too often make errors that human readers would be unlikely to make; however, by using an SVM ranker to combine the realizer’s model score together with features from multiple parsers, including ones designed to make the ranker more robust to parsing mistakes, we show that significant increases in BLEU scores can be achieved, and that vicious ambiguities can frequently be avoided. Finally, to conclude I’ll briefly present work in progress suggesting that self-monitoring can be effectively used in generating disambiguating paraphrases in order to crowd-source judgments of meaning.
If you would like to meet with the speaker please contact Vera Demberg.
Dr. Michael White is an Associate Professor in the Department of Linguistics at The Ohio State University. After obtaining his Ph.D. in Computer and Information Science from the University of Pennsylvania in 1994, Dr. White worked for eight years at CoGenTex, Inc., where he focused on developing practical applications of natural language generation technologies during multiple SBIR and DARPA projects and industrial consulting engagements. In 2002, Dr. White crossed the pond to Scotland where he worked for three years as a Research Fellow at the University of Edinburgh, managing Edinburgh’s effort on the COMIC dialogue system project as part of the EU’s Fifth Framework Programme. During this time, Dr. White also took over the development of the open source OpenCCG library, the first practical system for parsing and realization with Combinatory Categorial Grammar. With his colleagues in Edinburgh, Dr. White developed grammar-based and data-driven methods for producing utterances that use prosody to help highlight trade-offs among the available options that are important to a user. Since joining the faculty at OSU in 2005, Dr. White has continued to develop OpenCCG, extending it to a broad-coverage setting, supported by multiple NSF grants. He has also conducted research on evaluation, both on using eye tracking to evaluate prosody in synthetic speech and on using machine translation metrics to evaluate surface realization systems.