The LDC Institute is a seminar series on issues in language data and database creation. The goals of the seminar series are to create a forum of UPenn affiliates with an interest in language data, to communicate LDC and UPenn experience in data collections, standards, and annotation, and to work with researchers and others who may be interested in LDC data or who may wish to contribute new data to the archives.

The topics covered will be descriptions of data-driven projects, both at the LDC and elsewhere and tutorials on specific methodologies and systems for dealing with language data. We assume attendees would be UPenn students, both undergraduate and graduate, as well as LDC staff who are interested. The seminars will be scheduled on Thursday afternoons. For most seminars, food will be provided, since they will take place at noon or shortly after.

Title and Instructor

ABSTRACT: In the past few decades there has been a pressing demand and need to develop speech and language corpora for training, testing and benchmarking of speech technology systems for various applications. Richly annotated corpora and labeled databases are needed to develop models of spoken languages and also to understand the structure of speech and the variability that occurs in the speech signals due to a variety of factors.

There are twenty-two official Languages and more than 300 dialects used in India. These languages have their specific structure and properties which have significant differences from European languages. This talk presents some of the phonetic differences in Hindi compared to English and presents an overview of the efforts made by CDAC, Noida and some other institutions to develop text and speech databases, tools, and some techniques for the processing of some Indian languages. For the collection of speech databases, issues and procedures related to text selection, e.g. multiform units and variables such as demographic, dialectal, environmental, emotional, linguistic background etc. have been included. Special tools developed for the analysis of text are described. The objective has been that tools should be adaptive to other Indian Languages and also to other languages.

Details of some of the application-oriented task specific databases such as the ELDA-sponsored database for Hindi and the CFSL Speaker Identification database for forensic applications will be described in detail.

A PowerPoint version of this presentation is available:
Development of Resources and Techniques for Processing of some Indian Languages

Title and Instructor

In this talk Walker seeks to reduce this gap by outlining the apparent problem and solution spaces; and by opening for discussion the utility of annotation scoring metrics in the various domains of empirical, computational and corpus linguistics -- and more cogently in the domain of quality control for linguistic data creation.

The talk will be organized into four main sections, followed by a brief discussion:

1. ACE Annotation: An Illustrative Introduction
2. Exploring the problem space
3. Exploring the solution space
4. Extending the metaphor: Building better linguistic data

Dr. Draxler will describe the Ph@ttSessionz web application and its major components, SpeechRecorder and WebTranscribe, and he will outline the infrastructure developed at BAS for WWW-based speech recordings. Dr. Draxler will also discuss the strategies employed to enlist schools in the project and present preliminary analyses of the Ph@ttSessionz speech database.

Title and Instructor

.pdf, postscript and Powerpoint versions of this presentation are available.

Over the past decade of creating and managing speech corpora, LDC staff have developed literally hundreds of utilities, user interfaces and file formats. These databases are becoming increasingly complex in their structure, with rich standoff annotations organized across multiple layers. At the same time, the range of contributing specialties has become more diverse, as illustrated by LDC's future publication plans in such areas as field linguistics, sociolinguistics, gesture, and animal communication.

In this talk I will outline the traditional corpus production process and catalog the problems we have experienced. This provides the backdrop for LDC's R&D effort over the last four years, which has created new software infrastructure and a suite of annotation tools. I will introduce the principles and key concepts of the annotation graph toolkit (AGTK), describe the current tools, and give a brief overview of the tool development process. Finally, I will introduce OLAC, the Open Language Archives Community, and demonstrate how it is being used for describing and discovering language resources of the kind we create at the LDC. The talk will be followed by an informal demonstration session.

More information about AGTK and OLAC is available at agtk.sf.net and www.language-archives.org.

Note: This presentation will be accessible to a general audience.

Developers of the work I will report include Kazuaki Maeda, Xiaoyi Ma, Haejoong Lee, Beth Randall, Salim Zayat, Craig Martell, Chris Osborn, John Breen, Jonathan Dick, Eva Banik, Alan Lee and Jonathan Wright.

A PDF version of this presentation is available:
New Methods for Constructing Annotated Speech Corpora

The objective of the ACE Program is to develop core automatic content extraction technology to enable text processing through the the detection and characterization of entities, relations, and events. As part of the DARPA TIDES program, ACE supports technology research and development for various classification, filtering, and selection applications by extracting and representing language content (i.e., the meaning conveyed by the data). The ultimate goal of ACE is the development of technologies that will automatically detect and characterize this meaning.

For the past three years, the Linguistic Data Consortium has been developing annotated corpora for the ACE program. Data for ACE consists of newspaper, newswire and broadcast news transcripts. To support Entity Detection and Characterization, ACE annotators label selected types of entities (Persons, Organizations, etc.) mentioned in text data. The textual references to these entities are then characterized and multiple entity mentions are co-referenced. The Relation Detection and Characterization task requires annotators to identify and characterize relations between the labeled set of entities. The LDC's role in ACE has recently expanded to encompass annotation of all data for the ACE program as well as development and maintenance of annotation guidelines and annotation tools.

In this talk, we describe corpus development for the ACE program, focusing on annotation procedures and guidelines as well as quality assurance measures. In addition, we touch on particular annotation challenges including classifying generic entities, metonymic entity mentions (including the concept of GeoPolitical Entities) and identifying the temporal attributes of relations.

A powerpoint version of this presentation is available:
Corpus Development for the ACE (Automatic Content Extraction)

What is a bilingual dictionary? Most of us have used bilingual dictionaries, so the answer seems obvious. But when it comes to defining the structure of a dictionary as a database on a computer, the obvious becomes non-obvious.

I will talk about the structure of a bilingual lexicon, and in particular that of a lexical entry, from a computational and linguistic viewpoint. There are (at least) three levels at which one might define such structure. Proceeding from the most concrete to the most abstract, these are: the file format level (e.g. in terms of an XML structure); a model (using a modeling language such as UML); and an ontology of concepts.

The talk will in fact begin with the abstract ontology and proceed to a model, which I have been developing in collaboration with several computational linguists in SIL (the Summer Institute of Linguistics). I will have little to say about the file format level. Some issues which arise include:

• The users of the lexicon (human users vs. computational linguistics systems, such as parsers)
• On-line vs. printed dictionaries
• Data integrity issues (what happens when you delete the lexical entry of a word that was the synonym of another lexical entry?)
• Modeling areas outside traditional lexicography, especially morphology
• Multi-dialectal lexicons

For the brave, a model for lexicons, morphology and other linguistic components is located at http://fieldworks.sil.org/ModelDoc/. (However, when I tried to go there today, the site was down.)

A powerpoint version of this presentation is available:
Modeling Lexical Entries in Bilingual Dictionaries

Virtually all text corpora are drawn from "found" data -- material that already exists in electronic form to serve some purpose other than corpus-based language research, such as: publication of books, periodicals or daily news; archival preservation of public, commercial or government transactions; online discussions on various topics among diverse interest groups; and so on. The problem is that each data source has its own unique set of needs and conventions that dictate the data formats used to store and transport its particular content -- as well as its own rate of failure in making sure the data satisfy its needs and conventions.

The task for the LDC, working on behalf of corpus researchers, is to design and apply the tools needed to distill each source into a common, standardized form that will (1) maximize the usability of the data on any researcher's chosen computer system, (2) preserve as much information as possible from the source, and (3) discard as much interference and noise as possible -- and do all this with a minimum
of manual effort. The present talk will discuss strategies and tools that have been developed and used at LDC over the years for this purpose.

BITS and other Machine Translation Collection Projects
Xiaoyi Ma, Mark Y. Liberman

Parallel corpus are valuable resource for machine translation, multi-lingual text retrieval, language education and other applications, but for various reasons, its availability is very limited at present. Noticed that the World Word Web is a potential source to mine parallel text, researchers are making their efforts to explore the Web in order to get a big collection of bitext. This paper presents BITS
(Bilingual Internet Text Search), a system which harvests multilingual texts over the World Wide Web with virtually no human intervention. The technique is simple, easy to port to any language pairs, and with high accuracy. The results of the experiments on German - English pair proved that the method is very successful.

Postscript and .pdf versions of this paper are available.

Our goal is qualitatively better methods for automatically extracting information from the biomedical literature, relying on recent progress and new research in three areas: high-accuracy parsing, shallow semantic analysis, and integration of large volumes of diverse data. We are focusing initially on two applications: drug development, in collaboration with researchers in the Knowledge Integration and Discovery Systems group at GlaxoSmithKline, and pediatric oncology, in collaboration with researchers in the eGenome group at Children's Hospital of Pennsylvania. These applications, worthwhile in their own right, provide excellent test beds for broader research efforts in natural language processing and data integration.

In particular, we propose to develop and test new general methods for information extraction from text, based on on-going research at Penn in corpus-based algorithms for parsing, predicate-argument analysis and reference resolution. We will collaborate with groups led by Robert Gaizauskas at the University of Sheffield (UK) and by Jun-ichi Tsujii at the University of Tokyo (Japan), as well as with the GSK and CHOP groups, in applying these general methods to particular problems in biomedical information extraction. The GSK group has already made effective use of the best available information-extraction technology, so that the new techniques can be assessed in the drug-development application for their added value relative to the state of the art.

To give a simple, concrete example, we want a program that will read a phrase like

Amiodarone weakly inhibited CYP2C9, CYP2D6, and CYP3A4-mediated activities with Ki values of 45.1--271.6 μM

and add to a database a set of entries whose ordinary-language presentation is

amiodarone inhibits CYP2C9 with Ki=45.1--271.6
amiodarone inhibits CYP2D6 with Ki=45.1--271.6
amiodarone inhibits CYP3A4 with Ki=45.1--271.6

This project will also address several database research problems, including methods for modeling complex, incomplete and changing information using semistructured data, and also ways to connect the text analysis process to an information integration environment that can deal with the wide variety of extant bioinformatic data models, formats, languages and interfaces. Such problems are central to current database research at Penn. Our work will build on the progress represented by Penn's K2/Kleisli data integration environment, by extending it to deal with semi-structured data. This will be important managing the information that is extracted, and also in making use of the many specialized data resources that can be brought to bear in generating the high-accuracy text analysis required to extract the information effectively in the first place. Because the K2/Kleisli system is widely used in bioinformatics applications, and in particular has been used for several years at GSK, these extensions will facilitate collaboration with biomedical researchers as well as supporting the IE research itself.

The engine of recent progress in language processing research has been linguistic data: text corpora, treebanks, lexicons, test corpora for information retrieval and information extraction, and so on. Much of this data has been created by Penn researchers and published by Penn's Linguistic Data Consortium. As part of the proposed project, we will develop and publish new linguistic resources in three categories: a large corpus of biomedical text annotated with syntactic structures (Treebank) and shallow semantic structures ("proposition bank" or Propbank); a large set of biomedical abstracts and full-text articles annotated with entities and relations of interest to researchers, such as enzyme inhibition by various compounds, or genotype/phenotype connections (Factbanks); and broad-coverage lexicons and tools for the analysis of biomedical texts.

Data and Annotations for Sociolinguistics (DASL): Using digital data to address issues in sociolinguistic theory
Stephanie Strassel, Linguistic Data Consortium

A longstanding focus of sociolinguistic research has been the quantitative analysis of language variation and change, an endeavor that necessarily begins with the empirical observation and statistical description of linguistic behavior. Current technology encourages the collection and analysis of such data, and even the presentation and publication of research findings, wholly within the digital domain. Within the field of human language technology, such benchmark data has proven to be an essential ingredient for progress, as it reduces the cost of analytical infrastructure within research communities, frees researchers to focus on their interests, encourages collaboration and reduces impediments to new participants. However, most empirical data within sociolinguistics continues to be collected and analyzed by individuals or individual research groups, and is never made available to the wider research community. This proprietary approach to data hampers collaboration, the replication of studies and the comparison of models, methods and results, all necessary components of rigorous science. The prospect of digital data sharing in sociolinguistics also raises theoretical and methodological questions: Can sociolinguists make effective use of existing corpora? Do the insights gained from corpus data differ qualitatively from data most commonly used in quantitative sociolinguistics, namely recordings of sociolinguistic interviews? What are the best practices for the creation of new digital resources for sociolinguistic research?

The project on Data and Annotations for Sociolinguistics (DASL), currently underway at the Linguistic Data Consortium with support from NSF via the Talkbank project, begins to address these issues. DASL investigates the use of digital data in sociolinguistics through a series of case studies involving both analysis of variation in existing corpora and the creation of new data sets. This paper will introduce DASL's goals, assumptions, data and tools and will review annotation and corpus creation efforts and results to date.

Towards a Comprehensive, Empirical Analysis of Linguistic Data: the case of Regional Italian vowel systems

Any empirical study of language relies necessarily upon a body of observations of linguistic behavior even if the study fails to formally acknowledge its corpus. The decisions one makes in approaching data affect research profoundly by opening some avenues of inquiry while blocking others. Looking across research communities as diverse as sociolinguistics and speech technologies, one finds methods that may be integrated in order to both broaden research possibilities and to perform research more efficiently.

This presentation explores the relationship among data, tools, annotation (or coding) processes and the research they support. It will focus specifically on the quantitative analysis of linguistic variation. The data come from a series of sociolinguistic interviews undertaken to investigate the modeling of variation in the regional speech of central Italy.

After describing the motivation for this study, I will demonstrate a series of tools, processes and data formats that permit a comprehensive yet rapid analysis of vowel systems. Specifically, I will demonstrate tools for transcription and segmentation, lexicons and search tools that automatically select and categorize tokens of interest from the transcripts, batch processes that perform acoustic analyses of the selected tokens and an interface for managing and adding human judgments to these analyses. In the process I will offer a particular perspective on tool development, favoring information retention and annotator efficiency over computational efficiency and portability.

The tools and processes I will demonstrate were designed to interact with each other without losing precious information from the original audio such as the phonetic and lexical context of each vowel, the time and situation in which it was uttered, the demographic features of the speaker and so on. As a result, they make it possible or even trivial to explore issues that are otherwise impossible or at least difficult such as the interaction of phonological variation as a function of speech rate and time within an interview.

This presentation will discuss the various speech collection projects undertaken at the Linguistic Data Consortium at the University of Pennsylvania during the years 1995-present. Included will be a discussion of telephone speech data collection projects and on-site speech collection projects. Data collection projects covered in this talk will include: CallHome, CallFriend, Switchboard, ROAR and FISHER.

The presentation will start from the early standardization of Arabic and lead to the emergence of 'diglossia' and its linguistic and sociolinguistic consequences. The dominant attitudes towards linguistic reforms will also be presented. The second focal point will be on the Arabic reading process, its challenges and its consequences for reading performance and for education in general.

In a connected second part of the same presentation, Tim Buckwalter will focus on the Arabic NLP Issues. He will then present his morphological analyzer and his lexicon. A brief overview of the LDC Arabic Treebank project will follow.