Publishing Partner: Cambridge University Press CUP Extra Wiley-Blackwell Publisher Login
amazon logo
More Info


New from Oxford University Press!

ad

Language Planning as a Sociolinguistic Experiment

By: Ernst Jahr

Provides richly detailed insight into the uniqueness of the Norwegian language development. Marks the 200th anniversary of the birth of the Norwegian nation following centuries of Danish rule


New from Cambridge University Press!

ad

Acquiring Phonology: A Cross-Generational Case-Study

By Neil Smith

The study also highlights the constructs of current linguistic theory, arguing for distinctive features and the notion 'onset' and against some of the claims of Optimality Theory and Usage-based accounts.


New from Brill!

ad

Language Production and Interpretation: Linguistics meets Cognition

By Henk Zeevat

The importance of Henk Zeevat's new monograph cannot be overstated. [...] I recommend it to anyone who combines interests in language, logic, and computation [...]. David Beaver, University of Texas at Austin


Summary Details


Query:   Natural language dialogue systems evaluation
Author:  gas0 Null
Submitter Email:  click here to access email
Linguistic LingField(s):   Computational Linguistics

Summary:   Ramon Lopez-Cozar Delgado
Dpto. Electronica y Tecnologia de Computadores
Universidad de Granada
18071 Granada, Espana (Spain)
e-mail: gas0@elvira.ugr.es
Fax: +34-58-243230

Dear LINGUIST Subscribers,

A few weeks ago I sent out an E-mail inquiry on
natural language dialogue systems evaluation.
I would like to thank the following people for
their kind replies. I enclose a summary of their
answers.

Nicholas Ostler - nostler@chibcha.demon.co.uk
Jean Caelen - Jean.Caelen@imag.fr
Frederik Fouvry - fouvry@llsun10.essex.ac.uk
Hugo Zaragoza - Hugo.Zaragoza@laforia.ibp.fr
Gerhard Hanrieder - hanriede@vs.dasa.de
Gavin E Churcher - gavin@scs.leeds.ac.uk
Philips Bouwman - bouwman@pfa.research.philips.com
Eric Atwell - eric@scs.leeds.ac.uk
Matthias Denecke - denecke@cs.cmu.edu
Robert John Gaizauskas - R.Gaizauskas@dcs.shef.ac.uk
Boda Peter - peter.boda@research.nokia.com
Doug Arnold - doug@essex.ac.uk
Bill Manaris - manaris@ucs.usl.edu
Maria Teresa Taboada - taboada+@andrew.cmu.edu
Alan Biermann - awb@cs.duke.edu
Mai Kuha - mkuha@indiana.edu
Hans Dybkjaer - dybkjaer@cog.ruc.dk

Special thanks to Nicholas Ostler for sending "Lying in
Wait at the Heart of the Web", a very interesting
report on the development of linguistic technology in
Japan.

The summary follows:

******* Interesting www sites ***********************

- The DIET project:
http://www2.echo.lu/langeng/en/le3/diet/diet.html

- Dialogue systems evaluation
http://xxx.lanl.gov/cmp-lg

- A Generic Template to evaluate integrated components
in spoken dialogue
http://www.scs.leed.ac.uk/gavin/papers/acl.htm

- PARADISE: A framework for Evaluating Spoken Dialogue
Agents
http://xxx.lanl.gov/abs/cmp-lg/?9704004

- Metrics for evaluating Dialogue Strategies in a Spoken
Language System
http://xxx.lanl.gov/abs/cmp-lg/?9612003

- EAGLES
http://www.ilc.pi.cnr.it/EAGLES/home.html

- Tests suites for Natural Language Processing
http://clwww.essex.ac.uk/group/projects/tsnlp/

- "End-to-end evaluation in Janus: a speech-to-speech
translation system"
http://www.is.cs.cmu.edu/ISL.speech.publications.html

********** Interesting ftp addresses *****************

ftp://ftp.ilc.pi.cnr.it/pub/eagles/evaluation/ewg95.ps
ftp://agora.leeds.ac.uk/scs/doc/reports/1997/97_06.ps.Z

************ Interesting reports **********************

- Neal J. Feit E. et al (1992): "An Evaluation Methodology
for Natural Processing Systems", Final Technical Report,
Rome Laboratory.

*************** Interesting books *********************

- "Evaluating Natural Language Processing Systems"
Karen Sparck-Jones and Julia Galliers, Springer, 1995

- "An Architecture for Voice-Dialog"
Smith, Hipp and Biermann,
Computational Linguistics, vol. 21, no. 3

Best regards,
Ramon.

LL Issue: 8.778
Date Posted: 24-May-1997
Original Query: Read original query


Back

Sums main page