* * * * * * * * * * * * * * * * * * * * * * * *
LINGUIST List logo Eastern Michigan University Wayne State University *
* People & Organizations * Jobs * Calls & Conferences * Publications * Language Resources * Text & Computer Tools * Teaching & Learning * Mailing Lists * Search *
* *
LINGUIST List 20.256

Tue Jan 27 2009

FYI: Play an online game to evaluate Natural Language systems

Editor for this issue: Matthew Lahrman <mattlinguistlist.org>

To post to LINGUIST, use our convenient web form at http://linguistlist.org/LL/posttolinguist.html.
        1.    Alexander Koller, Play an online game to evaluate Natural Language systems

Message 1: Play an online game to evaluate Natural Language systems
Date: 26-Jan-2009
From: Alexander Koller <kollermmci.uni-saarland.de>
Subject: Play an online game to evaluate Natural Language systems
E-mail this message to a friend

First Challenge on Generating Instructions in Virtual Environments (GIVE)


Part of Generation Challenges 2009

Last call for participation
End date: January 31


Go to http://www.give-challenge.org,

interact with a natural language generation system,

and contribute evaluation data.

(But only until January 31!)


Dear colleagues,

A few months ago we announced the first GIVE Challenge and invited research
teams to develop natural language generation systems that generate
natural-language instructions to assist users in solving a puzzle in a
virtual 3D environment. In case you missed the original call for
participation, you can still find it at http://www.give-challenge.org/research.

Four teams of undergraduate students, graduate students, and researchers
from universities in three countries have risen to this challenge. Their
systems can now be used over the Internet. To evaluate the systems we need
as many users as possible who will play our 3D game and allow themselves to
be guided by the systems' instructions.

So we'd like to invite you to go to


and give it a shot! It shouldn't take more than ten minutes, you get
to try out your colleagues' systems, and you are contributing valuable
evaluation data. As an added benefit, we will give away a $50 Amazon
voucher each month from November to January to a random player.

If you could help us spread this invitation by passing it on to your
colleagues (not necessarily Natural Language Processing (NLP) experts),
friends, and students, we would be most grateful.

Notice that we will shut down the system after January 31.

Beyond helping us with our evaluation, there are various possibilities for
using this game as an educational tool which you may be interested in. Most
obviously, students can interact with and compare different Natural
Language Generation (NLG) systems by playing the game at
http://www.give-challenge.org. In addition, you could use the development
of an NLG server as a programming project and hold your own local
evaluation. You can download the software that you need to do this from:


Finally, consider for you or your students to participate in the next GIVE
challenge, which will most probably take place in 2010.


Donna Byron, Northeastern University
Justine Cassell, Northwestern University
Robert Dale, Macquarie University
Alexander Koller, Saarland University
Johanna Moore, University of Edinburgh
Jon Oberlander, University of Edinburgh
Kristina Striegnitz, Union College

Linguistic Field(s): Computational Linguistics

Read more issues|LINGUIST home page|Top of issue

Please report any bad links or misclassified data

LINGUIST Homepage | Read LINGUIST | Contact us

NSF Logo

While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed
on its pages, it cannot vouch for their contents.