Class project: Complex Word Identification
Andreas Vlachos and Fernando E Alva Manchego
The goal of the lab is to develop a system for the Complex Word Identification shared task. The demonstrators and myself will be available during the lab sessions to help you if/when you get stuck, as well as to discuss ideas.
For this assignment you are welcome to use any software or toolkit that you find useful. Note that initially you will only have access to the training and the development data. The test data will only be released to you in the last lab session on week 10.
The assessment (14.8% of the total grade) will be based on the report which should be 4 pages long (excluding bibliographic references) and it should address following questions:
Introduction: What is the task and why is it important?
Baseline system description: System descriptions in enough detail for the reader to be able to understand how to reimplement your baseline models and to appreciate why they are suitable for the task at hand.
Improved system motivation and description.
Experiments on development set. Does your idea work as expected? Evaluate on the test set the baseline and the improved system, is it still the case? Identify examples in development data which help showcase why the improved system works better.
Plot learning curves for the trainable systems you experiment with. Are some systems better than others when less training data is available?
Identify examples where your improved system fails to predict correctly and propose ideas for future work to address them.
Conclusions: what have we learnt from your experiments that could inform future work
Your final report should be written using the LaTeX or Word conference paper templates
Recommend the following timeplan:
Weeks 3,5 and 6: develop a baseline system and evaluate it on the dev data. Within a week after the session of week 6 (i.e. before the lab of week 7), you need to submit a short report (1 A4 page in PDF) to me (via e-mail) with what you have done so far. This will be graded to give you an indication of how good your progress is, but the grade will not be included in the final grade calculation. The purpose of it is to give you feedback and help as needed.
Weeks 8,10: develop an improved system and evaluate it on the dev and the test data.
Weeks 11-12: write your report and submit it via MOLE.
Attachment:- Assignment File.rar