Assignment
Write a program to gather information about word tokens in the sample database. You need to use Java programming language.
Use your program to generate the following information:
1.The number of word tokens in the database;
2.The number of unique words in the database;
3.The number of words that occur only once in the database;
4.For 30 most frequent words in the database, provide: TF, IDF, TF*IDF and probabilities.
(TF: term frequency
IDF:inverse document frequency)
5.The average number of word tokens per document.
What I need:
1. Documents of answers to the questions above (word is ok).
2. Source code.
Attachment:- Transcripts.zip