Part A: Word Frequencies:
1. Write a method/function that reads in a text file and returns a list of the tokens in that file. For the purposes of this project, a token is a sequence of alphanumeric characters, independent of capitalization (so Apple, apple are the same token).
2. Write another method/function that counts the number of occurrences of each word in the token list.
3. Finally, write method that prints out the word frequency counts onto the screen. The print out should be ordered by decreasing frequency. (so, highest frequency words first)
Part B: Intersection of two files:
1. Write a program that takes two text files as arguments and outputs the number of tokens they have in common. Here is an example of input/output:
Input file 1:
We reviewed the trip and credited the cancellation fee. The driver has been notified.
Input file 2:
If a trip is cancelled more than 5 minutes after the driver-partner has confirmed the request, a cancellation fee will apply.
Output:
(optionally print out the common words, then...)
6
You can reuse code you wrote for part A.