Create a distributed web crawler and indexer. Your application should make use of a system that uses multiple crawler processes to traverse a set of websites and process or organize the data in a way that is easy to search.
Using the Google App Engine you will work throughout this course to develop your own Google App Engine application that can either be written in Python or Java programming languages.