PROJECTS
Goals
Use file processing and data mining to discover patterns of earthquake activity around the world over the past year; plot results on a world map.
Practice with
- data mining and file processing
- reading, understanding, and revising a program that has mostly been already written. This will require a thorough understanding of the k-means cluster analysis algorithm and file processing.
Background - Data Mining
In this project we continue to study earthquake activity. We will use a file with data once again obtained from the United States Geological Survey website, about earthquakes of magnitude 5 or greater that have occurred across the planet over the past year (since November 1, 2016).
We will implement a k-means cluster analysis data mining algorithm from Ch. 7 of the text and use it to analyze the earthquake data. We will then use Python turtle graphics to graphically report the earthquake clusters discovered by the k-means algorithm on a world map.
Requirements
Functions to read and process earthquake data saved in a file are adapted from Ch. 7 of the text.
Function readFile is based on the function in ch. 7 of the text (p. 255). You will need to revise it to accommodate the earthquake data file analyzed in this project: the earthquake file we are using has a header line, each line has a return (\n) character, the values are comma (not space) separated, and the longitude and latitude are in different fields ("columns") than the data in the text.
(readFile will be very similar to equake_readf from Project 7-2, except that it will return a dictionary rather than a list.)
Functions euclidD, createCentroids, and createClusters are all copied from Ch. 7 of the text. You will need to add docstrings and helpful in-line comments.
Note that the createClusters function includes code to report the results of the cluster analysis (for c in clusters ...). This is helpful for testing createClusters with a small data dictionary (data file), but you may want to comment this out for the large earthquake file.
visualizeQuakes is based on the function in Ch. 7 of the text. visualizeQuakes in this project has k (the number of clusters) and r (the number of times to repeat cluster analysis) as parameters, along with the dataFile parameter. As in the text, visualizeQuakes should call readFile, createCentroids, and createClusters.
visualizeQuakes will also call a new function, eqDraw, to plot the earthquake data on a world map. visualizeQuakes should return None. You will need to write a docstring and add comments to visualizeQuakes.
eqDraw is a new function that will be called by visualizeQuakes to do the work of plotting the results of the k-means analysis on a world map. eqDraw will have three parameters: k, eqDict, the earthquake data dictionary, and eqClusters, the list of clusters generated for the earthquake data by the k-means cluster analysis.
eqDraw will comprise the drawing code that is included in visualizeQuakes in the text. You should change the screensize method arguments to 1800 and 900, to reflect the size of the world map .gif that we will use. You can also use the anonymous turtle from the turtle module (i.e., you do not need to define quakeT). Tip: include speed('fastest').
Finally, define a main function to assign values to k, r, and f, and call visualizeQuakes. main should return None. Include code in your .py file to call the main function.
Testing your code
doctest.testmod will be useful for testing individual functions. Use small, made-up examples to test each individual function before calling it from another function. Start with a short version of the earthquake file (around 15 lines not including the header should be small enough to understand, but large enough to generate results).
Getting Started
Open a new Python editor window. Save it with the name p8f_equakes_vis.py.
Download the earthquake file (earthquakes.txt) and the world map (worldmap1800_900.gif) you will use for plotting earthquake points. Make sure the earthquake and map files are in the same folder as p8f_equakes_vis.py.
This program is somewhat longer than prior programs you have written, but when it is viewed function by function it should not seem so different. A good strategy for implementing a medium-sized project such as this one is to understand the code from the top down, and implement the code from the bottom up, with careful testing of each function at each step.
For example:
Start by carefully reading Ch. 7 of the text and reviewing class notes. When you are comfortable with your understanding of the k-means cluster algorithm, have a closer look at the code that is given in Ch. 7 to implement the algorithm. Type the code into a Python file, one function at a time.
For each auxiliary cluster algorithm function in turn (euclidD, createCentroids and createClusters), write the docstring for the function.
Test each individual function to make sure you understand and trust the function. Create your own small test dictionaries to test the functions. Use simple examples, like individual scores or earthquake magnitudes, at first. Then, try examples with two-dimensional points, such as earthquake longitude and latitudes.
Next, write the function that will access earthquake data, readFile. Note that you wrote a similar function for Project 7-2. You may want to start with that function and revise it to access longitude and latitude data (rather than magnitudes) and store the data into a dictionary (rather than a list).
You may want to create a short version of the earthquake files, for testing readFile. The data in the earthquake files is structured as for project 7-2:
time,latitude,longitude,depth,mag,magType,nst,gap,dmin,rms,net,id,updated,place,type,horizontal Error,depthError,magError,magNst,status,locationSource,magSource
2010-07-28T16:12:05.610Z,43.756,-125.815,10,5.2,mwc,193,143.9,,0.93,us,usp000hh0t,2017-
T16:34:36.951Z,"off the coast of Oregon",earthquake,,,,,reviewed,us,gcmt 1993-12-04T22:15:19.720Z,42.2915,-
122.0086667,4.797,5.1,md,126,113,,0.11,uw,uw10316468,2017-04-
13T22:06:07.852Z,"Oregon",earthquake,0.468,0.56,0.04,7,reviewed,uw,uw
The first line of the file is a header, which gives the meaning of each field (column) of the data. The data itself is comma-separated values, with one value (or the empty string) per field. Use the header to determine how to access latitude and longitude data. Note that when you access the earthquake file to create the dictionary, you will need to skip the header to get to the earthquake data.
Next, start putting it together, by writing visualizeQuakes. First call only readFile. When this is working, add a call to createCentroids, and so on.
When visualizeQuakes is working, write the main function and add a call to main
(main()) to your program.
Programming style is important!
Programs are read much more than they are written. All programs must include a program header at the top of the file with
- project identification
- author
- credit for any and all sources of significant help with the project excluding the class text and CIS 210 instructional staff. This includes programming partners, other texts, online information, etc.
- short, precise file description
Also remember to
- include a docstring in every function (type contract, brief description of the function, examples of use)
- use whitespace between operators and operands
- use descriptive variable names
- add appropriate in-code comments (why, not what).
Attachment:- Assignment_Data.rar