Assignment: Advanced Analytics
Objectives: The assignment will rely on a series of workshops in which students will understand how to use SAS Enterprise Miner (EM) to combine analytics of structured and unstructured data (text) to make business predictions. The assignment will assume good working knowledge of all previously studied methods.
Mini Case Study: This mini case study will be used in all workshops of module. All amendments, extensions and assumptions should be recorded in the final submission.
Business Problem - Early Warning System
The client is US National Highway Traffic Safety Administration (NHTSA, pronounced "NITS-uh"). NHTSA is an agency of the Executive Branch of the U.S. government, part of the Department of Transportation. They are responsible for reducing deaths, injuries and economic losses resulting from motor vehicle crashes. They require an Early Warning System for potential safety issues associated with automotive vehicles due to manufacturing problems. In particular they require an analytic model to be developed, capable of predicting the likelihood of a vehicle crash, based on publicly available vehicle safety complaints. In the circumstances when the likelihood of crashes is high, NHTSA will initiate a recall of all vehicles likely to be affected.
The sample data for recreational vehicles (e.g. pick-up trucks, minivans, SUVs, etc.), is from the NHTSA. Each record was filed by individuals who had experienced problems with a specific vehicle component that may or may not have resulted in a vehicle crash.
Two data tables have been provided:
- trucks1.sas7bdat (20.5 Mb of complaints data - download from CloudDeakin)
- stoplst2.sas7bdat (word stop list - download from CloudDeakin)
There are 56,601 observations in this sample of NHTSA data, where each observation is a document (record) representing a single complaint filed with the NHTSA through their survey instrument.
Target Variable: "CRASH" Approximately 30% of the complaints (documents) describe a situation in which a vehicle crash resulted from the failure/malfunction of the specified vehicle component.
The NHTSA collects consumer complaints regarding safety related motor vehicle and motor vehicle equipment by make, model and year, and includes Vehicle Component.
Consumers are directed to a web site that guides them through submitting their complaint through a survey instrument. The complaint information is entered into NHTSA's vehicle owner's complaint database and used with other complaints to determine if a safety-related defect trend exists. (For consumers without web access, they may call NHTSA directly and an operator will collect their information and enter it into the database.)
- If a safety-related defect exists in a motor vehicle or item of motor vehicle equipment, the manufacturer must fix it at no cost to the owner. The complaint is the first step in the process.
- Government engineers analyse the problem. If warranted, the manufacturer is asked to conduct a recall. If the manufacturer does not initiate a recall, the government can order the manufacturer to initiate a recall.
- The NHTSA does not have to receive a specific number of complaints before they look into a problem. They gather all available information on a problem. Each complaint is important to them.
Mini Study Predictive Analytics Workshop and Assignment
3 of 5 Questions
Q1. Describe the business problem and the potential value of the predictive model to the Propose an analytic solution to the problem and support your recommendation with references to the conducted data and text analytics.
Q2. Provide a summary of the sample data using descriptive statistics and frequency Specifically identify any anomalous or inconsistent data characteristics, explaining the potential impact.
Q3. Describe any treatments or transformations undertaken to resolve, the anomalous or inconsistent data characteristics from question 2.
Q4. Perform text analytics on the "CSUMMARY" data item, generating at least 5 topic clusters. Provide a description for each of the clusters generated.
Q5. Develop at least three predictive models for each of the following input characteristics combinations:
a. Using only the structured data (all columns excluding: CSUMMARY and the text topic clusters)
b. Using only the unstructured data (using only the generated text topic clusters)
c. Using both structured and unstructured data
Q6. For all models provide a summary of the model assessment statistics over the and validation partitions
Q7. Select the best predictive model and provide a summary of the model and its performance.