Question: Cleansing Weather Data
The process of weather forecasting begins with the collection of as much data as possible about the current state of the atmosphere. Weather data (barometric pressure, humidity, temperature, and wind direction and speed) is collected from a variety of sources, including aircraft, automatic weather stations, weather balloons, buoys, radar, satellites, ships, and trained observers. Due to the variety of data types taken from multiple data sources, weather data is captured in a variety of data formats, primarily Binary Universal Form for the Representation of meteorological data (BUFR) and Institute of Electrical and Electronics Engineers (IEEE) binary. These observations are then converted to a standard format and placed into a gridded 3D model space called the Global Data Assimilation System (GDAS). Once this process is complete, the gridded GDAS output data can be used to start the Global Forecast System (GFS) model.
For purposes of this exercise, imagine that the accuracy of the weather forecasts has been slipping. In your role as project manager at the National Center for Environmental Information (NCEI), you have been assigned to lead a project reviewing the processing of the initial data and placing it into the GDAS.
Review Questions
1. NCEI is responsible for hosting and providing access to one of the most significant archives on Earth, with comprehensive oceanic, atmospheric, and geophysical data. Good database design would suggest that an enterprise data model exists for the NCEI. Why?
2. Define the domain of acceptable values for barometric pressure, humidity, and temperature.
Critical Thinking Questions
1. What issues could cause the raw weather data received to be incomplete or inaccurate?
2. How might incomplete or inaccurate data be identified and corrected or deleted from the forecasting process? Are there risks in such data cleansing?