1. Overview
This assessment is designed to test your achievement of course objectives 4 to 8 focussing on model calibration and rainfall-runoff modelling. This assignment is based on a hypothetical project and is divided into two main activities:
1. Familiarisation with AWBM software -described in Section 2
2. Calibration of AWBM for the Emu Creek catchment - described in Section 3
Details of the submission requirements for Assignment 1, as well as how the assignment will be marked, is described in Section 4.
2. Familiarisation with the AWBM model
Some students may have been previously introduced to a ‘cut-down' version of the Australian Water Balance Model (AWBM) when completing ENV3105 Hydrology. It is now important to understand the basic workings of the full AWBM which incorporates the attenuation of surface flows and base flows.
The AWBM is included in the suite of rainfall-runoff models contained within the Rainfall Runoff Library software (RRL Version 1.05). Download the RRL software from the eWater website and install. The RRL User Guide (Podger, 2004) is provided as a resource for Module 3 and gives information on how to install and run RRL. The guide also provides brief descriptions of AWBM (Chapter 4.1) and parameter optimisers (Chapter 5). It also provides useful advice on model calibration (Chapter 6).
AWBM has been progressively developed by Professor Walter Boughton and several publications are available that document its history, capabilities and application. Table 1 gives a list of selected papers and key information that they contain for you to complete this assignment. It is recommended that you peruse the selected papers.
Table 1: Selected AWBM publications
|
Publication
|
Key information
|
|
Boughton (2004)
|
Provides a general description of AWBM, including the 'self- calibrating' version AWBM2002 within RRL (under Custom method of calibration). This is also referred to as 'auto- calibration' based on an average storage capacity.
|
|
Boughton (2007)
|
Monthly potential evapotranspiration (PET) values considered acceptable as an AWBM input. Highlights the importance of areal rainfall as a critical AWBM input.
|
|
Boughton (2009)
|
Recommends to use PET adjusted by 0.85 as an AWBM input. Lists average sets of AWBM parameter values for various Australian states.
|
Quick Hints in Using RRL
RRL comes with sample projects and we will use the Jardine River (AWBM_Jardine.jobf) to illustrate running the AWBM. A brief description of AWBM can be found under the Model tab. As shown under the Input tab, the input data includes daily rainfalls and evapotranspiration from 1974 to 1989. Observed runoff in mm/day is also graphed - this measured data is used to calibrate the AWBM. Some basic statistics of input data are provided under Input Statistics. The catchment area is imputed under Edit/Project Details.
From the observed runoff plot, it can be seen that streamflows in the Jardine River exhibit a distinct and regular seasonal pattern. The Jardine River is located in the Cape York region of Far North Queensland and is considered to be Queensland's largest perennial river. The catchment is based on sandstone and contains numerous springs which slowly release water to the river system (QDERM, 2011).
Split sample periods for calibration and verification of the AWBM can be set by adjusting the slider bars under the Dates tab. Periods of highest and lowest runoff can be easily identified to assist in selecting appropriate periods. Alternatively a single calibration period can be set.
The Calibration tab allows the user to constrain the AWBM parameter range used in the calibration and/or to fix the parameters to constant values (under Boundaries and fixed parameters tab). The optimisation method can be selected from a range of Generic methods, or the AWBM Auto Calibration method can be used (under the Custom tab), or the values of each AWBM parameter can be set individually (under the Manual tab). The last feature is useful in testing how predicted streamflows may change if a parameter value is modified.
Under the Generic tab, the optimisation method (from a range of 7 options) can be selected as well as the primary objective function. The user can also apply a secondary objective function. The objective functions can be optimised on a daily or on a monthly basis. After the calibration run is made (by clicking on Calibration button), a range of output graphs can be selected and presented including scatter plots and runoff timeseries. More detailed plots can be made under the View time series tab including zooming into short periods of time (right click and Zoom In). The calibrated AWBM parameter values are updated and can be found under the Boundaries and fixed parameters tab, or under the Calibration results tab. Comparative statistics between the observed and predicted runoff can be found under the Data statistics tab.
RRL also have features to sensitivity analysis and to present and save output data. Refer to the RRL User Guide for details.
AWBM Familiarisation Activity - Jardine River
This activity is assessable and is included as a way for you to trial the RRL software before embarking on the full calibration exercise (See Section 3). Refer to Section 4 on the submission requirements for this part of the assignment.
Open the Jardine River project file (AWBM_Jardine.jobf) and use the RRL software to complete the following:
1. For the available data from 1974 to 1989, what is the average annual rainfall, evapotranspiration and observed runoff for the Jardine River?
2. Setup the following simulation periods: Warmup from 1 January 1974 to 31 December 1974 and a calibration period to 31 December 1986. Use the SCE-UA optimisation method and the RMSE objective function to optimise the AWBM. What are the AWBM calibrated parameter values that are generated from this optimisation? In terms of the degree of fit with the observed data, what are the RMSE, Nash-Sutcliffe E and Coefficient of Determination (Correlation) values?
3. Extract a scatter plot of monthly observed and predicted runoff. Also provide a ‘zoomed in' plot of observed and calculated runoffs covering the 1976 wet season.
As well as becoming familiar with the use of RRL, you should also understand the AWBM model algorithms (see good modelling practice Module 2.4). In particular, you should gain an appreciation of how the model results change when a parameter value is adjusted (see the ‘electronic control box' analogy Module 2.3). This understanding will assist you in calibrating the AWBM to measured streamflows. Complete the following:
4. The scatter plot and 1976 wet season plot generated from the above Step 3 provides a benchmark or starting point to compare how the AWBM streamflows may change in response to adjustment of the AWBM parameters. The total runoff which can be found under Data Statistics is also a useful measure of model results.
5. Taking each AWBM parameter in turn, rerun the AWBM with a higher and lower parameter value and regenerate the above outputs. Keep the values of the other parameters constant, so the effect of the selected AWBM parameter is isolated. You may need to substantially vary some parameters.
6. Describe the effect of adjusting each AWBM parameter on the plotted results. Describe, based on your knowledge of the AWBM model structure, why this effect has occurred.
3. AWBM Validation - Emu Creek streamflows
This part of the assignment involves the calibration and verification of an AWBM of the Emu Creek catchment, based on the modelling practices and validation procedures described in the Study Book Module 2 and 3. A gauging station is operated by the Queensland Department of Natural Resources and Mines (QDNRM) on Emu Creek at Emu Vale (GS 422313B). Details of the streamgauge can be found at the QDNRM Watershed website.
https://watermonitoring.derm.qld.gov.au/host.htm
As a background, the purpose of the analysis is to establish calibrated AWBM parameters for the catchment. These parameter values would assist in the development of an AWBM model required for proposed dam reservoir located on Emu Creek (this work is outside the scope of the assignment). It is intended that estimates of monthly streamflows over several decades covering a range of dry and wet conditions will be produced as inputs to a dam water balance assessment.
Data and Resources
Data and resources in order to complete the AWBM modelling are listed in Table 2 and can be downloaded from external websites.
Table 2: AWBM data and resources
|
Description
|
Source
|
|
Topographic map of the catchment and surrounding region
|
A topographic map can be extracted from the QDNRM QTopo online mapping tool.
|
|
Daily and monthly streamflows at GS 422313B for period of record
|
Available at QDNRM Watershed website.
|
|
Daily rainfalls at local raingauges and evapotranspiration data
|
Available at Bureau of Meteorology (BOM) Climate Data Online website
|
|
eWater River Analysis Package (RAP) software
|
Available from eWater website. Useful for gap filling timeseries data using Time Series Manager and for computing the base flow index at a streamgauge using Time Series Analysis.
|
Methodology
The following methodology is recommended to validate the AWBM and to apply the model to generate the required monthly streamflows. The tasks generally correspond to the good modelling practices described in Study Book Module 2.4. Chapter 6 of the RRL User Guide also provides useful information. The tasks are:
1. Check and prepare input data
2. Calibrate and verify the AWBM
3. Undertake a sensitivity analysis
4. Conduct a sanity check of the AWBM outputs
5. Generate monthly streamflow estimates
6. Document your work as a report
The tasks are described in more detail below.
Check and Prepare Input Data
BOM Climate Data Online provides a map showing rainfall stations in the local region of the Emu Creek catchment. Rainfalls are also measured at the streamgauge. Identify potential rainfall sites based on proximity and data availability. The hydrology data from BOM have quality codes indicating missing data or other problems that may affect the reliable use of the data. It is thus critical to review the data and to identify time periods when data is of poor quality or missing and unsuitable for modelling. This is essential to have confidence in the simulation periods that are selected to validate the AWBM.
The River Analysis Package (RAP) freely available from the eWater Toolkit website is useful software that can be used to identify gaps in timeseries data and plot the data. It also has the capability to fill in missing data by linear interpolation or by multiple linear regressions using data from other rainfall gauges.
The following tasks are suggested:
1. Download the historical streamflow data and identify missing periods and/or poor quality data. Evaluate the likely accuracy of the streamgauge rating curve based on the number and magnitude of the gaugings that have been done. This will provide an indication of the overall quality of the streamflow discharges derived from the rating curve. Identify periods of record that would be useful to test the AWBM streamflow estimates and generate a daily discharge file to load into AWBM. Ensure that days with missing data are assigned the correct RRL value (e.g. -9999).
2. Generate a topographic base plan using the QTopo website. Draw the catchment boundary of the Emu Creek gauge and the locations of nearby rainfall stations. If you do not have access to mapping or CAD software, it is suggested that a pdf editor will be adequate to do this task. PDF XChange Editor has basic polygon drawing and measurement tools. A free version can be downloaded from Tracker Software.
3. Plot the rainfall timeseries and check the available rainfall data for completeness and quality. Tabulate time periods for each station that you consider have data suitable for the AWBM modelling.
4. If rainfall sites have comparatively small gaps of missing data, use RAP to infill these gaps
5. Use the recommended activities described in RRL User Guide 6.1.2 to select rainfall sites and generate a catchment-average daily rainfall input file for AWBM
6. Obtain monthly potential evapotranspiration from BOM online maps. Generate a daily evapotranspiration input file for AWBM. RAP software may be useful for this purpose.
AWBM utilises daily rainfalls representative of the spatial average (or areal rainfall) across the catchment, whereas the measured rainfall data are point rainfalls. Thus, a geographical spread of rainfall stations should be selected that allow you to estimate the areal rainfall - stations outside of the catchment boundary may be required.
The BOM Water Data Online website includes the Emu Creek streamgauge within its compilation of Australian water information and also shows useful information for this site.
Calibrate and Verify the AWBM
The rainfall, evapotranspiration and observed streamflow data used for AWBM need to be loaded into the RRL. After the data files have been inputted into RRL, make use of the Input statistics to check that no errors have been made.
The setup and validation of the AWBM involves the use of three different approaches:
1. Testing of the AWBM using ‘initial guess' parameter values adopted from available publications.
2. Calibration using AWBM Auto Calibration
3. Calibration using a selected optimisation method Details of the three approaches are provided in Table 3.
Table 3: AWBM testing and validation approaches
|
Approach
|
Description
|
|
Test 'initial guess' parameters
|
Based on the AWBM publications given in Table 1 (and other resources that you can find), determine preliminary estimates of the model parameters applicable to the catchment. In the case of BFI, this parameter can be (approximately) estimated using the base flow analyser in RAP Time Series Analysis. Do a Manual AWBM run based on these 'initial guess' values for the full streamflow record. Report the fit with observed monthly streamflows.
|
|
AWBM Auto Calibration
|
Split the simulation period into a calibration and a verification run. As noted in the RRL User Guide (Section 6.2.2) ideally the calibration period should include both wet and dry extremes and has an average annual flow that is similar to that of the whole streamflow record.
Identify AWBM parameter values using Auto Calibration. Report the fit with observed monthly streamflows.
|
|
Selected optimisation
|
Use the same calibration and verification periods as above. Select one of the available AWBM optimisation methods (in the Generic tab) and justify your choice. Identify AWBM parameter values using the selected optimiser and report the fit with observed monthly streamflows.
|
Tabulate the sets of ‘calibrated' AWBM parameters based on the outcomes of the three different approaches and also summarise their performance in matching observed streamflows. Identify the AWBM parameter set that you consider to perform the best in predicting streamflows. If your selected approach is not based on ‘initial guess' parameters, rerun the AWBM using a single calibration over the entire simulation period (i.e. no split sampling). Report whether the fit with observed runoff has been significantly changed and/or improved.
AWBM Sensitivity Analysis
Computer modelling always carries a level of uncertainty and one approach to address uncertainty is to undertake a sensitivity analysis of the model parameters. The simplest approach is referred to as a ‘one-at-a-time' analysis. This involves selecting a parameter, increasing the parameter value by a fixed proportion (typically +10%), rerunning the model (keeping the other parameters to ‘calibrated' values) and reporting the model results. This process is repeated for a reduction of the parameter value (typically -10%). The whole process is then repeated for a different AWBM parameter until the sensitivity of model results to each individual parameter is established.
Use the selected AWBM parameter set as the starting point for the sensitivity testing. From the sensitivity analysis, you should be able get an appreciation of which parameters are more important in terms of their effect on model outcomes. This will guide you on which parameters should be closely checked (refer next task).
Sanity Check of AWBM Outputs
Sanity checks should (always) be made to give you confidence that the AWBM model is providing acceptable results and that no major errors of analysis or data input/output has been made. The sanity checks should include:
1. Checking that the selected AWBM values are consistent with the underlying principles of the model. For example, the store capacities should increase in size (i.e C1
2. Computing the observed and predicted mean annual runoff (ML/yr) for the full streamflow period. As a guide, these mean estimates should match to within 5%.
3. Compare the above estimate with the average annual runoff for the catchment based on Boughton and Chiew (2007).
4. Closely inspect how well AWBM daily flow estimates match recorded flows for a selection of large individual floods within the streamgauge record. This includes the peak discharge and the recession curve of the flood.
5. Compare flow duration curves for observed and predicted daily flows. These curves can be prepared within RRL.
These sanity checks may necessitate some adjustments to the AWBM parameter values. Manual changes to AWBM parameter values can be made in RRL.
Generate Monthly Streamflows
Once you are satisfied with the accuracy of model predictions, run the AWBM with the final set of parameter values to generate monthly streamflows for the available period of record. Provide a timeseries plot of monthly flow volumes..Also compute and plot the average flow volume for each month of the year to show how flows vary seasonally within Emu Creek. (Note, these volumes are the average over the simulation for each month of the year). The RAP software can generate monthly average statistics.