Bayesian hydraulic rating curve programs

Introduction

This web page presents my Bayesian hydraulic rating curve work, both when it comes to theory and to programming. The object is to estimate the parameters of the model:
Q=C(h-h0)^b *E
where E is independent log-normally distributed noise, E~logN(0,sigma^2), Q is what you want to predict (often water discharge) and h is the observation you want to use for the predictions (often water surface height). Typically this is used for making stage-discharge rating curves, that is the relationship between water surface height and water flow in a river (open channel flow). The regression formula says that we are looking for a relationship between stage and discharge on the form Q=C(h-h0)^b, but our discharge measurements are noisy and are greater the greater the real discharge values are. An introduction to my work can be found here, in PowerPoint and PDF version.

Documentation

I've made a presentation of the theory behind the Bayesian rating curve approach which I use: The theory is further presented in the article Trond Reitan and Asgeir Petersen-Øverleir
Bayesian power-law regression with a location parameter, with applications for construction of discharge rating curves
Stochastic Environmental Research and Risk Assessment

In addition, a project for making multi-segmented rating curves has been initiated, and has now culminated in a workable program, called 'bayes_multiseg_ratingcurve'. The article "Bayesian Methods for Estimating Multi-segment Discharge Rating Curves" by me and Asgeir Petersen-Øverleir has been published in Stochastic Environmental Research and Risk Assessment, Volume 23, Number 5, Pages 627-642. Link: http://www.springerlink.com/content/j132m04555074112/

Programs

The multi-segmented analysis has now been in usage at NVE for 3-4 years, now. In addition, I've made two programs that performs single segment Bayesian rating curve fitting. The basic estimation technique lies in the Hydrasub library.

Measurement file examples

In order to run one of the three programs, you need a stage discharge measurement file in the correct text format. Examples of that is given below. The data is provided by NVE (the Norwegian Water and Energy Administration) and should not be used for commercial purposes.

Ordinary stage-discharge data

(Either just stage and discharge or time, stage and discharge):

Backwater data

(Two stage values, the first at the measurement station, the second below it, one discharge):

Standalone solutions

The bare bones of the Bayesian one-segment fitting routines have now been put in a single source file (plus include file), so that the rest of the rest of the hydrasub library is not necessary. This has been done for increased portability.

The source code can be found here: bayes_generate_ratingcurve.C, bayes_generate_ratingcurve.H.

Test code: testrat.C
Test data: grimsmoen.txt


The bayes_file_ratingcurve program

This GUI program runs on linux, but can possibly be compiled on other UNIX-like systems, including advanced X11/Motif installations of Cygwin on Windows. It should make it easy for a user to make a rating curve based on measurements and prior information.

The priors are default set from prior information obtained from Norwegian rivers. You can however set your own priors or specify non-informational priors for some of the more troublesome parameters (h0 and sigma^2).

The initial window looks like this:

There are also windows for showing the curve (with uncertainty), for showing the posterior parameter distribution and for showing the residuals. A more detailed description of the program, with lots of screenshots, can be found here.

Linux

A Linux binary of the program is available here, though it is probably best to compile the Hydrasub library, then the program, which is found togeither with the library.

Windows

A Windows binary is available here. It should be mentioned that the program requires an installation of Cygwin with full X-support and with Motif/Lesstif. This can be specified during installation of Cygwin. You then need to run Cygwin and X (using the command startx in the Cygwin shell) in order to start the program.

Source code

The source of the program can be found in the Hydrasub library under the directory "prog". Use the full installation.

The commandline_bayes_ratingcurve program

This program is a command line version of bayes_file_ratingcurve. The user can set the priors using options in the command "commandline_bayes_ratingcurve". A file containing the measurements must be specified on the commandline. For instance, you can write "commandline_bayes_ratingcurve -b 2.4 2.6 -c measurements.txt" in order to read measurements from the file "measurements.txt". The prior for b will then have 95% of being in the interval 2.4 to 2.6, while the rest of the priors are set from Norwegian conditions. The '-c' options specified that only the parameters are to be shown in the command window. Omitting this, you will also see a large table of stage discharge estimations with uncertainty in the form of credibility intervals. You can also show residuals, if you want to. The output can be redirected to a file using the 'o' option. If the program is called without input arguments, a large help description will be shown.

The following image shows one run of this program:

Windows

The program is available here. Windows users has to use Cygwin or at least the dynamic library (the dll file) of Cygwin and the Windows command line command (Start->Run: write "cmd").
The most important commands you need, except the program itself, will be "cd" which changes the directory ("cd .." goes up while "cd catalog" goes down to directory "catalog") and "dir" which shows the content of the current directory. Go to the directory where you have stored the program, some measurement text files, and the Cygwin dll (though this can also be stored at the system directory under Windows). From this location, you can call the program.

If you want to compile this program yourself on Windows, you will probabily need the Cygwin library, though with some adjustment it could be possible to compile it using other compilers, too. You will however need the GSL library in any case.

Linux

Linux users will probabily be familiar with the command line, and will not have any requirements for dynamic libraries.

A binary is available here.

Source code

The source of the program can be found in the Hydrasub library under the directory "prog". Use the full installation.

The bayes_multiseg_ratingcurve program

Just as bayes_file_ratingcurve, this is a GUI program runs on linux, but can possibly be compiled on other UNIX-like systems, including advanced X11/Motif installations of Cygwin on Windows. It should make it easy for a user to make a (possibly) segmented rating curve based on measurements and prior information. Both the number of segments and the position of the segment limits can be found automatically. The program also contains an interface for specifying prior knowledge, both about the segmentations and the other rating curve parameters. Uncertainty and other quality measures are reported after the curve has been fitted. A quality insurance module is also well under well.

The priors are default set from prior information obtained from Norwegian rivers. You can however set your own priors.

The initial window looks like this:

There are also windows for showing the curve (with uncertainty), for showing the posterior parameter distribution, for showing scatter-plots of the sampled parameter and for showing the residuals (with a quick analysis of trends).

Linux

A Linux binary of the program is available here, though it is probably best to compile the Hydrasub library, then the program, which is found togeither with the library.

Windows

A Windows binary is currently not available, though such a program can be made on request using Cygwin. You would then need to run Cygwin and X (using the command startx in the Cygwin shell) in order to start the program.

Source code

The source of the program can be found in the Hydrasub library under the directory "prog"and in the library pagesw hydrabase/bayes_multiseg_ratingcurve.C/H and hydragui/bayes_show_curve.C/H. Use the full installation.

The procmod_nmod program

This is a command line program that utilizes the process model for fitting multi-segment rating curves, in stead of the interpolation model in bayes_multiseg_ratingcurve. Some MCMC options can be set, but the fitting is done using default prior information.

Linux

A Linux binary of the program is available here, though it is probably best to compile the Hydrasub library, then the program, which is found togeither with the library. Also, the vvgraph program is necessary for the graphical results.

Windows

A Windows binary is currently not available, though such a program can be made on request using Cygwin. You would then need to run Cygwin and preferrably X (using the command startx in the Cygwin shell), though the graphical results can be redirected files with minimal programming.

Source code

The source of the program can be found here. It uses the hydrasub library. Use the full installation.

The backwater program

This is a command line program that analysis the backwater model with Bayesian methodology. MCMC is used for the inference and MCMC run parameters cna be set in the input, thus letting the user set the resources used for the analysis.

Linux

A Linux binary of the program is available here, though it is probably best to compile the Hydrasub library, and the source code. Also, the vvgraph program is necessary for the graphical results.

Windows

A Windows binary is currently not available, though such a program can be made on request using Cygwin. You would then need to run Cygwin and preferrably X (using the command startx in the Cygwin shell), though the graphical results can be redirected files with minimal programming.

Source code

The source of the program can be found here. It uses the hydrasub library. Use the full installation.

Dynamic rating curve programs (ar1_vf_pmcmc5)

Programs for doing dynamic stage-discharge rating curve analysis are described on their own page.

R example script

A R script for doing single segment rating curve inference, using the same model as in the C++ programs, has been put on my R page: singlesegment_powerlaw.R.

NVE usage

The multi-segment Bayesian rating curve program has been used by NVE since 2009. Their version of the program has a number of adaptations to their database structure, described in this introduction, advanced usage manual and a "under the hood" look (only in Norwegian).

Links


Trond Reitan, email:trondr at math.uio.no, 19/5-2014