Bayesian hydraulic rating curve programs
Introduction
This web page presents my Bayesian hydraulic rating curve work, both when it
comes to theory and to programming. The object is to estimate the
parameters of the model:
Q=C(h-h0)^b *E
where E is independent log-normally distributed noise,
E~logN(0,sigma^2), Q is what you want to predict (often water
discharge) and h is the observation you want to use for the
predictions (often water surface height).
Typically this is used for making stage-discharge rating curves, that
is the relationship between water surface height and water flow in a
river (open channel flow). The regression formula says that
we are looking for a relationship between stage and discharge
on the form Q=C(h-h0)^b, but our discharge measurements are noisy and
are greater the greater the real discharge values are.
An introduction to my work can be found here, in
PowerPoint and PDF version.
Documentation
I've made a presentation of the theory behind the Bayesian rating curve
approach which I use:
The theory is further presented in the article
Trond Reitan and Asgeir Petersen-Øverleir
Bayesian power-law regression with a location parameter,
with applications for construction of discharge rating curves
Stochastic Environmental Research and Risk Assessment
In addition, a project for making multi-segmented rating curves has
been initiated, and has now culminated in a workable program,
called 'bayes_multiseg_ratingcurve'. The article
"Bayesian Methods for Estimating Multi-segment Discharge Rating Curves"
by me and Asgeir Petersen-Øverleir has been published in
Stochastic Environmental Research and Risk Assessment,
Volume 23, Number 5, Pages 627-642.
Link:
http://www.springerlink.com/content/j132m04555074112/
Programs
The multi-segmented analysis has now been in usage at
NVE for 3-4 years, now.
In addition, I've made two programs that performs single segment
Bayesian rating curve fitting. The basic estimation technique lies in the
Hydrasub library.
-
bayes_multiseg_ratingcurve
is a stand-alone program for fitting multi-segmented
rating-curves using Bayesian statistics.
This is a Linux/Unix program with GUI
interface, based on Motif and Hydrasub GUI components.
-
bayes_file_ratingcurve is a
Linux/Unix program with GUI
interface, based on Motif and Hydrasub GUI components.
-
commandline_bayes_ratingcurve is a command line program
that can potensially be compiled on several platforms. It uses the basic non-GUI
parts of the Hydrasub library which again uses the
GSL (Gnu Scientific
Library).
-
procmod_nmod
is the command line test version of a program for using the
process model for fitting multi-segmented. The program also
lists credibility intervals from the interpolation model,
which is the model used in
bayes_multiseg_ratingcurve.
-
backwater
is a rating curve program that fits the data to a
power-law parametric form, but with two stage inputs.
Both the stage at the measurement station, x, and the stage
further down, y, is used in order to take backwater effects
into account. The formula follows the usual power-law
when the stage further down is below a threshold, y0; Q=C*(x-x0)^b,
but follows another power-law above that threshold,
Q=C*(x-x0)^b*((x-y)/(x-y0))^c. The parameter y0 can be fixed to be
equal to x0. The five regression parameters, a=log(C),b,c,x0,y0,
are estimated using Bayesian statistics and MCMC.
-
ar1_vf_pmcmc5
is a program for analyzing dynamic rating curves. MCMC techniques
are combined with particle filtering (PMCMC) for doing this inference.
Measurement file examples
In order to run one of the three programs, you need a stage discharge
measurement file in the correct text format. Examples of that is
given below. The data is provided by NVE
(the Norwegian Water and Energy Administration) and should not be
used for commercial purposes.
Ordinary stage-discharge data
(Either just stage and discharge
or time, stage and discharge):
- Grimsmoen A small measurement set in a
small river, which shows how the Bayesian methods work for such sets.
An example of the same set without time specification is found
here.
- Nybergsund A larger measurement set
in a larger river. Here you will not get that much difference between
frequentistic and Bayesian methods.
- Kongsfjordelv
Measures streamflow from an upstream water power station. Small dataset.
- Sæternbekken
Few measurements done on a V-profile.
- Gryta
Many measurements done on a V-profile.
- Fiskum
Measurements done on a V-profile, used as a case study in the
multi-segment article.
- Målset
Multi-segment profile, used as a case study in the
multi-segment article.
Backwater data
(Two stage values, the first at the measurement
station, the second below it, one discharge):
Standalone solutions
The bare bones of the Bayesian one-segment fitting routines have
now been put in a single source file (plus include file), so that
the rest of the rest of the hydrasub library
is not necessary. This has been done for increased portability.
The source code can be found here:
bayes_generate_ratingcurve.C,
bayes_generate_ratingcurve.H.
Test code: testrat.C
Test data: grimsmoen.txt
This GUI program runs on linux, but can possibly be compiled on other
UNIX-like systems, including advanced X11/Motif installations of
Cygwin on
Windows.
It should make it easy for a user to make a rating curve based on
measurements and prior information.
The priors are default set from prior information obtained from Norwegian
rivers. You can however set your own priors or specify non-informational
priors for some of the more troublesome parameters (h0 and sigma^2).
The initial window looks like this:
There are also windows for showing the curve (with uncertainty),
for showing the posterior parameter distribution and for showing
the residuals. A more detailed description of the program,
with lots of screenshots, can be found here.
Linux
A Linux binary of the program is available here,
though it is probably best to compile the Hydrasub library,
then the program, which is found togeither with the library.
Windows
A Windows binary is available here.
It should be mentioned that the program requires an installation of
Cygwin with full X-support and with
Motif/Lesstif. This can be specified during installation of Cygwin. You
then need to run Cygwin and X (using the command startx in the Cygwin
shell) in order to start the program.
Source code
The source of the program can be found in the Hydrasub
library under the directory "prog". Use the full
installation.
This program is a command line version of
bayes_file_ratingcurve. The user can set the priors using
options in the command "commandline_bayes_ratingcurve". A file
containing the measurements must be specified on the commandline.
For instance, you can write "commandline_bayes_ratingcurve -b
2.4 2.6 -c measurements.txt" in order to read measurements from the file
"measurements.txt". The prior for b will then have 95% of being in the interval
2.4 to 2.6, while the rest of the priors are set from Norwegian conditions.
The '-c' options specified that only the parameters are to be shown in the
command window. Omitting this, you will also see a large table of stage discharge
estimations with uncertainty in the form of credibility intervals. You can
also show residuals, if you want to. The output can be redirected to a file
using the 'o' option. If the program is called without input arguments,
a large help description will be shown.
The following image shows one run of this program:
Windows
The program is available here.
Windows users has to use
Cygwin or at least the
dynamic library (the dll file) of
Cygwin and the Windows command line command
(Start->Run: write "cmd").
The most important commands you need, except
the program itself, will be "cd" which changes the directory
("cd .." goes up while "cd catalog" goes down to directory "catalog") and
"dir" which shows the content of the current directory. Go to
the directory where you have stored the program, some
measurement text files, and the Cygwin dll (though this can also be
stored at the system directory under Windows). From this location,
you can call the program.
If you want to compile this program yourself on Windows, you will
probabily need the Cygwin library, though with some adjustment
it could be possible to compile it using other compilers, too. You
will however need the GSL
library in any case.
Linux
Linux users will probabily be familiar with the command line, and will
not have any requirements for dynamic libraries.
A binary is available here.
Source code
The source of the program can be found in the Hydrasub
library under the directory "prog". Use the full
installation.
Just as bayes_file_ratingcurve, this is a
GUI program runs on linux, but can possibly be compiled on other
UNIX-like systems, including advanced X11/Motif installations of
Cygwin on
Windows.
It should make it easy for a user to make a (possibly)
segmented rating curve based on measurements and prior information.
Both the number of segments and the position of the segment limits can be
found automatically. The program also contains an interface
for specifying prior knowledge, both about the segmentations
and the other rating curve parameters. Uncertainty and other quality
measures are reported after the curve has been fitted. A quality
insurance module is also well under well.
The priors are default set from prior information obtained from Norwegian
rivers. You can however set your own priors.
The initial window looks like this:
There are also windows for showing the curve (with uncertainty),
for showing the posterior parameter distribution, for showing
scatter-plots of the sampled parameter and for showing
the residuals (with a quick analysis of trends).
Linux
A Linux binary of the program is available
here,
though it is probably best to compile the Hydrasub library,
then the program, which is found togeither with the library.
Windows
A Windows binary is currently not available, though such a program
can be made on request using Cygwin.
You would then need to run Cygwin and X (using the command startx in the Cygwin
shell) in order to start the program.
Source code
The source of the program can be found in the Hydrasub
library under the directory "prog"and in the library pagesw
hydrabase/bayes_multiseg_ratingcurve.C/H and hydragui/bayes_show_curve.C/H.
Use the full installation.
This is a command line program that utilizes the process model for
fitting multi-segment rating curves, in stead of the interpolation
model in bayes_multiseg_ratingcurve.
Some MCMC options can be set, but the fitting is done using
default prior information.
Linux
A Linux binary of the program is available
here,
though it is probably best to compile the Hydrasub library,
then the program, which is found togeither with the library. Also,
the vvgraph program is necessary for the graphical results.
Windows
A Windows binary is currently not available, though such a program
can be made on request using Cygwin.
You would then need to run Cygwin and preferrably X
(using the command startx in the Cygwin
shell), though the graphical results can be redirected
files with minimal programming.
Source code
The source of the program can be found here.
It uses the hydrasub library.
Use the full installation.
This is a command line program that analysis the backwater model
with Bayesian methodology. MCMC is used for the inference and
MCMC run parameters cna be set in the input, thus letting the
user set the resources used for the analysis.
Linux
A Linux binary of the program is available
here,
though it is probably best to compile the Hydrasub
library, and the source code.
Also, the vvgraph program is necessary for the graphical results.
Windows
A Windows binary is currently not available, though such a program
can be made on request using Cygwin.
You would then need to run Cygwin and preferrably X
(using the command startx in the Cygwin
shell), though the graphical results can be redirected
files with minimal programming.
Source code
The source of the program can be found here.
It uses the hydrasub library.
Use the full installation.
Programs for doing dynamic stage-discharge rating curve analysis are
described on their own page.
The multi-segment Bayesian rating curve program has been used by
NVE since 2009.
Their version of the program has
a number of adaptations to their database structure, described in this
introduction,
advanced usage manual and
a "under the hood" look
(only in Norwegian).
Links
- Back to the hydrasub page.
- Back to my main page.
Trond Reitan,
,
23/11-2012