Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Suggesting a new diagram and convention for characterising and reporting model performance
Uppsala University, Disciplinary Domain of Science and Technology, Earth Sciences, Department of Earth Sciences, LUVAL. Centre of Natural Hazards and Disaster Science, Uppsala.;Department of Sustainable Development, Environmental Science and Engineering, KTH Royal Institute of Technology, Teknikringen 10B, 100 44, Stockholm, Sweden..ORCID iD: 0000-0003-1149-5080
Uppsala University, Disciplinary Domain of Science and Technology, Earth Sciences, Department of Earth Sciences, LUVAL. Centre of Natural Hazards and Disaster Science, Uppsala..ORCID iD: 0000-0002-8180-4996
Show others and affiliations
2022 (English)In: EGU General Assembly 2022, European Geosciences Union (EGU), 2022, article id EGU22-8211Conference paper, Oral presentation with published abstract (Refereed)
Abstract [en]

A long-standing research issue, in the hydrological sciences and beyond, is that of developing methods to evaluate the predictive/forecasting skill, errors and uncertainties of a model (or model ensembles). Various methods have been proposed for characterising time series residuals, i.e. the differences between observed (or target) and modelled (or estimate) time series. Most notably, the Taylor Diagram summarises model performance via a single plot based on three related metrics: the (linear Pearson) correlation, standard deviation, and root mean squared differences of one or multiple pairs of target and estimate time series. Despite its theoretical elegance and widespread use, the Taylor diagram does not account for bias errors, which is an important summary statistic for evaluating model performance. Further, it is very common to evaluate, compare, and report on model “skill” by use of a single aggregate metric value, even when a vector of metrics is used to calibrate/train the model; most commonly this is a dimensionless efficiency metric such as Nash-Sutcliffe Efficiency (NSE) or Kling-Gupta Efficiency (KGE). Such “efficiency” metrics typically aggregate over multiple types of residual behaviours: for example the most commonly used version of KGE is based on correlation, bias, and variability errors, although the authors recommended that it should be applied in a context-dependent fashion based on which model behaviours are deemed to be important to a given situation. Nevertheless, the use of a single summary value fails to account for the interactions among the error component terms, which can be quite informative for the evaluation and benchmarking of models. In this study, we propose a new diagram that is as easy to use and interpret as the Taylor Diagram, while also accounting for bias. We further suggest a new convention for reporting model skill that is based on foundational error terms. Our vision is that this new diagram and convention will enable researchers and practitioners to better interpret and report model performance. We provide multiple numerical examples to illustrate how this approach can be used for evaluating performance in the context of multi-model and multi-catchment (large-sample) studies.

Place, publisher, year, edition, pages
European Geosciences Union (EGU), 2022. article id EGU22-8211
National Category
Oceanography, Hydrology and Water Resources
Identifiers
URN: urn:nbn:se:uu:diva-484158DOI: 10.5194/egusphere-egu22-8211OAI: oai:DiVA.org:uu-484158DiVA, id: diva2:1693766
Conference
General Assembly 2022 of the European Geosciences Union (EGU), Vienna, Austria, 23–27 May 2022
Available from: 2022-09-07 Created: 2022-09-07 Last updated: 2022-10-21Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records

Khatami, SinaDi Baldassarre, Giuliano

Search in DiVA

By author/editor
Khatami, SinaDi Baldassarre, GiulianoPool, Sandra
By organisation
LUVAL
Oceanography, Hydrology and Water Resources

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 22 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf