Re: How to Use Model Metrics to Gauge Uncertainty

Steve Schmidt

A few additional comments --
  1. I'd call this a "bad building". Based on the CalTRACK model fit results, energy use is not very predictable. The ASHRAE Guideline 14 requirement for a good model fit is CVRMSE < 0.25; this value of 0.46 is far above that target. Perhaps you can note this to users of your system, so they don't rely too heavily on the model.
  2. Savings calculations using such a [poor] model will be inaccurate. I'm no statistician, but I believe an R-squared value of 0.4 indicates some correlation, but is not considered useful for prediction. Current CalTRACK methods use any model with CVRMSE values below 1.0 to predict the counterfactual, so it's up to users to recognize when a fit is good and when it's not.
  3. It's odd that the cooling and heating balance points are the same; normally there are several degrees separation between the two. Maybe it's a strange building, or maybe Phil can explain this.

Join { to automatically receive all group messages.