Re: How to Use Model Metrics to Gauge Uncertainty


Michael S Uhl
 

Is it possible for energy loads that occur at specific times of day (unrelated to CDD or HDD), due to time-of-use pricing, to negatively impact the model accuracy? If so, how can these other variables be addressed? 


On Thu, Mar 5, 2020 at 4:53 PM Steve Schmidt <steve@...> wrote:
A few additional comments --
  1. I'd call this a "bad building". Based on the CalTRACK model fit results, energy use is not very predictable. The ASHRAE Guideline 14 requirement for a good model fit is CVRMSE < 0.25; this value of 0.46 is far above that target. Perhaps you can note this to users of your system, so they don't rely too heavily on the model.
  2. Savings calculations using such a [poor] model will be inaccurate. I'm no statistician, but I believe an R-squared value of 0.4 indicates some correlation, but is not considered useful for prediction. Current CalTRACK methods use any model with CVRMSE values below 1.0 to predict the counterfactual, so it's up to users to recognize when a fit is good and when it's not.
  3. It's odd that the cooling and heating balance points are the same; normally there are several degrees separation between the two. Maybe it's a strange building, or maybe Phil can explain this.

--
All the Best, M.
System Smart
484.553.4570
Sent on the go. Pardon grammar/spelling.

Join openeemeter@lists.lfenergy.org to automatically receive all group messages.