How to Use Model Metrics to Gauge Uncertainty

Si Chen <sichen@...>

We've fitted some models and would like to know how to use them to really understand the quality of the models.  The model metrics look like this:

and comparing it to ASHRAE 14 guidelines, which gives us these formulas:

My questions are:

1. Is the autocorr_resid the rho (p) is B-14?
2.  What are the right parameters for n and m?  According to an early page in ASHRAE 14, n and m are "number of observations in the baseline (or pre- retrofit) and the post-ECM periods, respectively"   If the model is a daily, should n be 365, so in this case, n' = 365 * (1-0.4792) / (1+0.4792) = 128.5?  If the model is used to compare energy savings over a year, should m be 365?  Or should m be 30 if we're comparing the energy savings on a monthly basis?
3.  How many model parameters are there?  In a combined heating and cooling model, should it be 5 -- 2 betas, 2 balance points, and an intercept -- or 3?

Calculating all this from my example model, I get a 25.8% uncertainty for F (energy savings) of 20% at 68% confidence (t = 1)  Does that seem reasonable for a daily model with this much CVRMSE?


Join { to automatically receive all group messages.