HA template for heat pump load forecasting

[BuhJuhWuh]

Hi all,

It's been a bit of a mission (templating has been possibly the least intuitive programming challenge I've taken on in living memory...), but I'd like to share a wee contribution: here is a home assistant template to predict heat pump demand from forecast temperature. It grabs the hourly forecast temperatures for the next 24 hours from the Met.no integration, and converts that to a predicted energy demand for use in Predbat's Load Forecast input based on a curve fit to data which you'll need to gather for your own house & heating.

Here, I've plotted my Vaillant system's total (heating + DHW) daily energy consumption against the daily average outdoor temperature, then fitted a quadratic curve to it. (I think it's probably important to have a dataset with consistent underlying settings, hence my somewhat reduced dataset here (made some big heat curve/scheduling changes a month or so back) - will probably update the coefficients in due course once this has been filled out a bit more.)

You will then need to put the coefficients into a, b & c (from line equation y = ax^2 + bx + c) in the template code below, and add this to your configuration.yaml file.

template:
  - trigger:
      - platform: time_pattern
        hours: /1
        #minutes: /1
    action:
      - service: weather.get_forecasts
        data:
          type: hourly
        target:
          entity_id: weather.forecast_home
        response_variable: hourly_forecast_data
      - variables:
          forecasts: "{{ hourly_forecast_data['weather.forecast_home'].forecast }}"
    sensor:      
      - name: "HP Energy Prediction"
        unique_id: HP_energy_prediction
        state: "{{ now().isoformat() }}"
        attributes:
          HP_Prediction: |            
            {# Choose coefficients for quadratic curve fit - specific to your house/heating demand #}
            {% set a = 0.1424 %}
            {% set b = -3.9174 %}
            {% set c = 38.081 %}
            {% set ns = namespace(hourly_energy_list=[]) %}
            {% set ns.incr_energy_list = [] %}
            {% for f in forecasts %}  
              {# For each hourly entry, get temperature value from 'forecasts' variable and store it in 't' #}
              {% set t = f.temperature | float %}
              {# Calculate hourly energy HP prediction based on temperature and known fit curve #}
              {% set hourly_energy = ((a*(t**2) + b*t + c)/24)|round(2) %}
              {# Add to list of hourly energy values #}
              {% set ns.hourly_energy_list = ns.hourly_energy_list + [hourly_energy] %}
              {# Calculate sum of hourly entries so far #}
              {% set incr_energy = ns.hourly_energy_list|sum %}
              {# Create dictionary entry #}
              {% set incr_entry = {"last_updated" : f.datetime, "energy" : incr_energy} %}
              {# Add to list of dictionary entries #}
              {% set ns.incr_energy_list = ns.incr_energy_list + [incr_entry] %}
            {% endfor %}
            {{ ns.incr_energy_list }}

This is working for me (can see the effect in Predbat’s load forecast column), by adding the following lines at the relevant place in Predbat’s apps.yaml:

# Load forecast can be used to add to the historical load data (heat-pump)
# To link to Predheat
# Data must be in the format of 'last_updated' timestamp and 'energy' for incrementing kWh
  load_forecast:
    - sensor.hp_energy_prediction$HP_Prediction

(The final part I'd like to add would be to update the state from its placeholder timestamp to show the total forecast energy (i.e. the sum of the hourly_energy values, or the final value in the incr_energy_list). Suggestions on how to achieve this welcome!)

Hopefully this is helpful to someone!

All the best,
Jo

[gcoan]

Ooh, this is very interesting. Thank you for all your hard work on this. I agree Jinja templating code is counter-intuitive, what bit I’ve done is lots of trial and error.

Some questions:

1. How do you get from average daily temperature vs heat pump demand graph to the calibration values a, b & c?
2. Do you need to subtract historical heat pump demand from house load history for then load_forecast to add the predicted heat pump demand back on in the predbat forecast?
3. The calibration is done on daily OAT but the forecast is hourly temp, does this cause any issues the granularity difference ?
4. What about the hot water heating cycle which in my case is a spike in the morning at 10am, this presumably can’t be modelled ?

Super, looking forward to incorporate into my own predbat forecast

[BuhJuhWuh]

Hmm, that does look a bit odd. Could it be as simple as an offset between the two datasets or something, so each consumption figure is matched with the wrong temperature data point? I'd expect a much closer correlation (though in my case I'm using the HP's external temp sensor so of course there's a good correlation - it's the key control signal driving the HP!). Could also be worth checking temp correlation between your sensor and a nearby weather station history (e.g. https://www.wunderground.com/history/weekly/gb/inverness/EGPE/date/2024-3-3 is generally within a degree or three of my HP's sensor) just to make sure it's sensible...?

Following on from your earlier Q about how to handle the DHW runs, I'm now swithering over whether I should be using heating+DHW consumption or heating only. Heating only gives a better correlation, but misses out the DHW chunk of the demand. DHW consumption is a bit more random, but does have a temperature element when it comes to efficiency (and therefore consumption). Going to stick with the combined values for now, but will mull that one over in due course.

[springfall2008]

I did start on Predheat which I suspect does something similar and links to Predbat but I haven't had any real feedback on how well it works

[BuhJuhWuh]

Thanks, yes, I did look at that, but felt that the level/complexity of input information required, coupled with the assumptions you have to make (e.g. counting and modelling the performance of all the emitters) was likely to result in a lot of work for a model of unknown accuracy. In my case I had the data handy, and had already spotted the correlation, so this felt like a simpler and more robust method.

The other aspect that didn't quite make sense to me was the idea of simulation at such high frequency - the system response (and changes to outside temperature) are so slow that simulating every 5 mins seemed like overkill - hourly should be adequate, and ought to help smoothing over HP cycling in mild weather.

In other words: I couldn't be bothered - my way seemed simpler 😜 (at least until I tried to learn Jinja for the first time inside HA's templating structures...)

The question that would be useful to know the answer to, though, is how Predbat handles the Load Forecast input: is this simply added to the historical data? Or does it subtract an overall general adjustment first before adding this more precise specific input back in?

[gcoan]

@BuhJuhWuh yes I was suspicious of the average air temperature and whether it was being averaged correctly.
Tried shifting the temperature results one day up and down and it made no difference.

Unfortunately I have no integration whatsoever with my ASHP. Its a pair of LG Therma V heat pumps which have a control panel that seems to double for air on units as well as heat pumps, and the manual doesn’t explain what things are either. There is a wifi module for the heat pump but from what I have read the main way of integrating has been via physical modbus connection. Not straightforward so is on the future project list!

I had looked at the met office for weather history but only found monthly averages, nothing more granular.

On wunderground there’s lots of local weather sites, two in the next village, but when I look at history it only shows me Cambridge Airport which is 20 miles away, and there’s no data for weekends, they must turn the weather station off or something!

Plotting Cambridge Airport temperature vs ASHP kWH gives a much better correlation line

I’ll see if I can find a more local source for temperature history, but this is an improvement.

[BuhJuhWuh]

Yes, that's looking better. I'd also try a few different trendline types and see what fits best - quadratic worked best for me, and looks like it might get you a closer fit than linear too.

We have a very local weather station on Wunderground but it doesn't seem to keep any history. Inverness airport is closest with a historical record - a good few miles away and lower elevation, but generally reasonably close to my HP sensor's readings.

[BuhJuhWuh]

Just a quick update - between March and now, the warmer section of my data curve has filled out nicely, with only fairly minimal changes to the curve fit:

We'll see what the left-hand end does over this coming winter...

[gcoan]

Thanks.

1. We're a family of 3 and my son has an electric shower, so really its only my wife and I using the hot water, so hence the strategy to only heat once a day (also not worth the overhead consumption of the heat pump starting every day).
   At the moment no integration between my heat pumps and HA so simplest to choose a cheapish import period and just leave it at that. Looking at my consumption history from May through September when the heating was off, it averaged 53kWh/month, 41 in July and there is a temperature related curve, but such small variations its not worth considering (for me).

2. One way to prove the double counting hypothesis would be to sum up the projected house load in the predbat plan and compare that to actual consumption. I would imagine you'd also see in the logfile that predbat is applying a large % uplift factor to today's consumption.
   I have to create my house load as a custom sensor as I have two inverters that share the load between them, so extending this to exclude heat pump usage would be easy:

# Home consumption sensor, updated every 5 minutes instead of the default of every minute
- trigger:
    - platform: time_pattern
      minutes: "/5"
  sensor:
    - name: "House Load Today"
      unique_id: "house_load_today"
      unit_of_measurement: kWh
      state_class: total
      device_class: energy
      state: >
        {% set x=(states('sensor.total_solar_energy_today')|float(0) 
          + states('sensor.total_battery_discharge_today')|float(0)
          - states('sensor.total_battery_charge_today')|float(0) 
          + states('sensor.grid_import_today')|float(0)
          - states('sensor.grid_export_today')|float(0) ) 
        %}
        {{ max(x,0)|round(1) }}

[BuhJuhWuh]

A couple of reasons for resurrecting this old thread:

1. I now have a full year-and-a-bit's data, which has filled out the cold end of the chart a little more:

2. From observations on a few days over this winter (notable for being particularly cold and/or having horrific Agile prices), it is apparent that @gcoan you are quite correct, and that there is double counting going on. For most of the year this hasn't really mattered, as it just gives plans which err on the conservative side, but not enough to be a big problem (beyond the day's cost estimates being too pessimistic, before being reeled in as the day progresses). But when it's been really cold out, and HP consumption has been particularly high, it can start to mess up the plans.

So:

3. How is load_forecast supposed to work? It seems to me that introducing any additional consumption prediction via load_forecast as per the documentation will result in this double counting (once from the historical data, once from the load_forecast). @springfall2008 is this correct? If not, what is predbat doing with the additional forecast data?
4. Which predbat entity should I be looking at for the historical data (and does that include or exclude the contribution from load_forecast)?
5. How often is the historical data evaluated (e.g. application of the days_previous_weight you specify in apps.yaml) - is this once at midnight? Rolling every hour? Every 5 mins? Something else?
6. Is there an obvious way to make the predbat historical data point to the house baseload (which should be easy enough to calculate - think I might have that as an entity somewhere already) rather than the total energy? EDIT: would this just be a case of pointing load_today at a different entity in apps.yaml?

Thanks,
Jo

[BuhJuhWuh]

Current plan of attack:

1. I've set up a template sensor baseload_energy_today_kwh (total consumption - heat pump consumption);
2. Change load_today in apps.yaml from load_energy_today_kwh to baseload_energy_today_kwh;
3. Shorten days_previous to just one or two days (had been using days 2-8), so that the history is more responsive;
4. Reintroduce my heat pump load prediction to load_forecast;
5. But before doing (2-4) above, need to wait for baseload_energy_today_kwh to populate a long enough history to cover my new days_previous.

[gcoan]

As mentioned earlier I create my own house load sensor and use that in apps.yaml, works fine although I do get warnings about gaps in the sensor history that I don't understand the cause of.

As well as the house load I created a net house load sensor, similarly a time based trigger that subtracts today's heat pump energy from the house load so I can see the base load.

The approach you suggest of separating out the base load and feeding the heat pump forecast in separately makes sense

[BuhJuhWuh]

This seems to be working well. The only slightly odd feature I've noticed (which is totally understandable/expected, it's just a thing) is that because the forecast I'm using only gets data for the next 24 hours, any hours in the plan beyond that won't include the HP usage. But that's no big deal as long as you remember what you're looking at!

[zarch1972]

Sorry for resurrecting this discussion, but I think this is fab, thank you for your work so far.

I've copy and pasted your code into HA and it works, i'm seeing incrementing hourly numbers that predbat seems to recognise.

Many questions to come though.
Is there sensor to see the total prediction for the day?
So that I can compare this to what my heat pump is already using (I have Open Energy Monitor - SDM elec meter and Sontex heat meter), so know what my heat pump is up to.

You can see my heat pump data here
https://emoncms.org/energystatsuk
https://energy-stats.uk/third-winter-with-our-vaillant-heat-pump/

Thinking about making this calculation open to more people, making in customisable?

Could there be a way to create the prediction graph based off a few fundamental system design numbers?

• Heat Loss of Property (kW)
• Design Outside Temp
• Design Inside Temp
• Design Flow Temp

ie, 4kW heat loss, -3C, 20C and 40C

We could use a some standard COP values. I have all the Vaillant raw figures that we could use to create some default tables.

I bet there are some heat pump users that will not have enough sensors and the suchlike to input into Preheat (as it's quite an amount of information).

if we could create an 'average' sort of ballpark figure based on some fundamental figures I think that would be great.

I'd be happy to help and have talked about something similar here
#2386

Cheers, Mick

[BuhJuhWuh]

Hi Mick,

I'll have to go and remind myself what I actually did, as it took a little bit of monkeying around to avoid double counting, etc. A bit flat out at present, but off next week and should be able to find a moment to go and refresh my memory.

Cheers,
Jo

[zarch1972]

No worries Jo. Sorry to going back in time. LOL

I think what you've created there is your own HTC, Watts required per degree of temperature difference.

I think what i'm suggesting is how can we make something like that for more people based on some simple inputs.

[BuhJuhWuh]

Not at all. It works well for me, so happy to help make it more useful for others too. And yes, you're quite right - basically using the house as a calorimeter!

…

On Thu, 16 Oct 2025, 14:31 zarch1972, ***@***.***> wrote: No worries Jo. Sorry to going back in time. LOL I think what you've created there is your own HTC, Watts required per degree of temperature difference. I think what i'm suggesting is how can we make something like that for more people based on some simple inputs. — Reply to this email directly, view it on GitHub <#837 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3BD7Y2VR3BCYOMZNJ6V6QL3X6M25AVCNFSM6AAAAACJLZFQX6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINRZHEZDINA> . You are receiving this because you were mentioned.Message ID: ***@***.***>