Discrepancies between Enlighten and HA production data on IQ Envoys

[madbrain76]

I have 3 Envoys - 1 legacy (2010), one combiner 4 (2022) and one IQ Envoy (2023, as of a few days ago).
For the legacy Envoy, the production data in Home Assistant exactly matches the (per gateway) energy production report.
For the IQ Envoys, however, the data diverges. With my 2022 combiner, the error was usually 0.5-2 kWh per day undercounted in HA I never filed a bug until now - it was close enough. With my newest 2023 Envoy, though, there is a more significant difference. Here is a per-gateway report for its first two days :

The 7/24 day was a partial day (Envoy installed in the afternoon) so the data can be ignored - I didn't add it to HA the very minute it got installed. 7/25 was a full day, however, and shows 33.01 kWh produced.

In HAOS however, it shows 26,565 Wh, ie 26.56 kWh when rounding. This is well short of the 33.01 kWh number from Enlighten for the same gateway. It's undercounting the kWh by about 20%.

[madbrain76]

Note that there is something else curious about this new Envoy, which is a separate problem, but possibly related.
That 2023 Envoy has 12 IQ8+ connected with 445W panels oriented southeast. In the summer in California where I'm located, the system clips heavily, roughly between 9:30am and 3pm right now, since the IQ8+ individually are rated for only 290W AC, but the panels are 445W (DC to AC ratio of 1.53).
In theory, the clipping should be at 12 x 290 = 3480W approximately.
but it happens around 2880W instead, about 17% less than predicted,

I noticed however that the sensors for the 12 individual inverters were all above 290W as predicted. I created a group in HA that adds all 12 values together, and the power curve looks like this :

As you can see, it's a similar flat line during the day, but closer to 3580W., not 2880W.
2880/3580 = approximately 80%, the same 20% difference that I'm seeing for the energy production.

I know the higher number is the correct one.

There may also be an Enphase firmware or setup issue at play, here. When I login to https://envoy-3.local during that clipping period, it also shows 2.8 kW production, not 3.5 kW . I don't know why that is. I'm not sure if it's related to the fact that my installer hasn't yet updated the map of micro-inverters in Enlighten. I can't see how that would affect sensors, but I thought I would mention it. I expect the map issue to be resolved this week.

[madbrain76]

Some more gateway-level production data from 7/25 .

1. Enlighten .
   

2. HA Energy dashboard

As previously noted, the 2023 Envoy has a huge difference between Enlighten and HA, but there is also a difference for the 2022 Envoy. - 66.1 kWh in Enlighten vs 65.27 kWh in HA . I have been seeing this small daily difference on the 2022 Envoy since it was installed and running with HA. I wasn't too worried about it when it was small, but now it is large with the 2023 model.

[catsmanac]

Are your newer ENVOY types metered ones with current clamps or standard types?

I guess Enphase is getting data differently as we do. I know installer and DIY account have access to more data and maybe more accurate data but that requires different custom integration like this one, but these lack support for legacy.

If you want to validate if HA is showing the data it's getting from ENVOY then enable debug on it, let it run for a couple of minutes and disable it again. Then look for the numbers in the logfile.

My ENVOY-S Standard is running since mid February and HA reports ENVOY 5.814 MWh, Enphase website 5.8, my HA Integrator 5.759Mh for lifetime. For my yesterday day production ENVOY Reports 54.08kWh, Enphase 54.1 and my HA integrator of all panels 53.52kWh. And here in the Netherlands I haven't experienced clipping with 380Wp panels and IQ7+. Yesterday my 24 panels reached ~6.7 kW

Limiting can also be caused by powerline voltage getting to high, but pattern may look different, not sure.

[madbrain76]

The 2 newer (IQ compatible) Envoys have current clamps. I think one may not have been configured yet on the newest Envoy.
I got the same difference yesterday for the 3rd Envoy as the day before - about 26 kWh production and about 33 kWh in Enlighten.
I'm in California and the panels are southeast facing so there is clipping with a 1.53 DC to AC ratio. That was expected. The clipping should go away in other seasons. In the meantime I'd still like to get the production right.
I do have debug enabled on the integration and will check the log later when I'm home.

[madbrain76]

BTW, PVWatts predicts 961 kWh production in July for the array on my 3rd Envoy, which is 31 kWh per day. That's in line to the 33 kWh per day that Enlighten reports for the last few days. Certainly much higher than 26 kWh/day. These past few days have been full sun with hardly any clouds, as are most July days in San Jose.

[madbrain76]

@catsmanac , I downloaded and installed your integration. So far, so good. I noticed that there is no "issue" tab on github at https://github.com/catsmanac/home_assistant_custom_envoy . You might want to fix that.

[catsmanac]

@catsmanac , I downloaded and installed your integration. So far, so good. I noticed that there is no "issue" tab on github at https://github.com/catsmanac/home_assistant_custom_envoy . You might want to fix that.

Thanks @madbrain76 , I added the issues tab, with the first issue being the note it's a test environment only as preparation to update @briancmpbll repository.

[catsmanac]

The 2 newer (IQ compatible) Envoys have current clamps. I think one may not have been configured yet on the newest Envoy.

Metering gateways without clamps connected have been reported as 'acting different'. The debug log should show you what is received from the envoys so you can better understand what's going on.

[madbrain76]

FYI, I put together a spreadsheet at https://docs.google.com/spreadsheets/d/18pqb03oafMNzuim3zTxiafPIXmhwnV8803hyFJvN3P0/edit#gid=0 that shows differences between HA data and Enlighten for each of my 3 Gateway for the last 2 weeks approximately -13 full days.

The executive summary is

1. there is no difference in numbers for the Envoy-R, except the total which is 0.1 kWh different, probably a rounding issue
2. for my IQ combiner (2022 gateway) there is a 10.55 kWh underreporting in HA vs Enlighten over that period
3. for my IQ PCB (2023 gateway), there is a 78.47 kWh underreporting in HA vs Enlighten over that period

I'm not sure where Enlighten is getting its kWh data from, but it doesn't seem to be available to any sensor in HA. Maybe it could be made available ?

[catsmanac]

Don't know how Enlighten is getting it's data, the envoy is pushing it afaik. There's more data pages but you need an Enphase installer account to access those.

The Wh data comes from the device itself, no W to Wh integration done by this custom integration.

Looking at the data the combiner 4 is off by 1.2% but the IQ PCB by ~25%. So looks like the real issue is with the PCB. Think you mentioned it before but don't recall, what firmware is it running, is it a metered type and does it have current clamps (CT) installed?

On the numbers, did you derive them from Lifetimeproduction of from Daily numbers?

In the end this is probably a question to ask Enlighten themselves. Only potential issue I can think of is you have a metered envoy without clamps installed and wrong number is used.

[Steve2017]

Hi @catsmanac - the test version is still working for me - that's the good news.

You might recall that I have been experiencing dropouts where all sensors become unavailable for about a minute at 23:00 my time (13:00 GMT)

I notice that you appear to have a similar issue, looking at your graph from this post above: #122 (comment)

I'd be interested to see whether other sensors within the integration also go off-line at that time - all mine do. Is there something in the integration that causes it to reset at the local time of 23:00? For example, does it revalidate the credentials at that time? The Enphase website is not superfast.

[catsmanac]

@Steve2017, I checked some other days and I see some brief drop, but not every day. And, I see drops certainly when I'm testing code changes from 1 or 2 other HA instances on my dev systems next to my production HA on my NAS. I've noted more often that when multiple devices are requesting data it may result in one having a short outage. And the 2 I found in a week were not same time, 1 at 10:30 pm the other at 11:30 pm. Both when no solar production was going on though.

[Steve2017]

Both when no solar production was going on though.

That is the same for me - no solar production. I had another drop-out at 06:36 this morning - still with zero production - but I have only seen that twice in the past week. The 23:00 dropout is very regular.

If there is a drop-out when there is production, it is much harder to detect.

It's 22:50 here so I'll see. I'll also see whether my next test on the "available" function works.

[dsabathi]

I have a one minute drop every day around midnight so I guess this is "normal behaviour" of my envoy-s gateway.

[catsmanac]

If you want to monitor this you can set an automation. I have it on current power production of the envoy.

[Steve2017]

Regular as clockwork. It has just become unavailable at 23:01. Returns at 23:02.

But - the Reimann Sum (Utility Meter) sensor that uses data from integration sensors to calculate export energy is now unavailable and I expect it will remain that way until solar production resumes.

The Reimann Sum sensor measuring grid import started again at 23:03 - 60 seconds after the dropout ended. This sensor uses the same format and functions as the solar export sensor!

The same sensors generated via the REST integration have not skipped a beat. I cannot find any sensors from other devices that follow this pattern, so I am back to thinking there is something in the Enphase integration that causes it to go off-line or re-set.

[catsmanac]

Did you catch it on debug? Ever tried stopping the rest interface around that time and see if the other continued?

The integration is using the timestamp lastReportDate it receives from the envoy for the inverter values, not the current time when data comes in. This implies that in the next day there is no actual value to use until the inverters start again. Last timestamp is at 23:02. An Integrator can only integrate when next timestamp/value comes in.

[madbrain76]

@catsmanac,
I derive the HA data from Daily. But the "Lifetime" number is also wrong (undercounted).
I don't have access to the installer account to see other data.

As far as asking Enphase for assistance, I have, but so far I have not received any help.
Obviously, they can't help with HA data. And you can't help with Enphase data.

@Steve2017 , I get this issue as well with dropouts, but not at 23:01 specifically. May I ask you create a separate issue from this one to track that problem ?

[madbrain76]

As far as firmware versions, my IQ Combiner 4 has firmware D7.3.130 (91f206) .
The Envoy PCB has firmware D7.3.466 (9debea) .
As I understand, the PCB one is the newest hardware revision.
Envoy-R LCD has firmware R3.7.31 (11b4e9) .

[catsmanac]

I have ENVOY-S EU Standard running D7.6.175. That brings 5 minute updates for Energy and Power (enphase website updates every 15). In HA I run Riemann Integrator on every individual inverter. I have an daily utility meter on the summed integrators that I can compare to the Envoy Daily energy. And I sum the running Integrator s into a total as well to compare to Envoy lifetime.

.

On the left Envoy Day Energy, then sum of HA Integrated panels, ENVOY Lifetime and Summed running inverter integrator s.

This shows that the Envoy Integration and the HA integration are close, within 1% for lifetime and far less for day.

Enphase website says I produced

• today: 50.80 Kwh compared to my local numbers of 50.793/50.816
• lifetime: 6.15 MWh compared to my local of 6.153/6.098

Individual panels integrated on Enphase website are ~1% higher then my HA integrated ones.

[madbrain76]

Another serious (but different) discrepancy happened yesterday. One of my IQ Envoys (starting with 2022xxx) went offline for a few hours during the prime sunshine hours. I think it had crashed. I had to reset it at the breaker box outside. The panels kept working, though, they just couldn't be monitored during that time.
However, Enlighten still got the expected kWh still at the end of the day in the report that I ran after midnight for that IQ Envoy.

By contrast, the total in HA was about 19 kWh short for that same Envoy.

Somehow, Enlighten was able to "catch up" the kWh even while the Envoy was down. It must be the micro-inverters themselves that store their production while Envoy is down. Unfortunately, the same "catch up" did not work for HA.

[aperezva]

Hi

I have the same drop at 23h , and another this at 6:55. As you say, is not everyday, but in the same time at 23h more or less. I all have the same problem I guess is a normal behaviour

Any further information?

[madbrain76]

Another odd thing I noticed is that the problematic Envoy (starting 2023) does not have a "Current Power Production" sensor. It has these for in L1/L2, but not a combined one. The sum of L1+L2 is still a maximum of about 2.8 kW, though, not 3.6 kW as it should be. Oddly enough, even though a consumption CT is not installed, there is a "Current Power Consumption" sensor, and it is adding up to 2.8 kW.

My IQ 2022 gateway has the missing sensor. It also has a "consumption" sensor which appears the same value as the production sensor. No consumption CT is installed on any of my gateways (I have a Rainforest eagle for that purpose, connects directly to my SmartMeter over Zigbee).

[catsmanac]

What the Envoy is reporting depends on how it's configured. An Envoy metered without CT connected/configured does not reportedly have reliable data in production and consumption segments. In the json that is visible, in a debug log file look for the line

Fetched from https://x.x.x.x/production.json?details=1: <Response [200 OK]>:

In there look for

      "type": "eim",
      "activeCount": 1,

If active count is 0 then the data in that segment is not filled from a CT and is probably not reliable.

[madbrain76]

@catsmanac ,
The activeCount is 1 . There is definitely a CT . The problem in the title is related to the fact that this CT is returning a 20% undercount (2.8 kW instead of 3.6 kW) during peak times. My installer tried to disable the CT but then the Envoy doesn't report production. I also verified that it's really producing 3.5 - 3.6 kW by turning off every other circuit (load and solar) and checking my PG&E meter - and it read -3.45 kW, not -2.8 kW like the production CT would have us believe.
I attached the JSON (as txt, since github won't allow .json) .
production.txt

HA shows 5 sensors, 2 for production, 3 for consumption.

The sensor names are unfortunately truncated, but the two production ones are actually L1 & L2. And there is no combined L1+L2 sensor. There shouldn't be anything on the consumption side, only production.

To get back to the discrepancy problem, I'm wondering if it's possible for HA to grab the per-micro-inverter kWh data. I believe adding these all up will give me the correct production data matching Enlighten data, and not the bogus data that I'm currently getting. Here are the sensors I have access to for Wh. Every single one is off, unfortunately.

[madbrain76]

@catsmanac
In the JSON file I attached, under production / 0 / wNow, it shows 3583 W . That is the correct wattage for the 12 IQ8+ micro-inverters being maxed out each between 290-300W. I believe this data is the aggregate of the micro-inverters.
production / 1 / wNow on the other hand shows 2874 W . That is the bogus data from the CT.

Unfortunately, I didn't see any value that looked like the correct lifetime kWh. The whLifetime is wrong for both of the production meters. 0 (inverters) has it at just 147,232 Wh . 1 (the CT) has it at 1,077,040 Wh. actual total, which I get when doing an Enlighten gateway-level energy production report from the installation date, should be upwards of 1,318,000 Wh.

[catsmanac]

@madbrain76

In the JSON file I attached, under production / 0 / wNow, it shows 3583 W . That is the correct wattage

This is a behavior of a Metered Envoy/Gateway without CT clamps configured.

Unfortunately, I didn't see any value that looked like the correct lifetime kWh

A Metered Envoy/Gateway without CT clamps configured reportedly resets lifetime kwh to 0 when reaching ~1.2MWh. But you should be able to see that in history I think.

The previous is just behavior that sounds familiar but does not match the activecount=1 for the production CT. That is the reason the integration uses the numbers from production / 1 and not production / 0 if activecount where 0.

Can you use your browser to access the envoys and open ivp/meters page and ivp/meters/reports. You will have authorize with the token. Don't open these on the Envoy Standard, it doesn't have these.

[madbrain76]

@catsmanac
Thanks for your reply. This particular Envoy, with serial ending 1048 which has inconsistent data does have a CT clamp for production. That is returning data in production / 1 / wNow. I believe an Envoy without clamps wouldn't have data in production/1 , as this is data from the CT. Unfortunately, that CT data is 20% too low. I'm still trying to figure out why. That is an Enphase issue, not an integration issue. I'm investigating it in https://support.enphase.com/s/question/0D53m00009N2g8NCAR/production-ct-on-iq-envoy-underreports-by-20?fromEmail=1&s1oid=00DA0000000aQ9D&s1nid=0DB2G000000Kz2X&s1uid=0053m00000CMyBg&s1ext=0&emkind=chatterCommentNotification&emtm=1694207239319 .

Since this 1048 Envoy has CT clamps, the lifetime value shouldn't reset to 0.

ivp/meters returns the following :

[
    {
        "eid": 704643328,
        "state": "enabled",
        "measurementType": "production",
        "phaseMode": "split",
        "phaseCount": 2,
        "meteringStatus": "normal",
        "statusFlags": []
    },
    {
        "eid": 704643584,
        "state": "disabled",
        "measurementType": "net-consumption",
        "phaseMode": "split",
        "phaseCount": 2,
        "meteringStatus": "normal",
        "statusFlags": []
    }
]

ivp/meters/reports returns this :

[
    {
        "createdAt": 1694292874,
        "reportType": "production",
        "cumulative": {
            "currW": 2865.361,
            "actPower": 2865.361,
            "apprntPwr": 2875.330,
            "reactPwr": -125.187,
            "whDlvdCum": 1093333.218,
            "whRcvdCum": 6.590,
            "varhLagCum": 101475.743,
            "varhLeadCum": 53357.581,
            "vahCum": 1227476.109,
            "rmsVoltage": 249.775,
            "rmsCurrent": 23.025,
            "pwrFactor": 0.99,
            "freqHz": 60.00
        },
        "lines": [
            {
                "currW": 1433.458,
                "actPower": 1433.458,
                "apprntPwr": 1438.320,
                "reactPwr": -62.582,
                "whDlvdCum": 546761.824,
                "whRcvdCum": 3.136,
                "varhLagCum": 50744.498,
                "varhLeadCum": 26709.323,
                "vahCum": 613470.019,
                "rmsVoltage": 124.979,
                "rmsCurrent": 11.509,
                "pwrFactor": 0.99,
                "freqHz": 60.00
            },
            {
                "currW": 1431.903,
                "actPower": 1431.903,
                "apprntPwr": 1437.011,
                "reactPwr": -62.606,
                "whDlvdCum": 546571.393,
                "whRcvdCum": 3.454,
                "varhLagCum": 50731.245,
                "varhLeadCum": 26648.258,
                "vahCum": 614006.090,
                "rmsVoltage": 124.796,
                "rmsCurrent": 11.516,
                "pwrFactor": 0.99,
                "freqHz": 60.00
            }
        ]
    },
    {
        "createdAt": 1694292874,
        "reportType": "net-consumption",
        "cumulative": {
            "currW": 0.000,
            "actPower": 0.000,
            "apprntPwr": 32.022,
            "reactPwr": 0.889,
            "whDlvdCum": 54.757,
            "whRcvdCum": 0.000,
            "varhLagCum": 61.141,
            "varhLeadCum": 0.753,
            "vahCum": 10963.653,
            "rmsVoltage": 250.070,
            "rmsCurrent": 0.256,
            "pwrFactor": 0.00,
            "freqHz": 60.00
        },
        "lines": [
            {
                "currW": 0.000,
                "actPower": 0.000,
                "apprntPwr": 14.158,
                "reactPwr": 0.889,
                "whDlvdCum": 27.372,
                "whRcvdCum": 0.000,
                "varhLagCum": 29.604,
                "varhLeadCum": 0.753,
                "vahCum": 4858.700,
                "rmsVoltage": 125.116,
                "rmsCurrent": 0.113,
                "pwrFactor": 0.00,
                "freqHz": 60.00
            },
            {
                "currW": 0.000,
                "actPower": 0.000,
                "apprntPwr": 17.864,
                "reactPwr": -0.000,
                "whDlvdCum": 27.385,
                "whRcvdCum": 0.000,
                "varhLagCum": 31.536,
                "varhLeadCum": 0.000,
                "vahCum": 6104.953,
                "rmsVoltage": 124.954,
                "rmsCurrent": 0.143,
                "pwrFactor": 0.00,
                "freqHz": 60.00
            }
        ]
    },
    {
        "createdAt": 1694292874,
        "reportType": "total-consumption",
        "cumulative": {
            "currW": 2865.361,
            "actPower": 2865.361,
            "apprntPwr": 2910.996,
            "reactPwr": -124.298,
            "whDlvdCum": 1093278.460,
            "whRcvdCum": 0.000,
            "varhLagCum": -101414.603,
            "varhLeadCum": -53356.828,
            "vahCum": 10963.653,
            "rmsVoltage": 250.070,
            "rmsCurrent": 23.281,
            "pwrFactor": 0.98,
            "freqHz": 60.00
        },
        "lines": [
            {
                "currW": 1433.458,
                "actPower": 1433.458,
                "apprntPwr": 1454.101,
                "reactPwr": -61.693,
                "whDlvdCum": 546734.452,
                "whRcvdCum": 0.000,
                "varhLagCum": -50714.894,
                "varhLeadCum": -26708.570,
                "vahCum": 4858.700,
                "rmsVoltage": 125.116,
                "rmsCurrent": 11.622,
                "pwrFactor": 0.99,
                "freqHz": 60.00
            },
            {
                "currW": 1431.903,
                "actPower": 1431.903,
                "apprntPwr": 1456.894,
                "reactPwr": -62.606,
                "whDlvdCum": 546544.008,
                "whRcvdCum": 0.000,
                "varhLagCum": -50699.709,
                "varhLeadCum": -26648.258,
                "vahCum": 6104.953,
                "rmsVoltage": 124.954,
                "rmsCurrent": 11.659,
                "pwrFactor": 0.98,
                "freqHz": 60.00
            }
        ]
    }
]

There is no consumption CT installed, so it's pretty weird this URI returns data for consumption . It appears the (undercounted) production data is reported as consumption .

[catsmanac]

@madbrain76, the first page (meters) shows correctly production as enabled and net-consumption as disabled. Total consumption is regardless calculated as net-consumption - production (Solar Included mode) bu has no meaning as net-consumption is marked as disabled and should be ignored.

And it shows the same (too low) production number as the production page.

Sounds like an Envoy issue indeed.

[madbrain76]

Looks like the production CT is reporting incorrect wattage and kWh. Definitely an Envoy issue and not an integration issue. I disabled the production CT, and I'm now getting correct data, from the micro-inverters for now.
I have ordered a new CT and will try to get it installed to see if I can get more accurate data in the future.
I think this issue can be closed as it's not the integration's fault.
However, there is unexpected behavior in the integration when switching the production CT on/off. I propose we discuss that as part of #153 (comment) .

[catsmanac]

Closing as proposed.