I live in the UK and have been using the API for a little while now with great success!
However, API prediction vs actual production has been way off in recent days with an error margin of 50-75%.
Historically error margins have been in the 2-10% range.
I realise that the weather by its very nature is unpredictable, but could there be a reason as to why estimates are suddenly so far off?
Thanks in advance
An error margin above 10 % is not acceptable and thank you for bringing this up to our attention.
Please post some of your data discrepancies here or send them to support@solcast.com.au.
This is important for us to fix or explain errors of this magnitude.
If you could provide us your latitude/longitude (perhaps only to 2 decimal places to make it somewhat anonymous i.e. 51.50, -0.07 for London) or a town/region.
Second part the data logs you have showing the large discrepancies.
I have two roof mounted 3.54kW systems with a 3.68kW inverter connected to each with no production cap. I add both results together to get my estimates. Estimates are from ~5am each day.
Install date: 2018-02-18
Longitude: -3.48
Latitude: 50.71
Tilt: 36
PV1 Azimuth: -130
PV2 Azimuth: 50
As you can see, apart from the odd discrepancy estimates and generation are pretty close until the 24th when estimates are repeatedly out by a significant amount.
2018-05-10 Estimated: 36.606 kWh Generated: 34.827 kWh
2018-05-11 Estimated: 18.997 kWh Generated: 19.195 kWh
2018-05-12 Estimated: 36.633 kWh Generated: 25.948 kWh
2018-05-13 Estimated: 38.380 kWh Generated: 41.418 kWh
2018-05-14 Estimated: 40.601 kWh Generated: 44.064 kWh
2018-05-15 Estimated: 37.243 kWh Generated: 41.987 kWh
2018-05-16 Estimated: 20.642 kWh Generated: 17.198 kWh
2018-05-17 Estimated: 39.458 kWh Generated: 41.245 kWh
2018-05-18 Estimated: 32.578 kWh Generated: 32.821 kWh
2018-05-19 Estimated: 39.284 kWh Generated: 42.633 kWh
2018-05-20 Estimated: 40.511 kWh Generated: 41.744 kWh
2018-05-21 Estimated: 37.816 kWh Generated: 38.337 kWh
2018-05-22 Estimated: 34.133 kWh Generated: 35.234 kWh
2018-05-23 Estimated: 40.754 kWh Generated: 39.737 kWh
2018-05-24 Estimated: 20.377 kWh Generated: 14.472 kWh
2018-05-25 Estimated: 30.387 kWh Generated: 26.581 kWh
2018-05-26 Estimated: 32.935 kWh Generated: 21.525 kWh
2018-05-27 Estimated: 35.753 kWh Generated: 19.101 kWh
2018-05-28 Estimated: 37.962 kWh Generated: 36.934 kWh
2018-05-29 Estimated: 32.049 kWh Generated: 18.873 kWh
2018-05-30 Estimated: 16.297 kWh Generated: 5.467 kWh
Today looks like this so far, it’s overcast and with only a few hours of daylight left I’m not expecting the generation figure to hit 50% of the estimate:
2018-05-31 Estimated: 35.072 kWh Generated: 12.599 kWh
Hope this provides the info you need - let me know if you need more. And thanks in advance!
Update 1
We are running comparison historical forecasts against code changes over this period versus current code, this will take some time as there are many factors involved and after the results are complete they must be scientifically analyzed for the various input weights etc.
Thanks for the update.
Would it be of any use if I periodically provided an update to my ongoing results?
It would be thanks. Nothing obvious so far is sticking out as to why the day-ahead forecast performance dropped by a higher than expected amount, we’ve passed this issue onto one of our modelers to dig into the data more deeply. If you could post with updated numbers, having more days might help nail down the issue. 
The lead Forecast Systems Scientist has analyzed and reviewed all the data along with running forecasts and would like to respond with the following update.
On May 24th we did indeed make a change to our forecasting systems to fix a bug in the first few hours of the forecast period. We have reviewed this change to double-check that it was implemented as intended, and can confirm it was. In your case, (generating forecasts around 5am local), the change would have led to higher forecasts than before, which is consistent with your data.
It may just be happy coincidence that out previous (pre-May 24th) too-low radiation forecasts match with your particular forecast - we suggest reviewing the provided parameters on your system, in particular the efficiency factor, to correct the bias you are seeing in post-May-24th forecasts.
Also note that some of the errors will be due to the messy nature of weather forecasting where some days are just harder to predict than others.
Thanks for your continued support.
My observation is that PV estimates for sunny and partly sunny days are generally accurate, but PV estimates are still way too high on cloudy days.
Here’s an update on the results:
2018-05-31 Estimated: 35.194 kWh Generated: 13.903 kWh (Very Cloudy)
2018-06-01 Estimated: 32.118 kWh Generated: 23.255 kWh (Cloudy)
2018-06-02 Estimated: 38.439 kWh Generated: 42.871 kWh (Sunny)
2018-06-03 Estimated: 35.140 kWh Generated: 34.742 kWh (Partly sunny)
2018-06-04 Estimated: 32.765 kWh Generated: 25.409 kWh (Partly sunny, then cloudy)
2018-06-05 Estimated: 29.774 kWh Generated: 10.004 kWh (Very Cloudy)
Historically, cloudy days haven’t been too much of a problem - it’s only recently that prediction has started to be wildly inaccurate. Is a tweak to the loss factor really going to make such a huge difference? The system is less than 4 months old, and I don’t have any shading. I’ll try using loss_factor instead of the install date and see if it helps.
Thanks again!
Manually defining the loss_factor to 0.9 has reduced my PV generation estimates by around 2kWh/day.
This would not account for yesterday’s inaccuracies - the difference between 30kWh & 28kWh in forecast does not offset much of the 10kWh generation difference.
Even generation from dissipated / ambient light on days where there is full but light cloud cover was being predicted fairly accurately.
It’s almost as if cloud density isn’t being predicted correctly any more? (Not that I’m sure you even do that (but this is how I observe it).)
Any other ideas? I’m confident my system sizing is correct, because forecasts used to be good and are still very close on sunny or partly sunny days (when the clouds aren’t so dense).
It won’t account for a single days performance no but might be better representative of your system if you are seeing on average estimates that are too high. As your system is new and if is without shading (at this time of year), the install date method might be more accurate. This is something that definitely changes system by system.
We’ll continue to monitor our validation and appreciate you posting your PV system actuals here, feel free to update us at the end of this month which will give us a bigger sample of days after the change that we can dig into after that. At this stage, we aren’t seeing larger representative errors that I’m aware.
We do indeed estimate and forecast cloud opacity (density) which you can see on the /radiation/* endpoints (both forecasts and estimated actuals). Eg,
Estimated Actuals:
https://api.solcast.com.au/radiation/estimated_actuals?longitude=-3.48&latitude=50.71
Forecasts:
https://api.solcast.com.au/radiation/forecasts?longitude=-3.48&latitude=50.71
The ‘estimated actuals’ seem to reflect your cloud/sunny descriptions at your location for recent days, remembering that these are based off our satellite cloud detection. Like wise with 0-4 hour forecasts, beyond that we rely on other data sources for our systems. For Europe, our forecasts are updated every 15 minutes, and for some use cases, it can be better to perform more regular forecasts to get the best results.
Would still be interested in any data from your system that you would be willing to share here or to our support email.
Hope that helps.
Thanks for your response.
Would it serve all parties better if future correspondence regarding this topic was communicated via email instead of this thread?
I guess I could follow up this thread with a conclusion if/when some stability is achieved and avoid the noise here…
@spoonwzd what ever suits you, can understand if you don’t want to share measurements publicly, feel free to send them to our support email.
We will post up conclusion here if we think it might help others users in the future be more out of our API.
That graphic can be produced by leveraging the API. It is two calls merged together (estimated actuals + forecasts) into the chart graphic library (metricsgraphics library) although any compatible charting library that can work with time series data will work as well.
If you are going to use the data in a chart like this you will need to credit the source of the data from Solcast to be in compliance.
If you are wanting to redistribute or embed this data directly from Solcast for example you want the html chart output in an iframe I will need to discuss this with others if this will be publicly exposed as a free level service as it is currently intended with the transition to a site oriented system that direct chart access falls under paid level support.
