Looks interesting!


#1

I’m a hobbyist who has written my own app in PHP to monitor power and weather at my home. It integrates and analyses data from Enphase, Efergy, Pvoutput, WeatherZone and BOM sources to provide historical, real time and predictive information on power and weather and a limited home automation function through a Python controlled WeMo.

I’m excited to have discovered your API, which looks like it could enhance my app by applying a weather element to my solar generation predictions, rather than just the raw insolation data that I currently use for this purpose. Thank you for making it available to use!


#3

Great to hear. Out of interest, from a hobbyist user perspective, I was wondering if you could provide some feedback for how many times per day you might call this API to satisfy your use case? Your feedback would be appreciated :slight_smile:


#4

Just as an FYI, one aspect of the PV forecasts API is loss_factor which we try to default to something reasonable without information about installation date. If you find the shape of the forecast is correct, you might want to experiment with the loss_factor which represents losses in a PV system due to soiling, age etc.

Hope that helps!


#5

The plan was to call it just after sunrise to see what solar generation day lays ahead and perhaps update it mid morning, lunch time and mid arvo for greater accuracy for the latter half of the day. I use it to refresh a progressive daily electricity usage cost estimate that is displayed to the user (me!). My afternoon estimates are already reasonably accurate, as most of the solar day has passed. It’s my early morning estimates I am trying to improve.


#6

I noted that a loss_factor attribute exists but didn’t think it was necessary as my panels are just over 2 years old. Also, I noticed that the maximum estimated generation amount over is a lot lower than my system is actually able to generate at this time of year. eg. the highest pv estimate at any 30 minute window over the next week estimated for my system is 2360 at 2017-08-09T02:30:00Z, yet my system has already been generating (as reported by my Enphase Envoy) between 2600 and 2700 over a 40 minute period around midday on 02 Aug 16. ie. I don’t want to be de-rating my system given my system is easily producing more than the best estimate of your API.

Re the shape, yesterday it was reasonably representative but today it is all over the shop. eg.
image
I’ll keep observing and double-check I am comparing the right data. Please let me know if there’s anything else you think I should be doing to improve the accuracy of the estimate for my system.

Edit: here’s the API call I’m making (plus my API key of course). I have double checked lat long and system size is correct: https://api.solcast.com.au/pv_power/forecasts?longitude=149.213&latitude=-35.384&capacity=3720&api_key=XXXX


#7

Do you know the azimuth and tilt of your panels? Looking back using the estimated actuals endpoint, I used a clearer day to get try and better match the output you might see on system your size. Try the following settings to see if they are closer to the shape.

https://api.solcast.com.au/pv_power/estimated_actuals?longitude=149.213&latitude=-35.384&capacity=3720&azimuth=50&tilt=45&loss_factor=0.95

Some estimated actuals from August 2nd.

image


#8

Thanks. That curve looks more repreaentative of what was generated that day. Re azimuth, 10 out of 12 panels face 353 degrees and the other 2 face 83 degrees. Given there is only 1 azimuth parameter, I guess I either go for a composite heading or make 2 separate API calls with each sub array. Re tilt, my roof is standard pitch, somewhere between 25 and 30 degrees. I’ll try these various combos to see what gives the best representation.


#9

Yeah, 2 separate requests will probably be more accurate, at this stage the API only takes single property values for the system properties. This is to make the API easier to use and also to make it possible to composite multiple requests to represent complex PV installations.

More information on the different properties here -> https://solcast.com.au/api/docs/pv_power.html

loss_factor has a default of 0.9 to represent a 10% loss of output efficiency, however install_date can also be used to generate the loss_factor based on known rates of efficiency loss of PV systems overtime.


#10

Will try 2 separate API calls then and will put in the install date.

If you are interested, you can see the live output of my science project here: PowerWeather. Your API may help refine the dynamic daily cost estimate that shows up (bracketed) every third or so screen refresh.


#11

I’m getting better results now that I have written a routine to automatically download and combine the Solcast estimations for my two arrays at 5:30am. I then apply a shading profile, which changes daily for half of the year around the winter solstice, to account for known losses that my system experiences due to shading from a neighbours tree.

After smoothing my actual readings to last 30 minute averages to match the Solcast data set interval, I am now getting very high goodness of fit in mostly clear / clear conditions eg. 21 Aug 17

Cloudy / rainy days are still a bit hit and miss however, particularly in the early morning when the Solcast estimate is at its freshest eg. 15 Aug 17 and 20 Aug 17

Having said that, there are days when the opposite is true ie. afternoon forecasts prove less accurate than morning eg. 17 Aug 17

Here are the graphs of the subject dates:

My next step is to revise Solcast estimates hourly throughout the day and see whether that improves goodness of fit on cloudy / rainy days, particularly later in the day.

Thanks again for giving me the opportunity to evaluate your generation forecast data!


#12

Appreciate you sharing your graphs. Great to see :slight_smile:

You will definitely see an improvement if you update the forecasts hourly. The satellite based forecasts that are updated for Australia/Asia/NZ every 10 minutes as new imagery comes in, however it’s only for the near future of ~4 hours. Beyond ~4 hours results are blended with our longer term forecasts based off numerical weather models and some machine learning which are only updated daily (a bit after 5am AEST usually). For the absolute best result, forecasting every 10 minutes will keep your forecasts up to date with the very latest we have available, but generally every hour will also give you much better results. I can see on your 15th/20th day examples the forecast starts to deviate at ~7:30am which makes sense based on a 5:30am forecast time in complex cloudy situations. The frequent short term forecasts (nowcasts) are much better situated to give you a more accurate forecast as our cloud tracking can much more accurately predict where the cloud will be and how they will effect PV output.

I’d love to see the results when your forecasts are more frequent (hourly or every 10 mins) given you are accounting for shading yourself.


#13

I’ve implemented updates to Solcast forecasts every 30 mins during generation hours in my App and am already seeing much greater accuracy of current versus predicted generation. eg. Screen shot just now of my app shows current instant generation amount (see red arrow) with font colour dependent on how accurate the Solcast estimate for this five minute window (interpolated) is. A font colour of MediumSeaGreen, which it currently is and has been since implementing this feature, indicates the estimate is within +/- 5%.

I’m saving the shading-compensated and combined N & E array 30 minute data and will graph up accuracy versus time out from estimate when we get a rainy day again. Sunday is looking promising.


#14

That’s fantastic! Within +/-5% matches our validation work closely for within the first hour after a satellite update.

Love seeing your visualization work that is incorporating our data. Feel free to post any interesting visuals here, it’s always interesting to see visualization where our data can add value!

Thanks again for sharing :slight_smile:


#15

I have now integrated the solcast forecast (updated every 30 minutes and corrected for shading) into my existing 24 hr rotating weather forecast information bar such that it overlays daylight hours with forecast generation estimates (edit: now colour coded to indicate deviation from ideal insolation at each time interval). It is the yellow line on the following graphic and is scaled to show maximum possible generation (3096W) at the top of the bar down to 0W at the bottom. The graphic advances one pixel right every 6 minutes.

timeline_icons


#16

I have a question regarding API call csv data format. In the last week, the PeriodEnd parameter has changed from being in one format, ie. 2017-09-20T11:30:00Z, to being in three potential formats, namely:

  1. The previous format eg. 2017-09-20T11:30:00Z (most records)
  2. A different format eg. 2017-09-20T11:00:00+00:00 (usually the first 8 records)
  3. An abbreviated format with no time information at all eg. 2017-09-21 (all midnight records)

This caused some issues with my reading of the data until I adapted it to account for all three formats possible but it’s a bit ugly. I have checked the JSON version of the same data sets and there is only one format, as it should be.

I may end up switching to JSON format anyway but I was just wondering whether the multi-format date/time data field change to CSV was intentional :smiley:


#17

That’s no good regarding format change for the CSV endpoints. Thanks for letting us know about the issue and sorry for the delayed response.

I’ll look into the problem from our side and post back here when I understand what has happened.

I know about the missing timezone offset for midnight UTC, and it is on our todo list, but all records should be without timezine offset as all dates should be in UTC.

Could you give me an example of a request with all 3 different formats so I can reproduce?


#18

Sure. Below is an extract of the latest 30 min Solcast forecast csv file I have for my North facing array showing each of the 3 data formats. Line 9 and onwards are in the standard format. Lines 1-2 and 4-8 show the second format (Z dropped and with the extra +00:00). Line 3 shows the date only format. The API call I used was: https://api.solcast.com.au/pv_power/forecasts?longitude=149.213&latitude=-35.384&capacity=3100&azimuth=7&install_date=20150318&api_key=XXXXXXXXXXXXXX&format=csv

PeriodEnd,Period,PvEstimate
2017-10-04T23:00:00+00:00,PT30M,1679.63711162445
2017-10-04T23:30:00+00:00,PT30M,1749.67692615641
2017-10-05,PT30M,1763.28031333703
2017-10-05T00:30:00+00:00,PT30M,1882.10264424809
2017-10-05T01:00:00+00:00,PT30M,1621.47695667698
2017-10-05T01:30:00+00:00,PT30M,1822.57680296399
2017-10-05T02:00:00+00:00,PT30M,2022.02755621531
2017-10-05T02:30:00+00:00,PT30M,2090.16718724436
2017-10-05T03:00:00Z,PT30M,2167.9624822652
2017-10-05T03:30:00Z,PT30M,2022.38465598681
2017-10-05T04:00:00Z,PT30M,1786.75722920438
2017-10-05T04:30:00Z,PT30M,1531.05067625976
2017-10-05T05:00:00Z,PT30M,1272.47597804145
2017-10-05T05:30:00Z,PT30M,1052.31213981049
2017-10-05T06:00:00Z,PT30M,846.543370782682
2017-10-05T06:30:00Z,PT30M,629.5515419379
2017-10-05T07:00:00Z,PT30M,406.505091365806
2017-10-05T07:30:00Z,PT30M,187.223734840439
2017-10-05T08:00:00Z,PT30M,38.4564473592096


#19

BTW, it looks like we have an interesting day ahead in Canberra on the solar forecasting front. I now overlay both solcast and weatherzone-derived solar forecasts on my rolling 24 hour forecast timeline and you can see the slight differences below. I have noticed over the last few weeks that the solcast forecast only seems to get appropriately pessimistic within a few hours of forecast worse conditions whereas the forecast I derive from weatherzone data gets onto it a lot earlier. While the Solcast forecast generally ends up being more accurate when produced within a few hours of the conditions actually hitting, further out (6-24 hours) Solcast tends to be less accurate and overly optimistic than what I can forecast with weatherzone.

Solcast:
timeline_icons

Weatherzone-derived:
timeline_icons2


#20

Thanks for the example, I can see the issue and it is indeed only with our CSV endpoint (and JSV). This seems to have been caused by an internal change in how we store and retrieve short term forecasts.

Fix should be deployed later today. Would a change to the following interrupt how you are parsing dates? This does truncate the timezone completely unlike the JSON endpoints but is still compatible with ISO8601.

2017-10-04T13:30:00Z
2017-10-04T14:00:00Z
2017-10-04T14:30:00Z
2017-10-04T15:00:00Z
2017-10-04T15:30:00Z
2017-10-04T16:00:00Z
2017-10-04T16:30:00Z
2017-10-05T04:00:00Z
2017-10-05T04:30:00Z
2017-10-05T05:00:00Z
2017-10-05T05:30:00Z
2017-10-05T06:00:00Z
2017-10-05T06:30:00Z
2017-10-05T07:00:00Z
2017-10-05T07:30:00Z

#21

Thanks for this, nice find showing differences. I’ll pass this onto James and Nick to dig into. Work to improve our forecasts, both longer term and short are always ongoing, good to have an example with your PV output data. :+1: