Page 1 of 2

Weather Scraper: 5 Day Forecast

Posted: Sat Apr 22, 2006 5:59 pm
by Osler
Find below a script that will pull the 5 day forecast for your area from NOAA's mobile users website (US only):

Code: Select all

'Step 1: Create a Script Device called "WeatherForecast"
'Step 2: Go to NOAA's website (http://mobile.srh.weather.gov/), enter your zip code, and select "Detailed 7-day Forecast"
'Step 3: Cut and Paste the web address from the top of the "Detailed 7-day Forecast" page into the sURLCurrent line below
'Step 4: Save the script into the HouseBot Script file
'Step 5: Click on the "WeatherForecast" Script Device and point it to the script you just saved
'Step 6: Run the script from the Script Device - a message box will appear initially showing you the results (you can do this prior to adding all of the properties above to ensure it works correctly)
'Step 7: Open the "Properties Manager" and add Date1 through Date10 and DateForecast1 through DateForecast10
'===Name and Description should be the same
'Step 8: Add the properties you added to the script device
'Step 9: Read through the comments to remove the MsgBox debugging code (i.e., add a ' in front of it)
'Step 10: Read through the comments to activate the code to write the variables to the device properties (i.e., remove the ' in front of SetPropertyValue lines)



'This is a script to scrape current weather information from your local observation site

Dim sCurrent, oXMLHTTP, sURLCurrent, Parse1, Parse2
Dim sDate(9), sDateForecast(9)
Dim sDateOneline(9)

'Enter the corresponding web address for your area into the following variables from the website as described in Step 3 above
sURLCurrent = "http://www.srh.noaa.gov/port/port_mp_ns.php?select=2&CityName=Flower%20Mound&site=FWD&State=TX&warnzone=TXZ103"

'Create the xmlhttp object so you can use it to gather data from the webpages
Set oXMLHTTP = CreateObject("microsoft.xmlhttp")

'Get the current XML data from the Observation Location
oXMLHTTP.Open "GET", sURLCurrent, True
oXMLHTTP.Send

'Do some error trapping to ensure the server is up and all data was sent
If oXMLHTTP.readyState = 4 Then
     If oXMLHTTP.status = 200 Then
          'Save the MXL from the site as text so you can manipulate it as a string
          sCurrent = oXMLHTTP.responseText
     Else
          alert("There was a problem retrieving the current weather data.")
     End If
End If

'Destroy the object you created
Set oXMLHTTP = Nothing

'Now parse the data out of the file
Parse1 = 1

'Search for the days listed and the forecasts for those days
'Though a 7-day forecast is given, only 5 days are parsed

For I = 0 to 9

sDate(I) = ""
sDateForecast(I) = ""

Parse1 = InStr(Parse1, sCurrent, "<b>")
Parse1 = Parse1 + 3
Parse2 = InStr(Parse1, sCurrent, "</b>")
sDate(I) = Trim(Mid(sCurrent, Parse1, Parse2 - Parse1))

Parse1 = Parse2 + 7
Parse2 = InStr(Parse1, sCurrent, "<br><br>")
sDateForecast(I) = Trim(Mid(sCurrent, Parse1, Parse2 - Parse1))
sDateForecast(I) = Replace(sDateForecast(I), Chr(10), " ")

'Add a ' in front of MsgBox when you are satisfied the script is working correctly
MsgBox sDate(I) & vbCR & sDateForecast(I)

'Remove the ' from in front of all the SetPropertyValue below to have the script populate the device properties
Select Case I
     Case 0
          'SetPropertyValue "WeatherForecast.Date1", sDate(I)
     Case 1
          'SetPropertyValue "WeatherForecast.Date2", sDate(I)
     Case 2
          'SetPropertyValue "WeatherForecast.Date3", sDate(I)
     Case 3
          'SetPropertyValue "WeatherForecast.Date4", sDate(I)
     Case 4
	  'SetPropertyValue "WeatherForecast.Date5", sDate(I)
     Case 5
	  'SetPropertyValue "WeatherForecast.Date6", sDate(I)
     Case 6
          'SetPropertyValue "WeatherForecast.Date7", sDate(I)
     Case 7
          'SetPropertyValue "WeatherForecast.Date8", sDate(I)
     Case 8
          'SetPropertyValue "WeatherForecast.Date9", sDate(I)
     Case 9
          'SetPropertyValue "WeatherForecast.Date10", sDate(I)
End Select

Select Case I
     Case 0
          'SetPropertyValue "WeatherForecast.DateForecast1", sDateForecast(I)
     Case 1
          'SetPropertyValue "WeatherForecast.DateForecast2", sDateForecast(I)
     Case 2
          'SetPropertyValue "WeatherForecast.DateForecast3", sDateForecast(I)
     Case 3
          'SetPropertyValue "WeatherForecast.DateForecast4", sDateForecast(I)
     Case 4
	  'SetPropertyValue "WeatherForecast.DateForecast5", sDateForecast(I)
     Case 5
	  'SetPropertyValue "WeatherForecast.DateForecast6", sDateForecast(I)
     Case 6
          'SetPropertyValue "WeatherForecast.DateForecast7", sDateForecast(I)
     Case 7
          'SetPropertyValue "WeatherForecast.DateForecast8", sDateForecast(I)
     Case 8
          'SetPropertyValue "WeatherForecast.DateForecast9", sDateForecast(I)
     Case 9
          'SetPropertyValue "WeatherForecast.DateForecast10", sDateForecast(I)
End Select

Next
Osler

Posted: Mon Apr 24, 2006 11:06 pm
by bjlamarca
Thanks Osler, works perfectly.

Posted: Tue Apr 25, 2006 7:45 pm
by Osler
You're welcome. I'm glad you find it useful. It will "break" if NOAA changes the format of the website, but that can be easily remedied when/if it happens.

Osler

Posted: Tue Apr 25, 2006 10:20 pm
by bjlamarca
Is it possible to write code like this that will pull data from an RSS feed, which I assume is less likely to change its format?

Posted: Wed Apr 26, 2006 12:07 am
by Osler
Yahoo provides a rss feed for weather that I have scripted. The problem is that it is not as informative as that from NOAA and is not updated as frequently. It also only provides a 1-day forecast. In addition, Yahoo has not updated the output format in the developers area to what is currently being put out by accessing the feed (meaning its changed since the developer area was implemented....telling me that their format may not exactly be "stable" either)

I have also scripted the xml feed from my local NOAA weather station (KDTO) (could be used for any NOAA station). However, this provides no forecast information whatsoever.

I doubt that NOAA is going to change the mobile user format as it has to remain simple. Pull up the mobile user page you are accessing in IE and go to view -> source. Look over my string function code (InStr(), Mid()) and you can see the points in the page I am using to pull the data from by comparing the two. It would be relatively easy to alter the script in case of any future changes (unlike a plug-in).

If you have a rss/xml feed you would like to use, point me to it and let me see what I can do.

Osler

Posted: Wed Apr 26, 2006 12:38 am
by bjlamarca
Good to hear, as I have spent the last hour and a 1/2 adding to your code to include 'Current Conditions.' I'll PM the script when Iam done, if you approve, you can post.

Posted: Wed Apr 26, 2006 1:16 pm
by Osler
You have PM.

Posted: Mon May 01, 2006 5:08 am
by wallebalboa
Works great...
Do any know of a weatherservice that do this for europe (sweden)?
kind regs

Posted: Mon May 01, 2006 11:16 am
by Osler
NOAA has worldwide weather available on their site. Give me your exact location in Sweden and let me take a look at the source for NOAA's page. If its doable, I don't mind writing the script.

Osler

Posted: Mon May 01, 2006 11:47 am
by wallebalboa
My location is:
Västerås, Sweden (might be named Vasteras ...)
aprox 100 km west of stockholm
thanks for looking.. ( i have tried to find my location on NOAA with out success... :oops: )
kind regs

Posted: Mon May 01, 2006 8:31 pm
by Osler

Posted: Tue May 02, 2006 3:07 am
by wallebalboa
da cool place 8) !

can you get this in to your script?

kind regs

Posted: Tue May 02, 2006 6:05 pm
by Osler
Yep...when I get a free moment i'll write you a script.

Osler

Posted: Tue May 16, 2006 11:23 am
by Osler
Here is the script to pull Vasteras, Sweden current weather data from the NOAA website. Cut and paste the text into notepad and save it as <some_cool_name>.vb as all files into the HB script folder.

Code: Select all

'Step 1: Create a Script Device called "CurrentWeather"
'Step 2: Under "Settings", "Property Manager" add the following properties: CWLocation, CWTime, CWTemperature, CWHumidity, CWWindSpeed, CWBarometer, CWVisibility, and CWDewPoint
'====Property Name and Description should be the same


'This is a script to scrape current weather information from the NOAA website for Vasteras, Sweden

Dim sCurrent, oXMLHTTP, sURLCurrent, Parse1, Parse2, WeatherArray(5)
Dim sLocation, sTime

sURLCurrent = "http://weather.noaa.gov/weather/current/ESOW.html"

'Create the xmlhttp object so you can use it to gather data from the webpages
Set oXMLHTTP = CreateObject("microsoft.xmlhttp")

'Get the current data from NOAA
oXMLHTTP.Open "GET", sURLCurrent, True
oXMLHTTP.Send

'Do some error trapping to ensure the server is up and all data was sent
If oXMLHTTP.readyState = 4 Then
     If oXMLHTTP.status = 200 Then
          'Save the html from the site to the appropriate variable
          sCurrent = oXMLHTTP.responseText
     Else
          alert("There was a problem retrieving the current weather data.")
     End If
End If

'Destroy the object you created
Set oXMLHTTP = Nothing

'If nothing was downloaded from the site then exit
If sCurrent = "" Then Wscript.exit

'Now parse the data out of the file

'Search for the location
Parse1 = InStr(sCurrent, "Current Weather Conditions:<BR>")
Parse1 = Parse1 + 31
Parse2 = InStr(Parse1, sCurrent, "</B>")
sLocation = Trim(Mid(sCurrent, Parse1, Parse2 - Parse1))
SetPropertyValue "CurrentWeather.CWLocation", sLocation

'Search for the update time
Parse1 = InStr(Parse2, sCurrent, "<OPTION SELECTED>")
Parse1 = Parse1 + 17
Parse2 = InStr(Parse1, sCurrent, "<OPTION>")
sTime = Trim(Mid(sCurrent, Parse1, Parse2 - Parse1))
SetPropertyValue "CurrentWeather.CWTime", sTime

For I = 0 to 5

Parse1 = InStr(Parse2, sCurrent, "<TD><FONT FACE=" & Chr(34) & "Arial,Helvetica" & Chr(34) & ">")
Parse1 = Parse1 + 33
Parse2 = InStr(Parse1, sCurrent, "</FONT>")
WeatherArray(I) = Replace(Trim(Mid(sCurrent, Parse1, Parse2 - Parse1)), Chr(10), "")

Next
 
SetPropertyValue "CurrentWeather.CWTemperature", WeatherArray(2)

SetPropertyValue "CurrentWeather.CWHumidity", WeatherArray(4)

SetPropertyValue "CurrentWeather.CWWindSpeed", WeatherArray(0)

SetPropertyValue "CurrentWeather.CWBarometer", WeatherArray(5)

SetPropertyValue "CurrentWeather.CWDewPoint", WeatherArray(3)

SetPropertyValue "CurrentWeather.CWVisibility", WeatherArray(1)
Enjoy,

Osler

Posted: Tue May 16, 2006 4:27 pm
by wallebalboa
Great tnx!
most of the time it works great, sometimes i get:

get error:0 'Object required: 'Wscript"

i think its some problem with this...

'If nothing was downloaded from the site then exit
If sCurrent = "" Then Wscript.exit
""


its pitty that noaa.gov dont have a 4cast for vasteras... :)

i need to do some extra parsing to get the metric units only...

this script will give me alot of ideas to get other "web" data in to HB :)

tnx again /AW