abstract image of ryan

Welcome to ryanwold.net


Scraper Wiki'ing

Yesterday, I really dug into ScraperWiki, creating 4 scrapers in about an hour.

The premise of ScraperWiki is simple: Write small snippets of programming code that read a web page and make it accessible in a standard, web-friendly format.

Tonight, I was able to pull data using Google Spreadsheet's API's through Scraper Wiki.

The CA Assembly Roster is now available in a nice standard formats, thanks to ScraperWiki. As are the NACO's findings for CA, MN, and NY.

ca-assembly ca-counties counties-mn counties-ny google-spreadsheet via Google Spreadsheets Sample Output (json) generated by ScraperWiki

Note:

Consider using Google Forms, providing input mechanisms for all sorts of data. Potentially very powerful.


Date
2011-02-22
Status
draft

Tags
crowdsource, data, nokogiri, opendata, python, ruby, scraperwiki, and scraping
Series
N/A