ryanwold.net

A civic-minded citizen seeking the singularity

An entry

Scraper Wiki'ing

Date: 2011-02-22
Status: draft
Tags: crowdsource data nokogiri opendata python ruby scraperwiki scraping

Yesterday, I really dug into ScraperWiki, creating 4 scrapers in about an hour.

The premise of ScraperWiki is simple: Write small snippets of programming code that read a web page and make it accessible in a standard, web-friendly format.

Tonight, I was able to pull data using Google Spreadsheet's API's through Scraper Wiki.

The CA Assembly Roster is now available in a nice standard formats, thanks to ScraperWiki. As are the NACO's findings for CA, MN, and NY.

ca-assembly ca-counties counties-mn counties-ny google-spreadsheet via Google Spreadsheets Sample Output (json) generated by ScraperWiki

Note:

Consider using Google Forms, providing input mechanisms for all sorts of data. Potentially very powerful.