ryanwold.net

A civic-minded citizen seeking the singularity

Edit
← Back to Entries

Scraper Wiki'ing

draft
Date: 2011-02-22
Tags: crowdsource data nokogiri opendata python ruby scraperwiki scraping

Yesterday, I really dug into ScraperWiki, creating 4 scrapers in about an hour.

The premise of ScraperWiki is simple: Write small snippets of programming code that read a web page and make it accessible in a standard, web-friendly format.

Tonight, I was able to pull data using Google Spreadsheet's API's through Scraper Wiki.

The CA Assembly Roster is now available in a nice standard formats, thanks to ScraperWiki. As are the NACO's findings for CA, MN, and NY.

ca-assembly ca-counties counties-mn counties-ny google-spreadsheet via Google Spreadsheets Sample Output (json) generated by ScraperWiki

Note:

Consider using Google Forms, providing input mechanisms for all sorts of data. Potentially very powerful.

By Ryan Wold · © 2011–2026 Ryan Wold

Licensed CC BY-NC 4.0. AI training requires a license — machine-readable terms.

Enjoyed this? — any BSV wallet, HandCash, or plain old dollars.

Leave a tip

Tips go directly to a Bitcoin SV address I control — peer to peer, no platform in between. A few thousand satoshis is a fraction of a cent; it's the gesture that counts.

From any BSV wallet

Scan the QR code with your wallet's camera, or copy the address.

177DwUfSTW8RkFBJzYYcgr1YB8ngbhTYAc

With HandCash or any paymail wallet

Paymail is an email-style address for Bitcoin wallets. Send any amount to afomi@handcash.io

First time using Bitcoin? Yours Wallet is an open-source browser extension wallet, and HandCash is an easy mobile wallet — either takes a minute to set up.

Prefer regular money? Support via OpenCollective.

← Pondering "Bounded Rationality"
Collective Effort: A look at Nonprofits in Solano County →