For a while now I have been gathering civic data by hand.

One city at a time. Find the departments page. Find the council. Find who holds each seat. Find what a resident can actually do: pay a bill, get a permit, report a pothole.

It is good work. It is also slow. And a lot of city websites block anything that is not a person with a browser, so I end up copy-pasting pages.

This week I tried a different approach.

---

Common Crawl visits a big chunk of the public web every month and publishes it, free. City websites are in there. Not all of them, but a lot.

So the question became: how much of local government is already sitting in Common Crawl, and what would it cost to pull it out?

I started from the federal list of .gov domains and kept the local ones. Cities, counties, school districts, special districts. 13,732 domains.

Then I looked them up in one month's crawl.

About a third of city domains were there. About 1.1 million pages. About 30 GB.

Finding all of that cost 77 cents.

I keep coming back to that number. Hand gathering costs hours per city. This costs pocket change per month, and nobody's web server gets hit twice.

---

The pages are only half of it.

The other half is turning a page into something structured. This department, this phone number, this council member, this service.

I am doing that part on my laptop, with a local language model. No cloud, no API bill, and the data does not leave the machine.

The model is fast enough and pretty good. It also makes things up. On the first real page I tried, it invented a department name and paraphrased every description.

So the model does not get the final word. It suggests. Plain code checks every value against the page, and anything not actually on the page gets dropped. The invented name: dropped. The paraphrases: dropped.

What is left is smaller, and true.

---

There is still plenty to do.

The model thinks some menu links are services. I need a set of cities I have checked by hand, to measure how often it is right. The full copy of those 30 GB still has to run.

And two thirds of cities are not in Common Crawl at all, so hand gathering is not going anywhere.

But the shape of it feels right. Public pages, from a public archive, turned into records that each point back to the page and date they came from. Same inputs, same outputs, every time.

That is the kind of civic data I want to build on.