A key concept behind DemocracyMap is the ability to lookup your entire government in one search. Thus, being a canonical (complete and up-to-date) source of information requires periodic updates. This is how I'll go about it: Source File -> Temporary Table --> Canonical Table -> Archive The process goes like this: A source file is downloaded, like irs/pub78.zip. The data is opened, formatted, and inserted into a dynamically created temporary table (or NO SQL store). The temporary table (or data store) attempts to insert in the canonical. If it exists. Check for update. Return add|update The canonical table then gathers all the records that were not touched and pushes them to Archive, and deletes them from active. Use Case: An ever-changing data source is available online. I'd like to develop a service that uses the data and adds both value and context to it. So, I'll need maintain my value-added data while refreshing with new data and expiring data.