Panel is led by a group from the California Watch, and it’s about how to get data and documents from the state of california.
Lance Williamson, Mark Catches, Chase Davis, Emily Calhoun (maplight.org),
Lots of data that you can get in other places, but not CA. CA doesn’t provide open access to crime data, for example.
We have a right to, as citizens, to much of the data produced by the government. However, it’s not always easy to get the actual data – you have to ask, you have to negotiate, and sometimes you have to litigate.
CFAC.org – California First Amendment Coalition. They are a good source of information and tools.
Some steps:
- Identify the person who has the data, and ask them – maybe they’ll give it to you.
- Formulate your request. Be as specific as possible about what you want.
- Write a letter. There are templates for this in different places. There are certain legal
- The agency decides whether to fulfill the request.
- If they don’t, you can litigate.
You are entitled to data in computer form, if the records are already computerized. Otherwise, they are not obligated to give it to you in computer format.
If you get back something that’s a scanned PDF, are there OCR tools to read it? Yes, but they are not perfect. Google Teseract (?).
Maplight.org sued to get legislative data from Sacramento. But this is expensive. If you’re just a blogger, and you need a lot of data, what do you do? Blog and complain and shame them into getting the information.
Best way to get data: Go straight to the IT department. They know how to get it, and don’t care so much about the politics (and a personal note: They are also probably most aware of the rules for how the data is supposed to be given out.)
It can take a really long time to get access to the data – whether through ignorance or malice, it takes a while to negotiate with the agency, and sometimes they will ask for money.
Municipal governments, especially now, are looking for money so they will try to charge for this information.
“When you meet resistance, and you meet continual resistance, you gotta document it.” – Lance Williamson.
At the local level, sometimes the data isn’t really available in any useful formats.
The California Public Records Act: gives us the right to inspect public records.
You can make a request for all public data sets that they keep in a county or a city. Start w/ the city manager, or county executive. Also, you can make a request for all public records requests made.
How do you ask for stuff?
You have to be specific for date, period covered, which agencies are impacted. You have to describe it in specific enough terms, but in general enough terms to prevent a refusal on a technicality.
You can ask for the email correspondence between city council members and their staff.
It helps to find someone who can help you explain how the data works. Also, specify file formats if you are asking for electronic records (e.g. delimited text, Excel) otherwise you are liable to get data in a weird format.
DocumentCloud – a Knight Foundation project that collects, in a central repository, documents and data sets that were requested and retrieved.
You have the right of inspection, because of the California Public Records Acts. You may inspect the actual records, at no charge, in the office where it is stored.
A common argument you hear when requesting data: “This data is proprietary to a corporation”. You can talk to other people who are working in the field, or has requested this type of data before. Academics can be pretty helpful.
Sometimes there are reasons that you can’t get data online, but you can get it when you download it (e.g. Street Addresses for campaign contribution data).
Is there a netwrk of technical experts that reporters and other people looking for data can tap? Profnet is available from the PR Newswire. You can post and ask for help contacting certain types of people. Get people interested in your project, and try not to burn the people that are helping you.
If you don’t know how to do data analysis, you can get a “data buddy” (@agahran) – or learn SQL, computer-assisted reporting skills, etc. You can also find universities where there is a data analysis program, and get help from a student. NICAR: National Institute for Computer-Assisted Reporting.
You can look for these types of courses at community colleges, Academy X in SF, other places where they teach computer stuff.
Note: Please let me know if there are typos and other gremlins in this post. I wrote and posted in a hurry.
Add to Google
