At Searchlight New Mexico, we believe open data can revolutionize civil society.
We're champions of this view, and we're far from alone. Journalists, businesses, public agencies and the informed public increasingly use open data to facilitate public participation in government, to connect residents with services, to measure policy impacts and to optimize economic development.
Regrettably government data sets in New Mexico – even on such critical topics as public spending — are far from open. Searchlight New Mexico has decided to open these data sets ourselves, through the Your Data portal.
We believe the first and most basic tool that citizens deserve is a clear idea of how their tax dollars are being spent by the representatives they elect and the public employees who implement policy.
We believe that before citizens can make meaningful decisions about whether their government is doing its job adequately, they deserve a clear and unfettered view of how their tax dollars are being spent. Everyone must know how much and why, how costs change over time, and how New Mexico’s state agencies, and county and local governments, compare to their peers.
Public data should be open. Access should be easy, free from restrictions on use and available in a machine-readable, non-proprietary electronic format. We have found that when datasets are locked into display formats, such as the commonly used PDFs, they become useless for meaningful analysis.
Simply put, a government agency that creates data and publishes solely in a non-open format is deliberately preventing the data's owner, the public, from gaining meaningful access. In a democracy, to wrap public records in inaccessible formats is to steal the public’s right to know.
Giving public data back to the public
Searchlight New Mexico's Your Data portal allows everyone — members of the public, journalists from every news organization, researchers and public officials — to search, compare and download data on a host of topics surrounding public spending. Our goal is to have data covering as many state and local governments and agencies as we can provide. These datasets will be continuously expanded and updated. They are free from any restriction on access or use.
The portal currently contains:
- Salary data for employees of 11 of New Mexico’s 33 counties, comprising 1.2 million New Mexicans, or about 61 percent of the state's population. Most counties reported data going back to 2014.
- Salary data for employees of 16 of New Mexico’s largest cities and towns. Those municipalities together are home to 1.1 million people.
- Salary data for all employees of the executive, legislative and judicial branches of state government, since 2013, as well as for employees of the state's public schools.
- Pension contributions from government and school agencies on behalf of public employees and educators, most going back to 2013.
- Financial disclosures for those public employees who are required by statute to file them, from 2013 to 2017.
Our journalists acquired the data through dozens upon dozens of requests under New Mexico's Inspection of Public Records Act (IPRA). We then tidied each dataset ("tidy data" is a term of art, see this reference for more about it) and innumerable hours jailbreaking data that certain municipalities chose to lock into PDFs or similar formats that restrict data analysis.
The data portal will continue to grow as we acquire and tidy data from more and more public agencies. We intend for our coverage to be universal — payrolls for all counties and all municipalities — and to add new data annually, as well as new types of data frequently. When we publish an update, the portal will show the additions in the What's New box on its homepage, and we will alert readers through social media and through our free email alerts.
How to use it and what it's good for
We designed the portal to open up data on public spending to the people, giving you the ability to see where your government dollars are going, to compare spending between governments and to ask meaningful questions of your public officials. Here's an overview of how it works, and an example of how you might use the data.
First things first. On the portal homepage, you will see menus at the top that point to data on payrolls, pensions and financial disclosures, as well as a box with what's new in the database, and a series of links at the bottom that explain more about the data and give answers to frequently asked questions.
Each top menu has a dropdown that performs an initial slice on the larger database. For example, under Payrolls there are options for state, municipal, county and school agencies. Clicking "counties" opens the table with all counties selected for the current year.
To the left, you can choose several ways to refine your query, including by the specific agency, by year and by job title. You can also make multiple selections. The table updates itself every time you make a change. You can also click the fields above the table itself to sort the results — total pay, for instance — either ascending or descending.
Each entry in the table has a plus sign to the left. Clicking on that expands the entry, showing additional fields like a person's title, type of employment and rate of pay. More about what the different fields mean is posted on the Data Notes page.
The real power of the data portal lies in making comparisons through time or across agencies. Here's an example of how you could use the data to answer a specific question: How has Doña Ana County's payroll changed between 2014 and 2017?
From the home page, choose Payrolls, then Counties. Expand the Agency field at the left of the table, and start typing Doña Ana, then click the box that appears with the county's name.
Now the table shows Doña Ana's 2017 employees, sorted by total pay. Above the table, you should see the following:
You can see it has 793 employees, paid a total of $47,383,274, and the highest-paid employee was Julia Brown, the county manager. This raises an important point about the Your Data portal: It contains point-in-time data. Brown's contract was terminated in April, but the county's response to our open records request included her because she was on the books at the time. We report data exactly as we receive it from public agencies. When we request the data next year, they will reflect Doña Ana's payroll for the new year.
Now, expand the Year menu at the left, uncheck 2017 and check 2014. You should see this above the table:
So the answer to how Doña Ana County's payroll has changed from 2014 to 2017 is 95 more employees and about $9.2 million more in salaries and benefits.
Your Data gives you the tools to answer questions like this. Journalists from all over the state may use it for data when writing stories about your state and local governments. We also hope members of the public use the portal to inform each other as they engage with others in your community and with your elected representatives.
Be sure to let us know what you're doing with it, and how we can make public data more useful to you. After all, you own it.