Blog
Dive into our public learnings on building community, process, and technology for civic society.
Modernizing our mapping stack with Maplibre
A significant reason for website failures is their reliance on external platforms, which frequently change or may even be shut down. However, a promising development in the past few years has been the emergence of GL-powered maps that operate entirely in the front-end browser. As a result, we have adopted Maplibre-gl as our preferred choice for data-rich maps and are in the process of transitioning many of our projects to this technology.
Our small team created a shared strategic vision–and you can, too!
In April 2025, the DataMade team gathered in Chicago to discuss and define strategies for the first year in a new era for our company. In two days, partners, mid-level staff, and junior staff collectively created and prioritized a list of specific goals and planned how we’d pursue them in the next year. For other teams interested in setting and executing strategic work in a way that’s inclusive of all experience levels, here’s what worked for us.
A new era for DataMade and Forest
Over the past twelve years, it has been my great privilege to help DataMade grow to an established, successful company and an important actor in Chicago’s civil society. At the end of January 2025, I will be leaving this mighty and hearty company. As I leave for my next endeavor, a reflection on lessons learned at DataMade.
Join the DataMade team! (Application closed)
Do you want to build mission-driven technology? Do you want to engineer fresh ideas about civic tech? Do you want to extract meaningful insights from data? We’re looking for a full-time, Chicago-based developer to start in August or September 2024.
Towards a less harmful data practice: The DataMade way
Data can be a powerful tool for illuminating our world, but it can also reproduce bias, perpetuate harm, and subvert truth. A significant amount of our work at DataMade involves making data available, accessible, and intelligible; we therefore must take steps to minimize the biases, harm, and fallacies that can come from utilizing data irresponsibly.
Join the DataMade team! (Application closed)
Do you want to build mission-driven technology? Do you want to engineer fresh ideas about civic tech? Do you want to extract meaningful insights from data? We’re looking for a full-time developer to start in September or October 2022.
Join the DataMade team! (Applications closed)
Do you want to build mission-driven technology? Do you want to engineer fresh ideas about civic tech? Do you want to extract meaningful insights from data? We’re looking for a full-time developer to start in December 2021 or January 2022.
Working against systems of oppression
Last summer, DataMade made a commitment to examine the ways in which our work has supported systems that destroy Black lives, particularly policing and incarceration. As we wrote then, this is slow work, and much of it is quiet work rooted in reflection, study, and relationship building. A year later, we’d like to share an update on the steps we have taken so far and reaffirm our commitment to continue working to redirect power from oppressive systems to those that support democracy, justice, and equity for all.
Hannah Cushman Garland is now a Partner at DataMade
Today we’re happy to announce that Hannah Cushman Garland has joined Derek Eder and Forest Gregg as a Partner and co-owner of DataMade. Hannah joined the DataMade team as a Developer in 2016. An alumna of the Missouri School of Journalism and a veteran of the Associated Press, she remains deeply interested in people, systems, and power. She brings a healthy skepticism to technology, believing that it can alleviate specific pain points but is rarely a solution in and of itself.
Why We're Switching from AWS to Heroku
After five years of maintaining a custom deployment framework on Amazon Web Services, the DataMade team is switching to deploying containerized Django apps on Heroku. Here’s why and how we decided to switch.
Accounting for our part in supporting systems of oppression
In this time of grief and uprising, we have been examining the ways that we, as a company, have supported systems that destroy Black lives, and what we can do to meet this moment of possibility. It is slow work and much of it should be quiet work. Over the past ten years, we have come to know that policing in America is not compatible with a healthy, equitable democracy; that policing in America destroys precious Black lives like George Floyd, Ahmaud Arbery, Breonna Taylor, Rekia Boyd, Quintonio LeGrier, Bettie Jones, Harith Augustus, and Laquan McDonald.
DataMade’s response to COVID-19
In response to the COVID-19 outbreak, the DataMade team is now 100% remote and practicing social distancing. We plan on maintaining this setup for the duration of the outbreak and will keep our office operating at full capacity. Here are a few resources for transitioning to remote teams and resources and actions to slow the spread of COVID-19.
Introducing Civic Exchange Chicago
Today, I am proud to announce the founding of a new civic venture! Civic Exchange Chicago is a coworking space and learning community centered on how news, information and technology can increase democracy and freedom.
Mapping Twenty Years Of CPS Closures During The DataMade Mentorship
Jay Yugen worked with twenty years of Chicago Public School data to begin creating choropleth maps illustrating public school closures, in the context of their demographics and that of their surrounding neighborhoods.
How to put crontabs under version control
DataMaders keep loads of things under version control – including a seemingly elusive part of scheduling infrastructure: crontabs. This post explains how we version crontabs and why.
Building Live Healthy, West Side During the DataMade Mentorship
During the mentorship program Karen Aguirre created LiveHealthyWestSide.com, a website that showcases the work of West Side United and the health indices of 10 Chicago communities.
Apply for the inaugural DataMade Mentorship Program
The DataMade mentorship is a six-week, one-on-one mentorship program for beginning technologists and the technologically curious working to develop a specific skill or address a specific problem. We’re accepting applications for the first cohort through October 25, 2018!
Transactional Testing with Pytest and Flask-SQLAlchemy
In this blog post we’ll cover DataMade’s approach to testing highly-stateful web apps like Dedupe.io, and introduce a new open-source plugin that we’re releasing to make it easier to test stateful Flask-SQLAlchemy apps using pytest: pytest-flask-sqlalchemy-transactions.
Introducing the Chicago Data Collaborative
DataMade is thrilled to take part in the launch of the Chicago Data Collaborative, a cooperative effort by civil society organizations to share capacity with the goal of strengthening local criminal justice reporting and advocacy in Chicago and Cook County.
Modern Approaches to Schema Matching
In this memo, we review the literature on schema matching—a set of principled approaches to automatically detecting what kinds of data are contained in databases of enormous size and uncertain format.
Existing research tools make it difficult for teams to pursue investigations when source material is uncertain or contradictory. This whitepaper outlines the requirements for a knowledge system that can manage ambiguity during investigations and produce useful, structured data as a byproduct.
WhoWasInCommand makes it harder for human rights abusers to hide
This week we’re thrilled to launch WhoWasInCommand, a fully auditable dossier for tracking security force activity around the globe that we built for the Security Force Monitor.
Councilmatic arrives in Los Angeles
DataMade launches a new version of Councilmatic for the Los Angeles County Metropolitan Transportation Authority to help the public keep track of their Board of Directors, meetings, and reports.
DataMade launches an email subscription service on Councilmatic.
Get the lead (and the word) out with City Bureau
We extracted lead test results from hundreds of PDFs so you don’t have to. You saw them in a special issue of South Side Weekly guest edited by City Bureau. Now, the data is yours to explore.
This whitepaper describes technical measures that can be used to protect data about immigrants in a municipal ID program.
Ain’t No Party Like a README Party
This holiday season, we read and revised several of our most popular READMEs – making the open-source world a happier, more festive place to be.
Brand new tools for analyzing community-level data
This week, DataMade is excited to announce three new additions to our open-source toolkit for developers. We’ve added libraries for dealing with data on crime, schools, and demographics in Illinois and across the United States.
Following the money in New Mexico politics just got a whole lot easier
We’re happy to announce the launch of The Openness Project with our partners New Mexico In Depth. Based on campaign finance reports collected by the New Mexico Secretary of State, The Openness Project lets you search and browse all political contributions, expenditures, and candidates in statewide and local New Mexico races.
We find your reps so you don’t have to
My Reps is a tool for looking up your elected representatives across the United States. Based on your address, we can find all the federal, state, county, and local officials who represent you in government. We built it to be completely open source, easy to repurpose, and easy to customize by political organizations.
Today, DataMade is launching something new. It’s not a map, or an app. It’s a product called dedupe.io. And it will transform your messy data into something much more useful.
This week, DataMade will be in Denver for NICAR 2016, a yearly conference hosted by Investigative Reporters and Editors (IRE) on data journalism. We’ll be presenting on the work we’ve been doing and sharing our open source tools for working with, cleaning up and visualizing data. News nerds, look for us in Denver! We’d love to talk to you!
Civic Tech Done Right – Community Driven, Community Responsive, Community Friendly
Neighborhood organizers and software developers don’t ordinarily move in the same circles. But in Chicago, thanks to some tool-hungry community organizers and neighborhood-oriented software developers, a couple of computer applications have emerged that are enabling neighborhood residents to play a greater role in improving their communities. This is a story, told from the perspective of three people, of how and why those applications were developed.
Demystifying Chicago politics with Councilmatic
Thanks to the support of the Sunlight Foundation, I am proud to announce that DataMade is launching a brand-new version Chicago Councilmatic. Track all things related to Chicago City Council.
Introducing: Illinois Sunshine
We teamed up with the Illinois Campaign for Political Reform to build Illinois Sunshine, a tool for browsing candidates, political committees, donations to campaigns and expenditures by campaigns in Illinois going back to 1994. Campaign finance data in Illinois has never been more accessible.
Better Living Through Documentation
Effective technical communication shouldn’t be an afterthought; it’s crucial to open source development. Here are 20 practical tips.
Parse names and parse ... anything, really
Based on the success of usaddress, The Atlanta Journal-Constitution and DataMade are happy to announce two new parser tools in our collaboration: probablepeople - a parser for person names and parserator - a toolkit for making your own probabilistic, domain-specific parser.
Parsing addresses with usaddress
Last week, the Atlanta Journal Constitution (AJC) announced the first public product from our collaboration - usaddress, a python library for parsing US addresses using advanced natural language processing (NLP) methods.
DataMade at the Code for America Summit
The Code for America Summit, a yearly open government and civic tech conference, was held in San Francisco last week. DataMade was able to share two of our client’s stories to this international community of government leaders, funders, policy experts, and technologists.
One of the wonderful things about civic technology is that no one really knows what it means. ‘Civic technology’ covers varying types of projects made by a variety of people trying to do very different things. This fuzziness is useful. It challenges civic technologists to think independently about what exactly they are trying to do, while providing a wide set of projects and models to steal from or define themselves against. All the same, we can begin to identify four major intentions of civic apps.
DataMade and the Illinois Campaign for Political Reform (ICPR) are launching a new app today called Election Money. It provies bulk downloads of campaign finance data from the Illinois State Board of Elections.
LargeLots.org is back, this time in East Garfield Park! If you own land in this west side Chicago neighborhood, you can buy up two vacant lots from the City of Chicago for $1 each.
The City of Chicago is selling empty lots to South Side residents for $1 until April 21st, 2014 as part of their neighborhood stabilization initiative. To lower the barrier to apply for one of these ~4,000 lots and increase the number of potential applicants, we made an app to help residents find eligible lots and navigate this process - largelots.org.
Introducing Spreadsheet Deduper
DataMade is launching a free online tool for de-duplicating spreadsheets.
Today, we are excited to announce a major update to the dedupe library! The new features include parallel processing support, improved record linkage across files, and a new asynchronous architecture.
In November, we got an e-mail from Triway International Group asking DataMade to share our open data work and what’s been going on with the Chicago Open Government community with 17 delegates from the National Bureau of Statistics of the People’s Republic of China.
Making a Living with Civic Tech
For the open government movement to sustain itself and continue to grow, we need to build businesses off of projects that emerge from our civic technology community.