When the Data Finally Matches the Work: Building Evaluation Capacity across a Civic Engagement Network
The Starting Assumption
Houston is one of the most diverse cities in the country and one of the hardest places to sustain civic participation across a fragmented, under-resourced landscape. For nearly a decade, a network of almost 100 nonprofits had been doing the slow, relational work of civic engagement: mobilizing voters, building power, organizing communities. Anecdotally, we knew it was working. You could see the relationships deepening, the coordination improving, the activity compounding. And yet, we could not say, with any confidence, whether it was working. That gap between what we believed and what we could demonstrate was where this work began.
The driving metric is, and always has been, voter turnout. Funders, partners, and the network itself needed to know whether all of this organizing and mobilization was actually moving people to the polls. But proving that connection requires more than counting events or contacts. It requires being able to trace a path from a specific outreach effort to a specific voter showing up on election day. Without that, the work remains anecdotal; compelling to those inside it, invisible to everyone else.
The Challenge
The reason came down to data. Most partners were small, community-rooted organizations without dedicated data staff. Some tracked participation on paper. Others submitted aggregated summaries (total contacts, total events) that confirmed activity happened but couldn't tell us much else. Two problems, each unique enough to require different solutions. The first was foundational. The second was more specific, and it shaped everything else that followed.
Moving to collecting individually-identifiable data isn’t just a technical issue. It’s a relational one. The communities at the center of this work had historically (and currently) good reasons to be cautious about giving their information to anyone. Trust couldn’t be assumed. It had to be built into the infrastructure itself. You can't ask communities with legitimate reasons to distrust data collection to simply trust you because your intentions are good. The goal I kept coming back to: improve data quality without eroding community trust. Those two things had to move together.
Designing with Intention
One of the first decisions was whether to build this capacity in-house or bring in a dedicated partner. It wasn't a simple call. There were real debates about which direction made more sense. Ultimately, what drove the decision was a desire to protect what the team was already providing partners, direct campaign support. Adding evaluation capacity building to their plate risked diluting the support organizations were already counting on. Keeping it separate meant both could be done well.
The RFP did as much framing work as it did scoping work. I named our evaluation values explicitly: participatory methods, equity at the center, data as a tool for learning rather than a compliance requirement. Prospective partners were held accountable to those values during selection. The document was setting a tone, not just defining a scope.
Out of all the proposals and interviews, Pivot Data Design was the right fit. Before any curriculum was built, we both wanted partners to have input into the final product. Pivot spent six months getting to know the team and the network conducting partner interviews, attending meetings, learning how organizations actually operated rather than how they describe themselves on paper. It was important for the curriculum to be shaped by the people who would actually live with it. If we skipped that phase and went straight into instruction, we’d build something technically sound that nobody trusted or found useful. Relationships before curriculum wasn’t a soft preference. It was how we protected the integrity of the whole program.
Alongside the program design, I invested in the data infrastructure and community trust work that would help make meaningful collection possible, but that work deserves its own telling.
The program that launched in June 2023, Harnessing the Power of Data, was designed so that learning happened in multiple directions at once.
The design was deliberate. Each component was designed and reinforced the other. What happened next showed whether the logic held.
Observable Shifts
Within the first year, we felt a noticeable shift across the network. The gains were clear, and visible in multiple ways:
- Data quality rose 32 percentage points, from 55% to 87%.
- Near-universal data submission replaced a baseline where many organizations had submitted nothing at all.
- 97% of participants found the workshops beneficial.
- Team enthusiasm and confidence in working with data increased significantly.
- Data collection became more streamlined and efficient across organizations.
- Partners improved how they communicated their impact to the communities they serve.
These weren’t isolated wins. They were network-wide shifts that showed the design was working. The more important change was in how the data was actually working for the organizations, not just the funders.
Behind the network-wide numbers were individual organizations doing something different. Two organizations shared what the shift looked like in practice.
Alief Votes, a young organization with big ideas, came into the program with energy and ambition. Through the coaching process, they rebuilt their logic model and honed in on their theory of change. Youth voter turnout targets weren't just met; they were exceeded by more than 700 percent. "When we got the data from that, we were like, wow, maybe we're doing something right…now we’re seeing, OK, how can we look at the data, capture this moment that we’ve activated, and then start to think of different types of ways to engage either returning voter or new voters." said Abby Triño.
For Bridges to Empowerment, the conceptual shift was as meaningful as the technical one. "A logic model is basically a power map," said Koretta Brown, describing how her organization came to see data not as a funder requirement but as a tool for telling their own story. They began producing videos anchored to their data: more focused, more pointed, more clearly connected to outcomes. And now the organization has a formidable database of community members and voters with whom they have established relationships it can continuously engage through meetings and community events.
This is data functioning as it should: not as a reporting requirement, but as a vehicle for community voice and organizational power.
Lessons to Carry Forward
The case for evaluation capacity building is easy to make in the abstract. What's harder is getting specific about what it actually requires, and honest about what it can and can't solve.
1. Start with the accountability question, not the training question. What does your organization actually need to know, and what infrastructure does that require? For us, the voter file matching requirement was specific and consequential. Naming it clearly shaped everything downstream: what curriculum Pivot built, what standards I set, what I asked of partners. Vague evaluation goals produce vague results.
2. Treat the RFP as a design document. The values you articulate in the scope of work set the tone for the entire engagement. I wrote our evaluation principles into the RFP (equity, participation, data as learning rather than compliance) and those principles shaped who responded and how the work unfolded. That document was doing more work than I realized when I wrote it.
3. Trust and data quality are not competing priorities, but they require separate interventions. Improving data standards without attending to community trust is a shortcut that tends to backfire. Organizations need to be able to collect better information in a way they can honestly defend to the people they serve. That work isn't glamorous, but it's load-bearing.
What didn't change is also worth naming directly.
Knowledge and skill are one type of organizational capacity. Time, budget, and staffing are another. This program excelled at addressing the first. The second remains a structural gap. Many organizations now understand what good data looks like; not all of them have the personnel to sustain it without continued support. That's not a failure of the program. It's the limit of what a training investment can accomplish without a corresponding investment in human capital.
When a backbone organization sets a clear standard and then invests seriously in helping every organization in the network meet it, the whole network rises. Not because reporting improves, but because organizations learn to use evidence on their own terms, to understand what they're producing, to adapt, and to tell their story with the kind of clarity that builds power.
