Data-Driven Policing Guide: How to Evaluate Technology Before Your Agency Wastes Six Figures

Estimated reading time: 14 minutes

In 2020, Chicago discontinued its Strategic Subjects List after years of criticism. The algorithm was supposed to identify people at highest risk of violence, but community groups and researchers found it couldn’t be adequately explained, disproportionately targeted Black residents, and became a flashpoint for community trust issues. The city had invested significantly in the system, but ultimately scrapped it.

Chicago isn’t alone.

Across the country, agencies are discovering that data-driven policing is far bigger—and far more complicated—than the sales pitch suggests. It’s not just about crime maps anymore. Social media monitoring tools are detecting rising community tensions days before they boil over. Surveillance networks are tracking suspects across entire cities in real-time. Technologies like gunshot detection systems are changing how agencies respond—though the evidence on whether they actually work is mixed at best.

Here’s the reality: Understanding the full landscape of data-driven policing isn’t just helpful anymore. It’s how you avoid becoming the next department that spends six figures on technology that gathers dust or, worse, damages the community trust you spent years building.

The Study: Taking Stock of Data in Policing

Researchers Muhammad Afzal and Panos Panagiotopoulos recently published a comprehensive systematic review examining how data is being used across law enforcement worldwide. They analyzed 192 peer-reviewed studies spanning five decades (1970-2022) to understand not just what data tools exist, but how they’re actually being integrated into core policing functions and whether they’re working.

What they found reveals a policing landscape in the midst of a massive transformation. One that offers real promise but also carries significant risks.

Four Ways Agencies Are Using Data (Beyond Crime Mapping)

The researchers identified four distinct data-driven processes. Here’s what each does and what the evidence says:

1. Crime Construction: Seeing Patterns to Prevent Crime

What It Does:

• Hotspot policing: Identifying high-crime areas using crime reports and calls for service
• Criminal profiling: Calculating “harm scores” to identify potentially high-risk offenders based on factors like offense severity, sentence length, and time since last offense
• Predictive mapping: Using machine learning to forecast crime in micro-locations

Real-World Example: Agencies identify hotspots, then deploy interventions like environmental design changes (adding lighting, removing graffiti), increased patrols, or community outreach.

The Evidence: Mixed. Hotspot policing has solid evidence behind it, when done right, it works. The research base is strong. But predictive mapping and criminal profiling? The results are all over the map. Some studies found no crime reductions whatsoever. Others found weird patterns where violence dropped but property crime increased, or vice versa. The multi-pronged interventions (combining enforcement with environmental changes and community engagement) showed better results than enforcement alone.

2. Crime Sensing: Listening to Communities and Social Media

What It Does: Mines community intelligence from public interviews and social media to detect early indicators of crime and disorder such as graffiti, littering, verbal harassment.

Real-World Examples:

• Researchers found that social media posts could be correlated with offline incidents of burglary, criminal damage, and violence
• Integration of police records with social media data helped identify patterns of human smuggling and drug trafficking

The Evidence: Promising for detecting community tensions early, though still developing.

3. Surveillance Automation: Real-Time Monitoring at Scale

Think of this as the difference between having an officer watch 50 CCTV feeds versus having software do it automatically and ping you when something looks off. It’s real-time monitoring at massive scale.

What It Does:

• Video analytics automatically flag suspicious activity on CCTV
• License plate readers scan and log thousands of plates per hour
• Acoustic sensors detect and triangulate gunfire locations
• Pattern recognition software tracks suspect vehicle movements and can even predict future locations
• Social media analysis during investigations

Real-World Examples:

• Shanghai: Smart CCTVs for traffic enforcement significantly increased seatbelt compliance
• Various agencies used facial recognition to identify rioters though research found it reinforced Black-white arrest disparities

Camden’s surveillance system in action: Their Real-Time Tactical Operations Intelligence Center monitors hundreds of cameras across the city. The results? Reduced crime and faster response. But replicating this requires serious investment and community buy-in on constant monitoring.

The Evidence: Effectiveness varies widely. Some tools show clear operational benefits (faster response, automated detection). But others raise serious accuracy and bias concerns, facial recognition being Exhibit A.

4. Disorder Automation: Predicting Social Unrest

This is where it gets really futuristic and controversial. Here’s how it differs from Crime Sensing: Crime Sensing relies on community interviews/post with human input, human analysis. Disorder Automation is fully automated: algorithms crawling millions of social media posts to detect brewing tensions before anyone reports a problem.

What It Does: Uses machine learning to scan social media platforms, looking for patterns suggesting communal tensions are rising. It monitors for hate speech spikes, signs of riot planning, or early indicators of unrest.

Real-World Example: Platforms like Dataminr and Babel Street offer these capabilities to law enforcement, monitoring social media in real-time for emerging incidents and potential threats. Academic research using natural language processing successfully measured racial tensions on social media following the 2011 Evra-Suarez abuse incident in UK football, demonstrating the concept’s potential.

The Evidence: No rigorous evaluations yet on operational deployment in law enforcement. This is still bleeding-edge stuff. The technology works in research settings, but we don’t yet know if it actually prevents violence in real-world policing or just creates new surveillance concerns.

What’s Holding Data-Driven Policing Back?

The researchers found barriers at three levels: institutional, organizational, and individual.

Institutional Challenges

Benefits: Yes, data can deliver faster response times, sharper risk assessments, and better visibility in high-crime areas. Some agencies found that when they presented data-backed decisions to their communities, it actually improved transparency and built trust. Done right, data helps you make smarter deployment decisions and measure what’s actually working.

Problems: But here’s the challenge with data in policing: garbage in, garbage out. If your historical data reflects uneven enforcement patterns, whether that’s due to resource allocation decisions, community complaint patterns, or where your agency has historically focused attention, your algorithm will learn from and repeat those patterns.

Take London’s gang database for example, 80% of the entries are Black individuals. Whether that accurately reflects gang membership or reflects where enforcement resources have been focused is debatable. But here’s what’s not debatable: if you feed that data into a predictive targeting system, the algorithm will send officers back to those same areas and populations. You find more activity there (because you’re looking), which feeds back into the algorithm, which directs you back there again. It’s a feedback loop that can entrench existing patterns, accurate or not.

And even when agencies are confident in their data quality, getting different jurisdictions to actually share information remains a nightmare. Everyone’s on different systems using different standards. The promise of multi-jurisdictional data sharing through fusion centers hasn’t fully materialized because of technological incompatibilities and good old-fashioned bureaucratic resistance.

Organizational Barriers

Here’s the tension playing out in departments nationwide: Civilian analysts with advanced degrees in statistics build deployment models based on crime patterns and call data. They present their findings to veteran officers who immediately spot gaps, areas the algorithm missed because of data quality issues.

This isn’t just about algorithms versus experience. It’s about fundamental problems with how data gets entered and categorized. Inconsistent incident coding means the algorithm is working with incomplete information. The officer’s local knowledge catches what the data misses. At the same time, officers often resist being directed by systems that don’t account for nuance such as understanding the social context of an area or the shifts in neighborhood patterns that haven’t shown up in the data yet. They describe feeling “deskilled,” reduced to button-pushers following algorithm orders rather than professionals exercising judgment.

Smaller agencies face even steeper challenges. They can’t afford the latest analytical platforms or dedicated data scientists, so they assign the role to soon-to-retire officers who may lack technical skills. Even when departments invest serious money in technology, they often skip the hardest part: changing the organizational culture to value both data insights and street-level knowledge. Without strong leadership bridging that gap and establishing clear goals, the technology sits unused or generates resentment.

Individual-Level Issues

Even when you have the technology, the budget, and the leadership buy-in, you hit a wall at the individual level: the people using these systems can barely keep up. Analysts are drowning trying to stay current with constantly evolving analytical tools. One month it’s ArcGIS, the next it’s some new machine learning platform, and training never quite catches up with the pace of change. Meanwhile, they’re simultaneously dealing with garbage data. We’re talking typos in incident reports, days-long delays in logging arrests, inconsistent coding across different shifts (one officer codes a fight as “assault,” another calls the same thing “disturbance”). The skill gap is real, the data quality issues are real, and together they mean that even good technology produces questionable outputs.

The Ethical Minefield: Privacy, Bias, and the Black Box Problem

Every surveillance camera, social media scrape, or predictive analytics deployment involves trade-offs: efficiency versus privacy, prevention versus presumption of innocence, scale versus oversight. The problems compound quickly. Data quality issues get baked into predictive models. The algorithm directs patrols to certain areas, more activity gets detected there, which reinforces the algorithm’s predictions. It’s a feedback loop that can entrench patterns regardless of whether they accurately reflect where crime is occurring. Add in mission creep and the lack of transparency around proprietary algorithms, and you’ve got a recipe for community trust problems.

Here’s what should concern every chief: most agencies don’t actually understand how their predictive tools make decisions. They’re black boxes. The vendor says “trust the algorithm,” but it’s proprietary and you couldn’t explain the decision-making process to a judge or community member if you tried. Look at Chicago’s Strategic Subjects List or Durham’s HART risk assessment, both were deployed with good intentions but became flashpoints for community backlash when people realized the tools ignored crucial context (family circumstances, employment) and couldn’t be adequately explained or challenged. The technology promises mathematical objectivity, but without transparency and rigorous evaluation, you’re betting your department’s legitimacy on systems you don’t fully understand.

CPD decommissions ‘Strategic Subject List’

The Chicago Police Department had used analytics to identify which prior arrestees would be most likely to carry out — or be victims of — shootings.

Four Key Recommendations for Law Enforcement Leaders

1. Keep Humans in the Loop

Technologies like PredPol and HART have become deeply integrated into planning even though effectiveness is mixed.

The Solution: Adopt “human-in-the-loop” configurations where human judgment augments data-driven decisions rather than being replaced:

• Analysts review algorithm recommendations before deployment
• Officers can override system suggestions based on community knowledge
• Regular audits compare algorithm outputs to actual outcomes
• Community input informs how predictive data is used

2. Don’t Let Data Replace Beat Knowledge

Officers resist technological control over their discretion for good reason. It often ignores the contextual knowledge that makes policing effective. They describe it as “deskilling” or “experiential discounting,” reducing them to mere line workers following orders from a machine.

What Agencies Should Do: Training can’t just be “here’s how to use the software.” It needs to explore the best ways to integrate officers’ cognitive abilities and local knowledge with data insights. The goal is building officer confidence in data-driven processes beyond mere compliance. Design systems that enhance rather than replace officer discretion, especially for complex situations where context matters more than patterns.

3. Evaluate, Evaluate, Evaluate

Knowledge about how data-driven transformation actually occurs in police departments is severely limited. Agencies are creating new analyst roles and implementing technology without clear metrics to measure costs versus benefits.

Before You Buy: The reality is that rigorous, independent evaluation data on these technologies is scarce across the industry. Vendors typically provide case studies and testimonials showing positive results, but these often can’t isolate whether the technology itself caused the improvements or if other factors were at play. Ask vendors what metrics they use to measure success and request any available evaluations, particularly from independent researchers rather than vendor-commissioned studies. Look for peer-reviewed publications when available, and get case studies from agencies similar to yours in size and context.

More importantly, plan to evaluate the technology yourself. Set clear metrics before implementation: What specific problem are you trying to solve? How will you measure whether it’s working? What baseline data will you collect for comparison? Build evaluation into the deployment from day one so you can make evidence-based decisions about whether to continue, modify, or discontinue use. This protects your budget and ensures you can demonstrate value to your community and leadership.

4. Don’t Ignore Service Provision

Data’s role in service provision (emergency response, accident management, social support) is largely ignored, despite these making up one-third of street-level police work. Data tools focused only on crime prevention are missing opportunities to improve your most frequent citizen interactions. Consider how analytics could:

• Identify patterns in mental health crisis calls and inform co-responder programs
• Improve traffic accident response and investigation
• Track wellness checks and vulnerable person contacts
• Measure effectiveness of community service programs
• Predict demand for non-enforcement services

Questions to Ask Before Adopting New Technology

1. What problem are we actually trying to solve? (Be specific)
2. What evidence exists that this tool solves it? (Demand rigorous studies)
3. Do we understand how the algorithm makes decisions? (Proprietary = problem)
4. What biases exist in our historical data? (Be honest)
5. How will we maintain human oversight? (Augment, don’t replace judgment)
6. What’s our transparency and accountability plan? (Educating the community)
7. Do we have capacity to use this effectively? (Staffing, training, leadership)
8. How will we measure success? (Establish metrics before implementation)

For a comprehensive guide with readiness assessments, procurement checklists, and implementation tools, see the Council on Criminal Justice’s Assessing AI for Criminal Justice: A User Decision Framework (released April 2026).

Assessing AI for Criminal Justice: A User Decision Framework – Council on Criminal Justice

Criminal justice agencies and practitioners face urgent questions about how to effectively and safely adopt AI tools. This decision guide provides a detailed and structured pathway to help them evaluate, implement, and oversee AI in the criminal justice system.

Final Thoughts: Trust is Your Most Valuable Asset

The researchers reviewed 192 studies, and their conclusion isn’t “data is bad” or “data is good.” It’s: Data is powerful, and with power comes responsibility.

Your community’s trust is your most valuable asset. It can be destroyed in moments if they believe you’re using technology to surveil rather than serve them, or if biased algorithms lead to discriminatory policing.

Algorithms can predict where crime might occur, but they can’t tell you whether deploying officers there will build or destroy community trust. Facial recognition can identify a suspect in seconds, but it can’t repair relationships if it disproportionately targets certain communities.

Before you sign that contract for the next predictive policing platform, ask yourself: Do we really understand what we’re buying? Have we thought through the implications? Do we have safeguards to prevent harm? Are we prepared to be transparent?

Data can make us more effective. But only if we use it wisely, ethically, and with our eyes wide open to both its potential and its pitfalls.

Enjoyed this post?

Get more research translated into plain language delivered straight to your inbox.

Subscribe below and let me know which topics you want to hear more about. It only takes a minute.

If you found this helpful, share it with your team or on your socials. It helps more people make sense of research that matters.

Carlee Ruiz

Carlee Ruiz is the founder of PoliceResearchHub, a platform that turns academic research into practical insights for law enforcement. She brings seven years of experience as a crime analyst and justice researcher, with a focus on evidence based policing, gun violence prevention, and public safety strategy. Carlee currently works as a researcher at National Policing Institute, where she supports evaluations of law enforcement programs and criminal justice initiatives across the country. Through PoliceResearchHub, she aims to make research easier to understand and more useful for the people doing the work on the ground.