Implementation Research

by Karin Martinson

What is it used to measure?

Implementation research is used as a general term for research that focuses on the question of "what is happening and why" in the design, implementation, administration, operation, services, and outcomes of social programs.

Implementation research explores fundamental questions about how programs function—what the real (as opposed to nominal) goals of the program are, whether those goals are internally consistent and shared among multiple stakeholders, what administrative and management procedures are engaged in the pursuit of the goals, how the sequence and timing of client activities are designed, whether these operations reflect the intent of the program designers, how well the program is achieving its goals, how policy is changed by frontline staff decisions, and whether clients are reacting to the program in ways intended by its designers. Implementation analyses can supplement and complement impact evaluations by providing information to interpret results from these studies; yielding more immediate suggestive evidence, such as hypotheses on how different components of the program might contribute to overall effects; and identifying lessons for improving policy and practice.

How does it work?

Because implementation studies may address several aspects related to the way a social program operates, they can encompass a wide variety of data and data collection strategies. Many of the primary data needed for implementation research are gathered firsthand in the "field", where program activities happen and client outcomes are realized. A large portion of the needed data are key stakeholders’ first-hand accounts of program processes, experiences, opinions, and results, including program planners, state agency managers, local office management and staff, service provider management and staff, advocacy and public interest groups, and clients.

To gather information from respondents, researchers use a variety of methods that commonly include the following:

• Open-ended, semistructured interviews that focus on the parts of the program most relevant to the informant;
• Focus groups of more than 10 to 15 people of similar backgrounds, organized around a set of topics;
• Extended participant observation to allow the program to be understood from the clients' view; and
• Program data on client service receipt and intensity collected from automated systems or case files.

Research examples

"The Application Process For TANF, Food Stamps, Medicaid, and SCHIP: Issues For Agencies and Applicants, Including Immigrants and Limited English Speakers"

"Reforming Welfare: Institutional Change and Challenges"

"The Indiana Welfare Reform Evaluation"

"The Implementation of the Welfare-to-Work Grants Program"