Environmental Settings – Evaluation Framework

Introduction

Within a social-ecological framework, environmental approaches may include multiple, complementary activities within a site or organization. Changes in written policies, organizational practices, and the observable (physical or “built”) or communications environments may include the adoption and implementation of new or enhanced organizational practices, rules, or procedures that make healthy choices easier and more desirable. Examples of such activities may include marketing and promotion of healthy foods and activities, greater accessibility and convenience to purchase or select healthier foods, healthier food-service policies, participation in wellness councils, point-of-choice prompts to action, and access to recreational facilities.

In the Environmental Settings level of the framework, the focus is measuring changes in policies, systems, and environments (PSE) and complementary promotional strategies in SNAP-Ed qualified sites and organizations. This level corresponds to SNAP-Ed Approach 2 as defined in the Food and Nutrition Act: comprehensive, multi-level interventions at multiple complementary organizational and institutional levels.

The overarching evaluation question is:

"To what extent does SNAP-Ed programming facilitate access and create appeal for improved healthy eating and physical activity choices in the settings where people eat, learn, live, play, shop, and work?"

Program implementers and evaluators seek to learn whether SNAP-Ed’s influence in creating organizational changes, policies, rules, marketing, and education make healthier choices easier and preferred. The initial goal of the evaluation is to assess whether healthier choices are available and appealing. Some of these changes may result from organizational partnerships or individual champions. If PSE changes are made with fidelity to research- or practice-based models, we can expect individual-level outcomes (as defined in the previous chapter) to improve. Eventually, evaluators could also measure changes in individuals at these settings, using the indicators in the previous chapter, if desired.

Example of Organizational Changes

Changes can occur within and across sites and organizations to prompt healthy behaviors. Layering different types of complementary activities within a single organization or site helps to maximize impact (Riley et al., 2010). Facilitating changes across an organization—that is, similar changes across multiple school sites within a school district and preschool or afterschool programs, or complementary changes with reinforcing nutrition messages among a chain of food retailers or changes across a company with multiple field facilities that employ working parents and caregivers—are examples of more comprehensive approaches that are known to amplify results.

An example of a whole-setting approach is applied in a school setting in Georgia (shown above). Schools are a prime setting for implementing multi-level interventions for obesity prevention; many SNAP-Ed services are delivered in schools. To be most effective, policy, systems and environmental interventions should be combined with other activities or components that reflect the needs and readiness for PSE change within an organization. Additional components of PSE change may include evidence-based education, marketing, parent/community involvement, and staff training on continuous program and policy implementation. Ongoing or continuous evaluation of PSE changes and their respective additional components are necessary to assess long-term organizational effectiveness and maintenance of PSE change.

The primary role of SNAP-Ed Implementing Agencies is to provide initial assistance, consultation, technical assistance, and a supportive inter-organizational infrastructure to help create appropriate organizational or environmental changes that benefit low-income households and communities. It is ultimately the responsibility of the participating site or organization to adopt, maintain, and enforce the changes.

Domains of Environmental Settings

To maintain consistency across states in reporting PSE changes, the framework categorizes SNAP-Ed settings into six domains: eat, learn, live, play, shop, and work. This categorization will help to aggregate activities across settings in a meaningful way and ensure that SNAP-Ed providers diversify PSE-related activities across priority settings. It will also help with tracking and reporting outcomes across multiple sites or different physical locations where SNAP-Ed services are provided. All settings should be categorized into a domain as shown on the next page:

Domains of Environmental Settings

* Low-wage SNAP-Ed worksites are defined as industries generally classified by the Bureau of Labor Statistics as low-wage, with at least 25 employees, and whose human resources department or management confirms that more than half of workers earn annual wages comparable to 185 percent of the federal poverty level for the state.

RE-AIM Model

RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) is a useful model for planning and evaluating the dissemination and implementation of evidence-based interventions. Indicators in the Environmental Settings level of the framework reflect a modified version of the RE-AIM model. For SNAP-Ed purposes, agencies would measure reach and adoption by the people and organizations/sites that could and do offer evidence-based interventions, then implementation of the essential components, effectiveness in terms of checks on progress, and lastly maintenance using the definitions shown below.