Population Results - Evaluation Framework

Introduction

The Population Results indicators are markers of low-income people’s achievement of recommendations put forth in the Dietary Guidelines for Americans and Physical Activity Guidelines for Americans, and their associated health and well-being. Population-level indicators measure changes over time in the behaviors that promote positive health outcomes. The over-arching evaluation question in this chapter is: To what extent does SNAP-Ed programming improve the low-income population’s achievement of the Dietary Guidelines for Americans recommendations and other health risk behaviors, compared to the general population?

Population-level indicators should present the overall profile of the SNAP-Ed eligible population and the distribution of outcomes for different subgroups based upon socioeconomic and demographic factors, such as income, race or ethnicity, age, educational level, languages spoken, and geographic scale. Comparing SNAP-Ed eligible persons to the general population is important to measure the degree to which the SNAP-Ed eligible population is doing better than, worse than, or about the same as the general population. These data will also be useful for SNAP-Ed needs assessments and intervention planning.

The distinguishing factors between the Individual-level indicators in Chapter 1 and the indicators in Chapter 4 are who gets surveyed, under what circumstances, and what the outcomes intend to show. In Chapter 1, the indicators measure program effectiveness for participants of SNAP-Ed’s direct education activities. In Chapter 4, the population results track and measure the behaviors and health status of low-income audiences that may participate in direct education but also benefit from PSEs and marketing activities in the communities where they live. In this chapter, the 11 indicators measure improvements in population behaviors and associated health statuses year-after-year, resulting from strategies and interventions across the framework. Here, we assess the combination of educational activities, marketing, and policy, systems, and environmental (PSE) changes in settings or sectors.

States can collect the needed data in many ways. Local, state, territorial, and tribal agencies that conduct surveys among cohorts of SNAP-Ed participants in SNAP-Ed qualified schools or community settings are able to routinely track and measure population-level results on a quarterly, bi-annual, annual, or biennial basis. Agencies may also conduct annual or biennial population-level surveillance of low-income audiences using state-run surveillance systems, such as the Behavioral Risk Factor Surveillance System (BRFSS) on adults or the Youth Risk Behavior Surveillance System (YRBSS) on high school students. The BRFSS and YRBSS questionnaires are in the public domain and no permission is required to use them.  Several state SNAP-Ed programs—including those in Arizona, California, and Hawaii—have funded additional items and modules on the BRFSS for surveillance in SNAP-Ed eligible populations. For instance, the Arizona Nutrition Network, found it cost effective to pay for demographic screeners to be included in its statewide BRFSS to identify respondents with a household member participating in the SNAP, WIC, or the National School Lunch Program. By adding these demographic screeners, the Arizona Nutrition Network was able to identify health behaviors and trends in nutrition assistance program participants that could be compared to the general Arizona population at no additional cost to SNAP-Ed. Contact your state or territorial BRFSS coordinator to learn how to add questions for the SNAP-Ed eligible audience to your state’s CDC-funded BRFSS survey: https://www.cdc.gov/brfss/state_info/coordinators.htm.

You can also measure population results by conducting a population-level 24-hour dietary recall with your local, state, territorial, or tribal SNAP-Ed eligible population. For instance, Land-grant Institutions that report results in the Web-Based Nutrition Education Evaluation and Reporting System (WebNEERS) are monitoring population results in low-income audiences over time. You can also conduct annual or regular data collection of your total SNAP-Ed eligible population or a representative random sample of it using the measures prioritized in this chapter. The same questionnaire or module should be used year-to-year for consistency. For instance, the Michigan Nutrition Network at the Michigan Fitness Foundation requires its SNAP-Ed partners to administer the National Cancer Institute (NCI) Fruit and Vegetable Screener as a pre-post instrument based on a convenience sample of program participants. If a partner’s direct SNAP-Ed eligible reach is less than 500, it is required to complete 75 survey pairs. If it is greater than 500, a representative sample of 15 percent is required. Representative is a key word for surveillance and monitoring. If necessary, oversampling must be done to obtain a sample reflective of the characteristics of the partner’s population. The University of Maryland’s Food Supplement Nutrition Education program also administers semi-annual surveys in affiliated SNAP-Ed schools to assess progress and trends over time resulting from comprehensive school-based approaches; their surveys are not tied to a specific educational curriculum. Some states, such as California and Washington, or large cities and counties, such as Los Angeles and New York City, also conduct their own omnibus state or city-wide health surveys.

Given the restrictions on using SNAP-Ed funds for general surveillance, and that funds can only pay for the portion of the surveillance activity reaching those within 185 percent of the federal poverty level, you may seek partnerships with other funding programs or use existing data sources that provide such breakouts. The following table contains a list of free, public data sources useful in tracking population results included in the framework:

Data Sources for Tracking Population Results and Reductions in Disparities

Healthy People 2020 Reference Points

The reference points used in this chapter for health outcomes or targeted food groups, beverages, or physical activity and reduced sedentary behaviors may align with population benchmarks used in surveillance surveys, and, where available, Healthy People 2020 targets. Healthy People provides science-based, 10-year national objectives for improving the health of all Americans. Most Healthy People 2020 targets rely on national data sets, such as the National Health and Nutrition Examination Survey (NHANES). Because it is not possible to use certain national surveys, such as NHANES, to report at the state or local data level, SNAP-Ed agencies may use Healthy People 2020 for goal-setting purposes. A table of selected Healthy People 2020 objectives appears below.

Selected Population Goals: Healthy People 2020 Objectives