Program Evaluation

Evaluation in first and second grade classrooms: 2003-04 school year

Study Design
This evaluation project was designed to answer the following questions:

• To what extent are the teachers and students satisfied with the program?
• Do students improve their knowledge and food choices after going through the program?
• What, if any, areas of improvements exist in the implementation and use of the program?

Both quantitative and qualitative data were used to answer these questions. Quantitative data were collected from both students and teachers in the form of self-administered surveys. Qualitative data were collected on the school site through teacher interviews and classroom observations. Dairy Council and WestEd conducted a total of 20 observations in second grade classes and eight observations in first grade classes. No comparison group was used for this evaluation and therefore no causal inferences can be made about the program.

Teachers Approve!
Teachers had very positive responses to the curriculum. Overall, students really liked doing the activities, playing with the food cards, and talking about foods they like to eat. Teachers liked teaching it. It was easy to use and easy to augment with other activities beyond the workbook. Teachers commented that the color pictures in the student workbook were very helpful for students. Teachers felt that their students were more aware of the foods they eat after participating in the HCHM lessons, and some teachers observed that students were bringing healthier snacks to school. Below are specific comments from teachers regarding the program.

"I really feel that this unit…meets a large portion of District and State guidelines for health in 2nd grade. The unit was easy to understand and teach. My children enjoyed it and learned a lot of important information."

"It is easy to use. It is a great workbook for the kids and they like it. The story is fun…and easy to integrate with other subjects (i.e. math charting, mystery genre, science)."

"Overall it's a great nutrition program which is much better than the lessons in our health books. I look forward to getting another set next year."

Second Grade Student Results
The final second grade student sample consists of 330 students at pre-test, 339 students at post-test, and 292 students at the 2-month follow-up. The data presented here are for all students who participated at any of the three aforementioned tests. The student survey consisted of 19 questions about foods and the relationship of food to health.

The findings suggest that the HCHM program had an impact on gains in knowledge and self-reported eating behaviors among second grade students in participating classrooms.

Knowledge
The knowledge questions asked students to:

1. Classify foods into groups. The foundation of HCHM is learning which foods belong in each food group. Statistically significant results demonstrate the percentage of students classifying foods into groups improved between pre-, post-, and 2-month follow-up for all six foods with the greatest improvement of correct answers for peanut butter, cheese, orange juice, and candy. (See Figure 1.)

Figure 1. Percent of second grade students correctly classifying foods

2. Identify the function of foods on the body. Students learned the significance of each food group and its unique contribution to health. Results show knowledge about the functions of foods on the body improved significantly across all food groups between pre-, post-, and 2-month follow-up tests. Significantly more students correctly answered that milk products were important for building strong bones and teeth at post-test and 2-month follow-up, compared to pre-test (p < 0.00). (See Figure 2.)

Figure 2. Percent of second grade students correctly identifying health benefits of food groups

3. Identify the missing food group in a meal. The culminating activity in the program is for students to identify the missing food group using the Food Guide Pyramid as their guide. They are then asked to suggest a food to balance the meal from the food group not included in the photograph of a meal. Significantly more students correctly classified the foods presented and correctly identified the missing food group from both the hamburger and turkey sandwich meals at post-test and 2-month follow-up, compared to pre-test (p < 0.00). The students retained at least 70 percent of their knowledge in the 2-month follow-up test in regards to the hamburger and turkey sandwich meal tests. (See Table 1.)

Table 1. Percent of second grade students identifying correct answers in the hamburger and turkey sandwich meal and knowledge retained at 2-month follow up.

Note: All differences are statistically significant for Pre/Post and Pre/2 Month follow-up

Eating behaviors
In addition to increasing knowledge of food groups and their nutritive benefit, putting this newly acquired information into action is a primary outcome of the program. Progress towards healthier eating was assessed through eating behavior questions focused on whether students ate breakfast and what foods they typically ate for breakfast. Students were also asked to report frequency of new foods and food groups using three response options, "never," "sometimes," and "a lot" to the question "How often do you eat 'x' food."

• More than 80 percent of students ate breakfast most days or always for all three tests. Although difference in frequency of breakfast consumption was negligible between the three tests, changes in the types of foods consumed were noticeable. (Table 2.)
• The percentage of students reporting "Never" eating "Milk & Milk Products" decreased steadily to 4.2 percent in the post-test and 2.8 percent in the follow-up test compared to 6.6 percent in the pre-test. This suggests that the program impacted students' consumption of dairy products for breakfast over time. (Table 2.)

Table 2. Breakfast behavior for second grade students.

First Grade Student Results
The student survey was administered to 98 first grade students at pre-test and 101 students at post-test. No 2-month follow-up test was administered. The primary goal of the data collection in first grade classrooms was to assess the efficacy and grade appropriateness of a newly created program specific to first grade students. While the outcomes from the survey were interesting, they were most likely not statistically reliable due to teacher survey administration methods, and significance tests were not conducted. For first graders, the student survey consisted of 10 knowledge questions-five about classifying foods into correct food groups and five behavioral questions about eating breakfast.

Knowledge
Generally, the percentage of first grade students classifying foods into groups improved between pre- and post-test for all six foods with the greatest improvement of correct answers for cheese, peanut butter, candy bar, and orange juice. (See Table 3.)

Table 3. Percentage of first grade students correctly classifying foods

Eating behaviors

• Eighty-nine percent of students reported that they had eaten breakfast in the morning before the pre-test, 93.9 percent in the morning before the post-test. (See Table 11.) Percentage difference of students reporting that they ate breakfast sometimes or almost everyday was not statistically significant at post-test.
• Similar to second graders, among the first graders who reported eating breakfast the day of the survey, consumption of each of the food pyramid groups increased for each group (except Fruits) in the post-test compared to the pre-test. (See Table 4.)
• The greatest increases from pre- to post-test were in the "Milk & Milk Products" (30.4%) and "Breads & Grains" (11.8%) groups. (See Table 4.)

Table 4. Breakfast behavior for first grade

Next Steps
Dairy Council of California believes that an important part of program evaluation is acting on educators' comments on how to improve our programs for classrooms. Program revisions are in progress based on teacher feedback on the program and in the 2004-05 school year a unique workbook will be available for first and second grade students. This will better reinforce basics skills to first grade students such as classifying foods into food groups. Revisions will allow for second grade students to build on these foundation skills and connect to more core curriculum areas such as through the Market Mystery story and more advanced lessons such as using Venn diagrams.

If you are a first or second grade teacher in California you may order Healthy Choices, Healthy Me! free of charge. Feel free to submit this electronic order form or call 1-877-324-7901.

Click here to learn more about ordering materials from outside California.