Issues in Evaluating Parenting Curricula

Douglas R. Powell, PhD
Professor and Head, Child Development and Family Studies
Purdue University


Summary

Evaluations of parenting education curricula generally have provided confusing results that fail to demonstrate support for the effectiveness of parenting programs (Cowan, Powell, & Cowan, 1998). These conclusions have been reached by a number of reviewers (Dembo, Sweitzer & Lauritzen, 1985; Levant, 1988; Powell, 1988; Medway, 1989). Recently, for example, Cheng Gorman and Balter (1997) reviewed quantitative studies of culturally sensitive parent education programs and concluded that methodological flaws are prevalent and may contribute to the finding of limited program efficacy. Reviewers typically recommended the following improvements in evaluation design and measurement: an integral role for research in each stage of program development and implementation; samples greater than 10 to 15 families; inclusion of fathers as well as mothers; the use of no-treatment or alternative-treatment controls; random assignment; systematic collection of data on program implementation; a conceptual or theoretical model of anticipated processes or mechanisms of change in parents; analyses that attend to differential program effects; and multimeasure and multimethod assessments that include parent self-reports, parents' reports about children, observation of parents' behavior, and independent assessments by both teachers and researchers of children's outcomes (e.g. Cowan et al., 1998). This is a tall order for a field that typically encounters numerous obstacles in implementing community-based programs and at the same time is often under-funded.

These recommendations represent good practice in program evaluation work and are appropriate standards for credible evaluations of a parenting curriculum. Three of these recommendations warrant special attention because collectively they can yield a research base to improve the responsiveness of programs to a particular population. The three areas are: (1) incorporating evaluation into all phases of program development and implementation, including program design decisions; (2) employing a theoretical or conceptual model of the anticipated mechanisms of change in parenting programs; and (3) systematically examining for differential program effects in analyses of program outcomes. In the past decade, the field of parenting education programs has moved away from the cookie-cutter notion that "one size fits all," but evaluation work has been slow to respond to the emerging perspective on programs as flexible, adaptive entities. The three evaluation strategies I have highlighted here hold good potential of helping parenting curriculum designers and implementers strengthen the responsiveness of programs to particular populations.

Today I want to share with you some lessons learned in attempting to implement these three recommendations in the development and implementation of a parenting program known as Links to Learning. This program is aimed at parents of elementary school-age children, and has been developed over a five-year period in diverse urban and small-town communities in Indiana and Ohio. The project has benefitted from grant support from the Lilly Endowment, Inc. and from the resourcefulness of many parents and professionals, especially Dr. Susan Peet, the coordinator of evaluation work carried out during the project's demonstration phase.

The Links to Learning curriculum was developed as a tool to strengthen family contributions to children's learning during the elementary school years. Development of the curriculum began with five guiding assumptions:

Society is changing:

The world is changing rapidly. Technological advances, global economic shifts, and demographic changes are affecting all segments of society. These changes increase the importance of problem-solving skills that enable individuals to be responsible citizens and successful participants in a competitive job market.

Families are unique learning environments:

Children's learning experiences in families and communities are special and as important as learning at school. The ways in which families contribute to children's learning are different from the ways in which schools support learning. Family-based learning should be approached on its own terms; there is limited usefulness in imposing a school-based model of learning on families.

Daily routines are important:

The fabric of daily life in families is influential in supporting children's learning experiences, including their problem-solving skills. Ordinary routine interactions may seem mundane yet they hold much potential for maximizing the contribution of families to children's learning.

Learning is an active process:

At any age, important learning is most likely to occur when individuals are highly engaged in things of interest to them. This principle speaks to the value of "beginning where people are" and enabling individuals to generate their own solutions to problems of interest to them.

Elementary school years count:

Early school years are a prime time to foster children's approaches to learning. This period of parenting is often overlooked. Most parent education programs are designed for parents of young children or adolescents. Yet the elementary school years offer unique opportunities and challenges to parents.

Curriculum development work proceeded in four stages. First, information was gathered from diverse sources for making decisions about program content and methods. Second, a pilot curriculum was developed. Third, the pilot curriculum was implemented in nine different communities and subsequently revised based on implementation experiences. Fourth, the revised curriculum was implemented as a demonstration program in five communities and evaluated through a wait-list control experimental design. The final curriculum involves 10 90-minute sessions in a group discussion format facilitated by a two-person team of professional and parent.

Incorporating evaluation into all phases of curriculum development

From the beginning, a goal was to infuse parent perspectives into the design of the programs in ways that integrated parent and professional or expert knowledge (see Figure 1). We garnered professional knowledge through customary means: we reviewed relevant empirical and theoretical literature and organized an advisory committee of experts who brought years of experience in developing and implementing parenting programs. We attempted to marshal parent knowledge by conducting an interview study with 151 parents in three Indiana communities, two of which were urban. We recruited parents for participation in the study through first- and fourth-grade public school classrooms, and conducted in-depth interviews mostly in the homes of study participants. By design, the sample consisted predominantly of families with limited levels of formal education and income. 85 of the families were two-parent households, and we interviewed both fathers and mothers in all but two of these families. The mothers represented Latino (24%), African American (27%), and Anglo American (47%) backgrounds; fathers represented a similar distribution. Our goal was to recruit for this study a sample of parents similar to those who were likely to be attracted to a parenting education group. The interview with parents focused on a range of topics related to their children's futures.

diagram
[full image: 33K]

Figure 1. Links to Learning Program Design

We learned a good deal from parents. A strong and consistent theme across nearly all interviews was parental worry about their child's future. An overwhelming majority of parents indicated that their child needs to do things differently than they did in order to succeed in today's world. Approximately one third do not expect their child will attain what the parent considers to be an ideal occupation for the child. We learned that parents gave greater emphasis to child characteristics than to environmental influences as contributors to success. That is, more parents emphasized child factors such as academic skills and abilities than environmental factors such as a supportive atmosphere for development and learning. We also learned that the weekday period from 4 to 8 p.m. is a good target time for contemplating enhancements in parent-child interaction within daily family routines. Television surfaced as a dominant organizing force in determining the nature of family interactions.

This type of information proved to be highly valuable in forming decisions about program content. We decided, for instance, that the program needed to address squarely the widespread concern about what parents can do to help their children prepare for unknown and uncertain futures. From these interview data we also envisioned program opportunities to emphasize the idea that child competence is developed through interactions between individual and environment, and we identified challenges in helping parents think about ways of tweaking daily family routines to enhance opportunities for children's learning.

Parent perspectives were tapped through more qualitative means at the second and third stages of curriculum development. We used focus groups frequently to try out various isolated curriculum ideas and activities. We heard stories about parent interactions with schools and with children, for instance, that became the basis of critical incidents incorporated into the curriculum for parent group discussion. We learned that role playing was an enjoyable group activity, and that relations with school personnel were emotionally-charged topics requiring two full sessions. Implementation of the full curriculum in nine pilot sites was chronicled carefully through observations, interviews with group facilitators after each session, and periodic interviews with parents. A main lesson of the pilot work was the need to better support a collaborative team of professional and parent; we were reminded of the deep roots in our culture surrounding the notion that professionals should be in charge. We also learned of the creative accommodations of curriculum ideas made by group facilitators. One facilitator who could not read, for instance, communicated core ideas to her peers through elaborate illustrations on the room walls; one of these illustrations led to an important cartoon in the curriculum about perspective-taking skills as a basis of problem-solving skills.

We have no data to support a claim that the end result -- the final program curriculum -- would have been less responsive to parents if we had pursued a program development path that did not incorporate parent perspectives. Compared to other curriculum development enterprises I have observed and contributed to, however, it seems we had fewer false starts and dead-ends in the Links to Learning program development journey, and that, if nothing else, we were able to approach the final implementation of the full curriculum with a good deal of confidence about its ecological validity.

Model of change mechanisms

Over time we generated a model of how the program might contribute to changes in parent beliefs and behaviors. It has three sequential parts. The first step focuses on parents' beliefs about influences on their child's education, including parents' sense of self-efficacy regarding these influences. This domain builds on a growing literature demonstrating the significance of parents' beliefs about children's development and learning (e.g. Sigel, McGillicuddy-DeLisi, & Goodnow, 1992) and parents' ideas about whether they have an appropriate role in their children's education that can make a genuine difference (e.g. Hoover-Dempsey & Sandler, 1997). The second step or domain is quite simply parental recognition of opportunities for children's learning in family and community settings. Recognizing a teachable moment in the context of riding on a bus, shopping, watching television, or preparing a meal is an illustration of this domain. The third step is maximizing the learning opportunity (the second domain) through parent-child interaction, especially through use of inquiry or question-asking. There is an extensive theoretical and research literature on the value of engaging children in conversations via open questions (e.g. Stemberg & Williams, 1995) that supports this third step. My working metaphor for this model is theatrical in nature: the first step establishes the actors and their roles, the second step provides the stage and the props, and the third step is the enactment through verbal interactions between parent and child.

model
[full image: 21K]

Figure 2. Model of Change Mechanisms

This conceptual framework was not tested in the Links to Learningcurriculum implementation. Rather, it guided curriculum content and it also helped us make decisions about outcomes and measures and to organize our data analyses. We used an experimental design (wait-list control) to test the efficacy of the program with a sample of 129 parents (71 program, 58 control group parents who subsequently received the program), a majority of whom had annual family incomes of under $40,000 and who represented African American (35%), Latino (26%), and Anglo American (61%) backgrounds in five different communities.

We found positive program impacts on parents' beliefs about the importance of different influences on their child's education, including positive change in parents' beliefs about their ability to have influence on these factors. With regard to maximizing opportunities for learning, we found positive program effects on the extent to which children participate in daily routine tasks including meal preparation, clean-up, and laundry, and on the level of parent participation in school-related activities with the child, but not on family activities with the child or communication with school personnel to support the child's education. In the third domain, we found positive program impact on parents' attempts to understand the child's point of view and to build on what the child says or does in hypothetical parental teaching situations, and a modest program impact on how often parents and children discuss school-related and personal topics. There was no effect, however, on joint decision-making, and on the frequency of daily interactions with the child.

Our strongest program impact, then, occurred in the first domain focused on parental beliefs. Effects were less robust in the behavioral areas of strengthening opportunities for learning and in the domain of parent-child interaction. These types of limited effects may be appropriate for a short-term parenting program.

Differential program effects

Income is defining variable in life that long has been linked to a number of individual beliefs and behaviors, including child-rearing socialization practices (e.g. Kohn, 1969). We organized the sample by three income categories: low income ($19,999 or less), moderate ($20,000 to $39,999), and higher income ($40,000 or above) and found important differences in some outcome areas. Lower-income program parents compared to lower-income control parents showed changes in beliefs about factors that influence their child's education and increases in level of personal influence on these factors, and increases in daily routine activities (involvement in meal preparation, clean-up, laundry). Moderate-income program parents compared to moderate income control parents had more frequent conversations with their child about school-related and personal topics. Higher-income program parents compared to higher-income control parents had greater participation in joint decision-making with their child.

It seems that parents of differing income levels "took" different types of messages from the program. Connecting these findings to the conceptual model of anticipated change mechanism, we see that changes in the first two domains -- beliefs and learning opportunities -- occurred among lower-income parents while changes in parent-child interaction were more likely to occur among moderate and higher-income parents. It appears that parents entered the program with different sets of lenses or frameworks for engaging core messages of the program curriculum. At baseline (pre-program), there were significant differences across many variables by family income, and hence we used baseline scores as a co-variate in analyzing post-program outcomes.

Concluding comment

I view our approach to evaluation in the Links to Learning program as a case study in how a curriculum development effort might strengthen the contributions of evaluation to the responsiveness of program design. As noted at the outset of this paper, certainly much more needs to be done to improve common practices in evaluating parenting programs. Yet evaluation has more to offer than "proof" of a curriculum's efficacy. A pressing issue in the improvement of parenting programs is sensitivity to the needs and characteristics of the population to be served, and the evaluation enterprise has an important role to play in achieving this goal.

Copyright © 1998 Douglas R. Powell.

stars icon For technical assistance:
Library Technology Group
General Library System
University of Wisconsin-Madison