Evidence and Other Eight-Letter Words: Musings from the National Evaluator

By Ashley Finley, National Evaluator, Bringing Theory to Practice, and Associate Vice President for Academic Affairs and Dean of the Dominican Experience, Dominican University of California

Sometimes, as BTtoP’s national evaluator, I wish I had a few quick data points I could rattle off when people ask me, “How do you know there’s a link between learning, civic development, and student well-being?” That would be much easier than my usual response of, “Well, that depends on the kind of evidence you want.” The thing is, evidence is a peculiar concept—there never seems to be enough, or the right kind, or the type that’s truly convincing. And yet, everyone still wants it. At Bringing Theory to Practice, we started with a hunch twelve years ago about the relationship between engaged learning, civic development, and well-being. And twelve years later, we’re still gathering the evidence to support that hunch. Because that’s the thing about evidence, or at least the kind required to take on a big, knotty question like, “How do learning, civic development, and well-being intersect?”—you’ll never really have all that you need.

But what we do know after some three hundred funded projects is that when campuses focus on learning, civic, and well-being outcomes as part of intentionally designed and implemented programs, the results are overwhelmingly positive. Because these programs often involve practices known to be among the most effective for engaging students (e.g. first-year seminars, learning communities, and service learning), that finding may not be much of a surprise. But the kind of evidence that BTtoP has encouraged points to how fruitful it can be to open the conversation about these types of practices (often referred to as “high-impact practices”) to more inclusively consider the degree to which outcomes related to students’ civic capacities and positive mental health may be affected as much as, if not more than, their traditional academic learning outcomes.

Over the last year, we have analyzed campus findings from eighty grant reports. Across a diverse range of campus projects that emphasized components of what would be considered high-impact learning experiences, nearly 60 percent of the eighty reports analyzed reported positive changes in students’ well-being as a result of participation, such as students’ increased trust in themselves, self-efficacy, flourishing, purpose in life, and self-acceptance. About 70 percent of campuses reported positive outcomes related to students’ civic capacities as a result of program participation, such as an increased knowledge of community needs, increased interest in civic issues, increased sense of civic duty, improved leadership skills, and a desire to advocate for someone other than oneself. Only a slightly higher percentage of project reports (approximately 75 percent) indicated positive outcomes related to traditional learning outcomes, such as increased class participation, ability to analyze ideas and synthesize information, and greater academic motivation.

In an effort to deepen what we know about campus programs aimed at connecting learning, civic development, and well-being, BTtoP launched the Well-Being Initiative in 2013. In addition to funding twenty-nine campuses to implement and assess the effects of targeted programs on student well-being, the initiative also sparked a research partnership with the Personal and Social Responsibility Inventory (PSRI) at Iowa State University (see http://www.psri.hs.iastate.edu/). Preliminary findings indicate that students’ sense of flourishing is more greatly influenced by their perceptions of being in a campus environment that supports personal and social responsibility than it is their individual civic experiences. Specifically, students’ perceptions of being in a campus climate that supports the development of moral reasoning and contributing to a larger community were most associated with higher levels of flourishing among student respondents. In a nutshell, the findings suggest the need for campuses to make systemic commitments to student engagement that go beyond one-off experiences but rather are connected over time, such that engagement is pervasive and expected.

BTtoP also has been invested in gathering evidence of institutional change. For this, we have relied heavily on campus case studies to understand how programs are established, how faculty and staff become engaged, how resources get aligned (or realigned), and how improvements are made. In 2014–15, we commissioned seven in-depth case studies of BTtoP campuses to gain insights into the institutional change process. We learned, unsurprisingly, that it helps to have a dedicated leader (or leaders) on campus to shepherd conversations and keep people moving, but we also learned those leaders can come in many forms—senior administrators, deans, faculty, and student affairs professionals. Those leaders also tend to understand the importance of working across academic and student affairs and can effectively build conversations that bridge those traditional divides. These campuses also found ways to connect their grant projects to the broader goals of their institutional mission or strategic plan, such that what could have been isolated project efforts seen as “this is what we’re doing now,” could just as easily be viewed as, “this is what we’re already doing but can now do more intentionally and systemically.” We will be sharing the case studies on the BTtoP website so that others can learn from the successes, challenges, and innovations of these campuses. 

What evidence do we have? Perhaps the best answer to that question is that for over ten years BTtoP has supported campuses in finding their own evidence. The very idea that campuses could collect evidence, would want to, and would share it with others has contributed significantly to the success of BTtoP’s research and its efforts. We have also been heartened by the overwhelming success of funded programs that have found that their participating students not only learn better, they also become better citizens and more fully developed individuals.

Maybe the truly peculiar thing about evidence is that we don’t ask enough of it. The expectations we have for evidence of programmatic success should go beyond just looking for success in one area—students’ learning or their civic development or their well-being. We should challenge the evidence to help us say something about all three. At BTtoP, we believe the extraordinary possibilities in higher education and for global flourishing lie at the convergence of them all, when we educate students to be critical thinkers with a sense of community who also possess the self-confidence and resilience to actually get things done.