Assessment the Key in the Quest for Quality
Jamie Merisotis, President/CEO, Lumina Foundation
Opening Keynote, International Assessment Conference, Association to Advance the Collegiate Schools of Business (AACSB), Austin, TX
Thank you, and good morning, everyone. I’m honored to be part of this event, and I want to thank AACSB for asking me to help kick off this year’s assessment conference. I’m excited to be here, and I hope I can bring a new perspective to the work that all of you do.
That work —assessing your students’ learning to ensure educational quality—that effort is hugely important … and it is central to our own work at Lumina Foundation.
If you’re at all familiar with Lumina, you know that we direct all of our resources and organize all of our efforts around a single, ambitious college-attainment goal—what we call Goal 2025. By the year 2025, we want 60 percent of Americans to hold a college degree, certificate or other high-quality postsecondary credential.
Certainly, there’s no need for me to convince anyone here of the value in boosting college attainment. We’re all aware of the transformative power of higher education and of the enormous benefits it affords—to individuals, to families, to regional and state economies, to a more equitable society, and to the strength and stability of our democracy. We also know that, in this increasingly complex and interconnected world, college-level learning is vital for anyone who hopes to maintain a middle-class lifestyle. Experts say that, by the end of this decade, nearly two-thirds of jobs will require some type of postsecondary credential. We also know that many of our international peers, representing countries who are participating in this conference, are already exceeding the U.S. attainment rate. That means a 60 percent attainment rate isn’t just a good goal … it’s a national necessity.
This reality is now understood by most Americans. The annual Lumina/Gallup poll administered a few months ago found that 61 percent of Americans agree with the statement that increasing higher education attainment is necessary to the nation — not just desirable or a good idea, but necessary. That number increased from 51 percent last year and 43 percent the year before. Gallup tells us they have never seen as rapid a shift of public attitudes on any broad-based indicator.
Almost all jobs that lead to the middle class are becoming higher-skill jobs.
This poll result reflects the broad acceptance of Goal 2025, and it’s easy to understand why this shift has taken place. It is about jobs, and the fact that we are now living in a knowledge-based economy. It is not just a shift to higher-skill jobs—it is that almost all jobs that lead to the middle class are becoming higher-skill jobs. This shift is most visible in industries like manufacturing and healthcare, but it can be seen across the economy.
I wonder, though: Do we who work in higher education fully appreciate the implications of this shift? I submit that living in an economy in which reaching the middle class depends on success in some form of postsecondary education demands that we rethink many of our assumptions about that education. In fact, at Lumina Foundation, we believe that the only way we can have a higher education system that enables most Americans to earn a high-quality postsecondary credential—in other words, a system that can help us reach Goal 2025 — is through fundamental redesign.
And I can summarize the nature of this redesign project in this way: We need to move from a higher education system focused on institutions and based on time, to one focused on students and based on learning.
What does it mean to move to a student-centered, learning-based system? For one thing, quantity matters. As a nation, we need millions more of our citizens to hold degrees and other postsecondary credentials. But we dare not lose sight of exactly what it is they wind up holding. In other words, quality matters, too. We must ensure that credentials have real and lasting value … that they demonstrate genuine talent … that truly reflect the knowledge and skills students need to succeed.
That’s what we’re talking about in that second part of Goal 2025, the reference to “high-quality” credentials. And for us, the long-used proxies for quality in higher education—seat time, faculty profile, institutional reputation, tradition, even endowment size—all of these indicators are insufficient, even invalid. What really matters—fundamentally, the only measure of educational quality that matters—is learning; that is, what students know, and what they can do with what they know.
Of course, as business and management educators—and as assessment experts—you made this connection long ago. You know better than anyone that any viable definition of quality must have relevance in the real world … that it must be firmly rooted in results, in specific learning outcomes. And others are making this connection as well. Employers, researchers, labor experts, policymakers … more and more people from many quarters say that what we’ve been doing to validate learning and award credits and degrees isn’t working as well as it should.
We’ve read all of those news stories about the skills gap. We’ve heard employers complain for years about the shallowness of the labor pool. We’ve read Academically Adrift and other research bemoaning the erosion of educational quality. We all know that state policymakers are pushing for greater accountability and higher returns on the investment of public funds for higher education. We can argue over the details and point fingers in all directions, but the main points are unassailable: The nation’s higher-ed system is producing too few graduates overall—particularly among the growing populations of low-income and minority students—and too many of those who do finish lack the skills, knowledge and abilities they need. Indeed, it is the way people integrate and synthesize these skills, abilities and knowledge—from technical and content-driven to broader and higher-level thinking—that leads to true proficiency. That proficiency, the mastery of those things, is what credentials with real quality behind them are all about.
Certainly higher education is stepping up to respond to this challenge—and one could argue that business schools are leading the way. For instance, there is growing acceptance of learning obtained outside the classroom—in part because the methods for assessing this prior learning are becoming increasingly effective. In addition, many new competency-based approaches are being pioneered nationally—by Southern New Hampshire University, Brandman University, the University of Wisconsin System, and Western Governors University, to name just a few.
Other sectors beyond higher ed are responding to the challenge, as well. Industry groups are stepping up efforts to offer or refine their own methods for assessing and certifying students’ learning and fitness for jobs. This is happening in the energy sector, in advanced manufacturing, in logistics—really, in almost every industry where blue collars are being supplanted by white lab coats.
All of these trends underscore an important message that is echoing through the halls of higher education: We need new and better ways to measure and assess learning—ways that are more responsive to current needs, more consistent, more transparent and clearly understood by all. We need to get beyond simple tests and find ways to ensure that students are learning across disciplines … that they can truly demonstrate mastery of skills, including those increasingly important “soft skills.”
I know you have heard these calls and that you understand this need for change. That’s one of the reasons you’re all here: to hone your ability to assess students’ learning in an increasingly complex and rapidly changing environment. Your presence here—your commitment to meeting this need — bodes very well for students and for society.
That’s because, in my view, there is no better way for these changes to be forged and implemented than under your leadership. As faculty members and assessment experts, you’re in the critical position to help higher education make the all-important shift to a learning-based system. You, better than anyone, know how students learn and how to best measure that learning. Perhaps even more important, as business and management educators specifically, you know how to ensure that learning links academic rigor with workplace relevance. You know how to make sure that a program’s value extends beyond classrooms and into careers … into lives.
Lumina Foundation has a deep respect for and, we hope, a growing understanding of the faculty’s critical, central role in ensuring educational quality. In fact, the concept of faculty-driven change is the centerpiece of our most visible effort in this area: the Degree Qualifications Profile.
The DQP—a baseline set of reference points for what students should know and be able to do at the associate, bachelor’s and master’s degree levels—was released in beta form in 2011. Since then, it has gained significant traction as a tool that can help redefine educational quality in terms of actual student learning. So far, the DQP has been faculty-tested in more than 400 institutions across the country. A newly revised version was released last fall, and it’s available on the Web, at degreeprofile.org. Work is now under way to further refine the DQP—including the important step of incorporating postsecondary certificates into the framework. I urge you to review the DQP, to test it and use it on your campus, and to give us your feedback as we strive to improve it.
We see the DQP as a step toward an even broader national dialogue about all postsecondary credentials—including degrees, certificates, and certifications. This dialogue will need to include national organizations representing not just education providers, but also industry groups and credentialing bodies across the spectrum of higher education and workforce development systems. In a knowledge-based economy, all of these players and systems need to talk to each other in a common language. Lumina believes this dialogue is absolutely necessary—creating a system where all learning counts wherever and however it is obtained; where the meaning of credentials in terms of skills and knowledge is understood and recognized by students, employers, educators, and the public; and where clear pathways to further education and career advancement are available to all.
As you explore the DQP, you’ll see clear evidence that we’re laser-focused on quality … that we’re not in any way seeking to lower standards or “dumb things down” just to increase degree-attainment rates. In fact, the very opposite is true. The DQP’s rich and well-considered learning outcomes show very clearly that quality—that fostering genuine learning—is very much the goal.
In fact, if I may, let me cite just three of the nearly 80 learning outcome statements you’ll find in the DQP:
In the category of “Civic and Global Learning,” for example, the DQP says the bachelor’s-level student must be able to (quoting here): “Develop and justify a position on a public issue and relate this position to alternate views within the community or policy environment.”
Next, to demonstrate “Communicative fluency,” a bachelor’s-level student must “Negotiate with one or more collaborators to advance an oral argument or articulate an approach to resolving a social, personal or ethical dilemma.”
Finally, to prove “Quantitative fluency,” a bachelor’s candidate must: “Construct mathematical expressions for complex issues most often described in non-quantitative terms.”
As you can see, the learning outcomes described here are clear and concrete, but they’re anything but simplistic. They can’t be demonstrated by any simple standardized test. In fact, they’re the absolute antithesis of that approach. Indeed, they can only be demonstrated through well-designed assignments and assessments … assignments and assessments that can only be crafted by experienced and thoughtful educators.
These outcome statements—and the entirety of the document—show the DQP’s commitment to defining and measuring the high-level, integrated learning that is vital to today’s students. It calls for knowledge that is both broad and specialized; for critical thinking; for ethical reasoning, for civic and global awareness. In short, it encompasses all of the things that students need to succeed—in the workplace and as an engaged citizen in our democracy.
As you consider the DQP, I’d also direct your attention to a paper by one of its four co-authors, Peter Ewell of NCHEMS. That paper is especially relevant, because as its title makes clear, it focuses on the DQP’s (quote) Implications for Assessment.
Peter’s paper, published in 2013 by the National Institute for Learning Outcomes Assessment, does a wonderful job of explaining the DQP and the thinking behind it. And it can help make the most of this tool as you apply it to your own work. I commend it to all of you. I won’t take time today to unpack the paper’s content, but I will direct you to one sentence that is particularly relevant. “The DQP asks faculty members to examine the entire instructional process from the inside out—starting from the perspective of learners and what they learn instead of the perspective of teachers and what they teach.”
Peter’s statement—much like the DQP itself—distills a wealth of thought and carries huge implications for all of us. It highlights two very important points. If I may, I’d like to examine each of those points more fully in the time I have left with you today.
The first point is that of perspective … the idea that the instructional process must begin and end with students—with “learners and what they learn.” This idea—of shifting the focus away from an institution-centric construct and toward understanding and meeting the needs of students—is absolutely central. This concept must be the central driver in the effort to redesign higher education for the 21st century. And as I said, fundamental redesign is a must—because the traditional higher-ed model is simply insufficient to our needs as a society. That’s not a criticism of any institution, individual or mindset. The fact is, the current system lacks the capacity and the flexibility to properly serve the millions of additional students who must be served if we are to meet the nation’s attainment goals.
Clearly, we need a revamped system: One that puts students firmly at the center by building it around pathways based on learning … one that challenges everyone to be accountable for the success of students … all types of students, in greater numbers than ever before, especially those who have traditionally been underrepresented in higher education—low-income students, first-generation college attenders, minorities, and adults. We need a system that requires transparency and cooperative effort, one that encourages innovation by rewarding actual outcomes, not mere process or effort or good intentions.
Certainly, a great deal of work needs to be done in the coming years to flesh out the details of this student-centered system of higher education. But the basic outline of this new system is already taking shape. For example, we know that, at its core, the system must offer multiple, clearly marked pathways to various levels of student success—learning pathways that are affordable, clear and interconnected, with no dead ends, no cul-de-sacs and plenty of on- and off-ramps.
We also know that these pathways must be built on the foundation of learning. Degrees and other postsecondary credentials can’t simply be defined by the amount of time a student spends in classrooms or labs. Rather, degrees must represent well-defined and transparent learning outcomes. Students should get credit for what they know and what they can do, and all learning should count—no matter how, when or where it was obtained.
When fully fleshed out, this new system will be one in which every student knows where he or she is going, how much it will cost to get there, how much time it will take, and what to expect at journey’s end—both in terms of learning outcomes and career prospects.
Again, the development of this student-centered system isn’t just an option. It’s an imperative, driven by the national necessity to educate millions more Americans—particularly the emerging majority of 21st century students, including working adults, low-income, minority and first-generation students.
I can’t conclude my remarks without mentioning several important resources, many or all of which may be quite familiar to you. We know that CLEP is being increasingly used to assess general education learning outcomes, and several others are following suit. We’ve seen tremendous progress in portfolio assessment being done by groups such as the Council for Adult and Experiential Learning (CAEL). There has been an uptick in assessment of corporate and military training by the National College Credit Recommendation Service and the American Council on Education. And the National Institute for Learning Outcomes Assessment (NILOA) has developed an online library of assignments developed by faculty from around the country to support competency-based assessment. If you visit the “Resources” section of the DQP website—degreeprofile.org—you can find that assignment library, and even contribute to it.
All of these resources and examples rest on the bedrock principle of student-centeredness … of focusing first on “learners and what they learn,” as Peter Ewell suggests. But let me end with an observation about the second part of Peter’s statement. He specifically charges faculty members with the task of changing the instructional process. He indicates—quite clearly—that this change effort cannot and should not be imposed from above or from the outside. For positive change to occur, and for it to endure, it must be made “from the inside out.”
At Lumina, we couldn’t agree more. We’re convinced that change—that fundamental redesign in higher education—is a necessity. And we’re just as convinced that faculty must embrace that change, and lead it.
Your talent and experience in assessing student learning are vital to the development and continuous improvement of the redesigned higher-ed system. The work you do in your classrooms and offices every day … forging the learning experiences and the assessment methods that most benefit your students … that work is truly transformative. What you do truly changes students’ lives.
And now, what you do can also help transform higher education writ large—at a time when transformation is absolutely vital. I urge you to seize this opportunity to help realize a new vision for a 21st century system, one that is student-centered, learning-based, and fully committed to quality.
How do we realize this vision? Focus on the students. Make sure they earn credentials that reflect genuine learning … that they gain the knowledge and skills they need to succeed. Make no mistake: Our nation’s future depends on their success—and their success depends on you.