How to Define and Measure Higher Ed's Value

Transcript

Michael Horn:

As college costs have sorted, people are paying more and more attention to the value of college. Is what you get from college worth the cost? To answer the question, there's been increasing focus on the ROI, or the return on investment, for students and their families.

Jeff Selingo:

Yeah, that's right, Michael. But as the focus on economic returns from college has heated up in recent years, there's also this recognition from some circles that value is multifaceted. The answer on ROI is a lot more complex than dollars and cents.

Michael Horn:

Indeed, how we think about the value of college is anything but straightforward. So to help us through this discussion today, we're welcoming Ed Smith-Lewis, from the United Negro College Fund, and Karen Stout, from Achieving the Dream.

Sponsor:

Support for this podcast is provided by the Bill and Melinda Gates Foundation, which is proud to support the work of the Postsecondary Value Commission. Because the question, "What is college worth," deserves answers. Learn more at postsecondaryvalue.org.

Subscribe to Future U wherever you get your podcasts, and follow us on Twitter at the handle Future U Podcast. If you enjoyed the podcast, please leave us a five-star rating, so others can discover the conversations we're having about higher education.

Jeff Selingo:

I'm Jeff Selingo.

Michael Horn:

And I'm Michael Horn.

Jeff Selingo:

So, Michael, I know that when I travel around the country, talking to students and parents and counselors, increasingly, a top question on their mind is, "Is college worth it?" Now, of course, this wasn't a question. I don't think a lot of people were asking 20 or 30 years ago, when we were going to college.

Jeff Selingo:

But given the focus on job outcomes for graduates, rising costs, especially as family incomes are not keeping up, the amount of student debt, which we talked about on a recent episode, and then there's the focus of think tanks and foundations on the conversation. And they're bringing a lot more data to this college value question than ever before.

Michael Horn:

Yeah, that's absolutely right, Jeff. It seems to me an important conversation and one, in which, frankly, as a field, we've started to take some big steps forward. But the notion of value is also a complex concept. So we want to go beyond the usual tropes on today's episode, and to help us with that, we have two people who have thought a lot about this question, and spent a lot of their careers focused on student success.

Michael Horn:

Karen Stout is the CEO of Achieving the Dream, which works with hundreds of community colleges on student success strategies. She's also president emerita of Montgomery County Community College in Pennsylvania.

Michael Horn:

Also with us today is Ed Smith-Lewis, who's the vice president for strategic partnerships and institutional programs at the United Negro College Fund. Welcome to Future U, Ed and Karen.

Karen Stout:

Thank you, it's great to be here.

Ed Smith-Lewis:

Absolutely. Thank you for having us.

Michael Horn:

The value of higher education is a term that's thrown around a lot these days. So we're curious to start with understanding how each of you define it. Let's start with you, Ed. How do you define value when you're talking about higher education?

Ed Smith-Lewis:

Oh, well, welcome to the deep end. Value's one of the hardest questions I think we've struggled with in higher education, perhaps from the beginning, but definitely today, when I define the value of higher education, I think much broader than what I think we typically land on.

Ed Smith-Lewis:

I think it's very clear that value, in one sense, is the economic value that a individual receives after he or she has attained that degree and moved into the workforce. But in many ways, that definition is a bit myopic.

Ed Smith-Lewis:

Not only does it not look at the type of employment that a student may decide to enter in terms of his or her career, thinking about the role of teachers, or socially oriented careers that have innate value for individuals because they chose that. Think about the arts. It doesn't really account for that.

Ed Smith-Lewis:

More importantly, I like to think about even beyond the individual. How do we think about the context in which that individual lives, getting to the institution itself, communities and the systems in which we operate.

Ed Smith-Lewis:

As we think about institutions, there's value to higher education, depending on the type of institution you attend, whether it's an affinity- or mission-oriented institution and the diversification it adds to a broader show context.

Ed Smith-Lewis:

When you think about the communities in which these institutions operate, many of them in rural communities operate as the sole workforce provider for institutions, and how do you start to think about the value of the institution in the place in which it sits? Then, most importantly, for me, the systemic value that institutions provide, thinking about how they change narratives for communities.

Ed Smith-Lewis:

As a proponent of Historically Black Colleges, I know that their value, the value of those institutions, stem back all the way to America's original sin, and today, continue to push a counter-narrative, that not only are their institutions themselves valuable, but the education they provide to generations of students are also valuable. So in short, it's a very complex, I think, answer

Michael Horn:

It's super helpful, though, to give us the round out version. And Karen, what about you? How would you define value?

Karen Stout:

Well, obviously, what is considered valuable really depends on the individual and the context that they're sitting in. And that's particularly important when we're talking about community colleges and marginalized populations. You just can't determine value in a vacuum.

Karen Stout:

Our movement, though, has, I think continually tried to do that, and to come up with one single data point that can never tell the whole story of the diversity of the institutions in the higher education ecosystem. And I've been part of a lot of those attempts, not necessarily to get to one data point, but a few data points, that never have been able to tell the whole story, like the Voluntary Framework for Accountability. Each of these has moved forward the field's definition of value, but have been incomplete.

Karen Stout:

The PDP, the Postsecondary Data Partnership, which is beginning to look at some leading indicators to help us understand valuable, but we really need to get to what is valuable for our students, their families, and the communities. That engagement, really, is an intangible value of community colleges.

Karen Stout:

The conversation can't be about individual value. It needs to be partially about individual value, what students get, but it has to also be connected to what improves communities economically and socially, and Ed touched on that. Of course, value is perceived and baked into our systems, in terms of funding, in terms of how we price tuition, and the experience, and in endowments, right? So this is really a complex topic, and I'm eager to dig into it today.

Jeff Selingo:

Defining value is one thing, but probably measuring it is another. So Ed, you've told us that to determine value, students should ask if they see themselves reflected in a college. And when you say that, what does that really tangibly mean? And how can prospective students, for example, do that?

Jeff Selingo:

And in turn, how can colleges themselves better help students understand that that's important, and then help show them how that happens? So could you give us a little bit more tangible examples about how students should see themselves in college?

Ed Smith-Lewis:

I'll start by saying, I'm a first generation in high graduate. When I headed off to Morehouse College, getting a career was top of mind. I have to say, I was looking to change the financial and economic outcomes from not only myself, but Karen, to your point, but my family, and the community in which I left.

Ed Smith-Lewis:

And I have to tell you, almost from the first time I stepped on campus, I realized how narrow my perspective was. By the time I graduated from Morehouse, I had visited 25 different countries. I never thought about value as getting exposure to the world. I had built a set of lifelong friends who had a variety of different experiences, that were separate from mine.

Ed Smith-Lewis:

As a first generation black male, I had assumed most black males in college were first generation black males, but I had a roommate that was a fourth generation college graduate, by the end. When I went and I learned about the diversity of people that looked like me, it changed my perspective so much, that I went from wanting to be a high school teacher, when I left for college to working at an investment bank, ending up in consulting, and now doing a career, I could have never fathomed.

Ed Smith-Lewis:

For me, the opportunities that colleges provide, for many students, go well beyond that sort of single point of economic value and career outcomes. When I think about how students should explore colleges, and I'll tell you, I went to a private day school, and I was told, "Hey guy, you might not want to go to that HBCU in Atlanta, because administrative struggles might be hard."

Ed Smith-Lewis:

I took a leap in doing that, and I took a leap for three reasons. The first reason was the fact that I had never been to the South. So getting away from the communities, in which I knew, was a plus. The second reason was, I had an alumni network that reached out to me fairly aggressively, to say, are all the reasons why I should attend that college.

Ed Smith-Lewis:

Number three, and probably most importantly, and I didn't realize the value of it, until well after I had graduated from the institution, is going to a school where people looked like me, and I had an affinity towards, allowed me to remove the need to be representative of. When I think of my community, right, I got to be myself.

Ed Smith-Lewis:

When I start to think about how do students today evaluate colleges, you have to look beyond what careers and majors do they offer, but for whom do they offer it? When you go to their website, do you feel like you could be there? Does it feel representative of you? Do the faculty and staff look like you?

Ed Smith-Lewis:

When you look at the outcomes for the students, are they speaking to your aspirations? And for colleges, if students do that appropriately, colleges, all you have to do is invest in a sense of belonging for the students that are on your campus. And you do that by investing in faculty and staff that look like the diverse student population you're seeking to serve.

Ed Smith-Lewis:

You speak to the voice and the experiences of those communities. Then you illustrate that in your website, in your promotional material, et cetera, because it's about seeing a student for more than what you have to offer them, but accepting also with that student will offer you as an institution.

Michael Horn:

Karen, I want to turn to you on this, because you both are pointing out the benefits of moving beyond just thinking about the value of higher education in terms of economics. Obviously, we've been getting better over recent years at measuring economic returns, so we tend to lead with that.

Michael Horn:

But obviously, as you both have pointed out, there's much more to this college equation, if you will. How can we shift that conversation, in your opinion, to something more than just economic benefits, as we think about value? How specifically do we start to capture and measure what else is important?

Karen Stout:

I don't think it's bad or wrong to start with those economic benefits, but I think the question has to be larger, and I think Ed started to lead us there, and it's about who's benefiting? And how wide is the impact of that benefit?

Karen Stout:

For us at ATD, thinking about the role of community colleges as important local institutions, and I know HBCUs play a similar role in a number of communities. That means thinking about benefits, economic, social, cultural, in terms of equity, and the community. I know, Jeff, you've written about the need to rethink what student success means.

Karen Stout:

We've been thinking about that a lot at ATD, as well, and we've been thinking about that in three phases. When the student success movement first started, there was really this important shift from access and measuring value, based on who's going to college, to thinking about who's completing, student completion.

Karen Stout:

Of course, now we're shifting again, and we have been shifting rightfully. Completion's now a progression metric, and we're thinking more about student economic gains. And some research that ATD did with strata education a few years ago, and Gallup, really shows some results around looking at value from the point of return on investment, as measured by student and community well-being. And that's really important.

Karen Stout:

I think it begins to get to whether the college experience is building that sense of belonging, and that social capital, that Ed has referred to in his response. And this is where many of our leading community colleges are now. They're thinking about their work in the context of community well-being, or community vibrancy.

Karen Stout:

How is their measuring their success, by the way they're contributing to the economic mobility of their student students, and the families that are associated with those students, and therefore the strengthening of their communities. That really requires us to think differently about the indicators we're using, and to move beyond graduation rates, to a lot of other indicators.

Karen Stout:

It really forces us to think about community vibrancy metrics, and our role as higher ed institutions, and moving the needle in our communities equitably around early childhood education, reading and math proficiency, on time high school graduation. There's a lot of those intergenerational mobility, that specifically, for community colleges, because we have been woven into the fabrics of our community, are really, really important for us to begin to measure value, and talk about value in a way that has broad public policy kind of reach.

Karen Stout:

I think, too, we have to acknowledge that at the heart of all of this is this question of equity, and how do we measure and bring equity measures throughout every one of those indicators that I just mentioned? Anthony Carnevale's quote about how American higher education is the capstone in an education system that's a primary cause of the reproduction of race and class privilege across generations is something that's really important.

Karen Stout:

However we're going to define value, we have to disassemble the gear wheel that we have become as a sector. We need to think about how those value indicators are going to begin to help us break down the systemic barriers. And I think the way we have traditionally defined value has not been beneficial, let's just face it, for Historically Black Colleges and Universities and community colleges.

Jeff Selingo:

I want to talk a little bit about the tools that we use to communicate value, right? During the Obama administration, the federal government launched the College Scorecard, that was very focused on the ROI of college. More recently, of course, the Gates Foundation, which is a supporter of Future U, launched its Equitable Value Explorer, in which students can evaluate colleges on factors, such as racial composition of the student body, net price of attendance, completion rates, et cetera.

Jeff Selingo:

I'm curious to see what you think of these tools, how should they be used, how they can be improved? Because institutions you both represent, as you've mentioned already, don't always come out on top when making these comparisons, using these tools. Ed, let's start with you on that.

Ed Smith-Lewis:

This goes back to, I think, where we've been in the conversation already. The idea that one metric out of context could actually help a student make a decision is problematic. I think we saw with the College Scorecard, that a pure aggregate average, then compared across a set of institutions, with different majors graduating, and different geographies, with different social opportunities and career opportunities for certain people, based off of their generational context, doesn't actually lead to a fair comparison.

Ed Smith-Lewis:

For me, I struggle with both the College Scorecard, Gates's Value Explorer tool, and the Value Commission's work, partially because of the single point comparison. We have to move beyond that, and allow for a context to matter, both for where institutions are located, what students aspire to be, and where they feel most comfortable. Until we can get tools that contextualize at first, just to geographies, right, understanding that economic outcomes vary greatly in the American South, versus some of our mayor metropolitan coastal cities, and that comparing Caltech, and an extreme focus on technical education, engineers, et cetera, to institutions that are producing more socially oriented graduates, teachers, et cetera, isn't a fair comparison for a student to make.

Ed Smith-Lewis:

As I think about the student, too, the tools aren't very intuitive, or engaging, for a student population that I would say is learning to understand what they should value from higher education. In many cases, these tools feel more oriented towards policy makers and decision makers at sort of a different stage in life than student.

Ed Smith-Lewis:

And I think that's where some of the problems I have lie, that the feeling, that the emotions, that the experience a student gets isn't taken into account, when we're sort of pushing tools to help students make decisions. That's really where my challenge is, is the context matters so much more for a student, and many of these tools, although are believed to be developed for students, seem more policy oriented than not.

Jeff Selingo:

So, Karen, how about you? What do you think about these tools?

Karen Stout:

I think you all know that Achieving the Dream has really, one of our central tenants is around helping our institutions with data-informed decision making. So data tools are really fundamental and important to the work that we do with our colleges.

Karen Stout:

I do think there are some benefits to tools that offer comparisons for internal continuous improvement, using tools to do benchmarking against, of institutions, so you can aspirationally see what's possible with your redesign and transformation work internally. I think that the tools are really attempting to put data in the hands of students and families and communities, and that's important.

Karen Stout:

That said, like Ed, I worry when we look to create these large comparative tools, and where we're only relying on the comparative data that we have, which we've already talked about, how that may or may not be shaped in a way that's comprehensive enough to understand the local context and local missions that we have. And therefore, the value is limited.

Karen Stout:

The Equitable Value Explorer and the larger work of the Values Commission is really, really important. I do think that this continuous movement in the field, to learn, and build these tools, is really important. And we keep evolving.

Karen Stout:

But even the Explorer comes with its own warning, and I'll, I'll read the warning. "This tool leverages the best available public data from the College scorecard and other sources, but those data are incomplete, so results should be interpreted with great caution."

Karen Stout:

So it's not surprising, to me, that our institutions don't come out on the top when making these types of comparisons, unless we really are serious about digging into two of the principles in the Values Commission report that really resonate with me. And that is, that equity matters, and that public returns matter.

Karen Stout:

Because, if I think, if we can really go deep into those, and think about new data sets and new opportunities, we may be able to create a tool that's more valuable for the students that are coming to our institutions. We really need tools that are accessible and relevant and locally-oriented.

Karen Stout:

Our students don't even know where to begin, and they aren't looking to come compare 10 or 12 colleges across the nation, or even the state, to figure out what the best path for them is. So localizing these tools is really, really important.

Jeff Selingo:

That's a great point. So let's end on this for both of you. The agenda of college leaders are clearly going to be packed in the coming years, given all the pressures bearing down on higher ed right now.

Jeff Selingo:

As we think about value, in particular, what's one area that you hope or think they should focus on, that will move the needle on a lot of the stuff that we talked about today? What's your advice for the institutional leaders out there, that might be listening to us today? And let's start with you, Ed.

Ed Smith-Lewis:

I think it depends on your starting point. As I think about institutions, let's consider sort of the larger, more well resourced PWIs. I think a big question on sense of belonging, and how do you ensure you're preparing for a diverse student population, is the space where you need to invest a bit more energy and resources, to really be prepared for the changing student demographic that's before us. And start to think about how your outcomes might shift, as that population, or those populations, start to grow on your campus.

Ed Smith-Lewis:

For institutions that have been historically mission oriented, whether those are community colleges, HBCUs, women's colleges, et cetera, really trying to understand how you elevate your narrative, and your value will be critical. What does that mean? I think, in addition to investing in much of the work that ATD leads, in data infrastructure, because I'll agree, that real data hard data will not, not go away, numerical data, quantitative data will not go away, but also, investing in that qualitative data.

Ed Smith-Lewis:

Start to gather your student stories, their narratives, how your educational opportunities has shifted their realities, changed their trajectories. Start to work more closely with your local communities to understand what impact your institution is having on your localized areas. Again, to Karen's point, that, in order to show value, you have to be able to document it. And for those institutions that may have value that is differential from the economic one, how do you start to supplement the data collection, to build that narrative differently?

Ed Smith-Lewis:

A lot of the work we do here at United Negro College Fund is around shifting how we see these data and these outcomes, and our recent report on social mobility understands the importance of economic outcomes for students, but then takes into account the input. It talks about the access rate that leads to those economic outcomes, and how we're pivoting individuals from one social strata to another.

Ed Smith-Lewis:

How do we start to think about the inputs of the institution to those outcomes, and then the lived experiences of the individuals going through those outcomes, or the communities in which these institutions sit? And I think much of the at data will be qualitative in nature, and require a much bigger lift, both in terms of the infrastructure needed, but also the narrative that must be built on that, in order to get others to believe in it.

Jeff Selingo:

So, Karen, how about you? For institutional leaders out there, or even for political leaders in others? What can they really do in the next couple of years to move the needle?

Karen Stout:

You know, I'm really seeing, from a movement perspective, at Achieving the Dream, I just want to say, I agree with Ed about the qualitative look at students, but that we're really beginning to look, because of any number of factors. We're peeling back and looking at programs.

Karen Stout:

This really gets to one of the principles, too, in the Values Commission Report, that institutions and programs matter. And I think we've waited too long in our movement to understand programs, who is coming into programs, and who's benefiting from the programs, and our program mix? And we're really challenged as community colleges at this moment around our relevancy.

Karen Stout:

That's not an easy thing to say, but I think it's because we really haven't focused strongly enough on that program mix. By program, I also am thinking about deconstructing the degree, Jeff, and what we mean by the degree.

Karen Stout:

But we do know, from some recent research, and the Brookings Institute research, that looking at program level data, that earnings and student loan repayment outcomes at more than 1,200 of the community colleges show that we've really misdiagnosed our problem. And we have to redesign our solutions.

Karen Stout:

And we're finding the programs that enroll more underrepresented minorities, black, Hispanic, Native American students, tend to have the worst track records, on average. What's surprising is that field of study, not student demographic characteristics, explains a lot of the variation in earnings across programs.

Karen Stout:

Racially minoritized students are disproportionately enrolling in the programs with lower earnings, and this is unequivocally a design problem, for us as higher education leaders. We have to get to the design problem. And the design problem carries into other programs, like dual enrollment programs, all types of programs, even the way we think about transfer, and transfer programs.

Karen Stout:

For community colleges, I think, moving our thinking to programs and understanding the student journey into and through programs, is really important at this moment. If we have relevancy, value is easier to demonstrate. When you're less relevant, you struggle with demonstrating value. I think we as higher ed leaders need to think about that.

Jeff Selingo:

Well, Karen, Ed, thank you so much for joining us on Future U, really appreciate it.

Karen Stout:

Thanks, Jeff.

Ed Smith-Lewis:

Thank you.

Michael Horn:

And we'll be right back.

Sponsor:

Support for this podcast is provided by the Bill and Melinda Gates Foundation, which is proud to support the work of the Postsecondary Value Commission. Because the question, "What is college worth," deserves answers. Learn more at postsecondaryvalue.org.

Sponsor:

Subscribe to Future U wherever you get your podcasts, and follow us on Twitter at the handle Future U Podcast. If you enjoyed the podcast, please leave us a five-star rating, so others can discover the conversations we're having about higher education.

Michael Horn:

Welcome back to Future U, off of one of the more nuanced conversations I think I've heard around value anywhere, Jeff, and I'm curious to start with, what was your reaction to their definitions of value, and how they're thinking about the topic, given the rise of people you talk to all the time, ordinary people in schools in high schools and so forth, talking about value of higher education?

Jeff Selingo:

Yeah, Michael, I want to zero in on something that Ed brought up, and this is on the type of employment. When we think about outcomes, the type of employment that students are getting after college graduation, there's the paper that Chris Howard, of Robert Morris University, soon to be at Arizona State University, and Matt Sigelman, from Burning Glass, recently did, and I wrote the forward to it. And it was really focused on job outcomes for underrepresented students.

Jeff Selingo:

What it really shows is that many underrepresented students go into higher education, and take part in majors that don't necessarily lead to the best job outcomes. Now, this focus on ROI as a key metric and outcomes really does make a lot of college leaders uncomfortable. And I think money is not the only outcome from college that matters, but it's really naive, I think, to downplay its importance.

Jeff Selingo:

In this paper, as Chris and Matt point out, black and Hispanic students are disproportionately concentrated in majors with lower earnings. And that contributes to the higher likelihood of being underemployed after graduation. What's more, depressed earnings and underemployment can cause really a vicious cycle after graduation, right? So it's resulting in less engaged alumni for colleges, and a more challenging enrollment environment for those institutions, as well.

Jeff Selingo:

Now, I realized that this is not the point Ed was making. Indeed, he was actually arguing a little bit of the opposite, that we need to lift up certain professions, like teaching and social work, which don't always fare well, in looking at these averages, and he's right. But then this gets to his other point about the types of institutions.

Jeff Selingo:

Should someone who's looking to get a teaching degree, for example, go to an expensive private college, where they need to take out tens of thousands of dollars in loans? And maybe that private college has a graduation rate of 60 or 70%, when the local regional public institution has a similar degree. I think that gets to the value equation, in terms of those jobs that Ed was talking about.

Jeff Selingo:

I agree that we need teachers and social workers, and so forth, but should they be going to the institutions where they are going to go deep in debt? Maybe they won't have necessarily its job outcomes of students from other colleges, or even if they do, though, they're going to be kind of in the whole coming out of those institutions. And I think that's the differentiation that I want to see when we talk about value.

Jeff Selingo:

Now, I know we're going to be talking about the tools, in a little bit, but I would love to see more engines like Amazon or Netflix, so that when students are looking at, for example, that expensive private college with not very good graduation outcomes, they also see something pop up, saying, "Have you considered these types of institutions, for that same degree, also within 50 miles of your home, so we're not talking about institutions several states away?"

Jeff Selingo:

Then there was Karen's point about individual value, versus what improves the community. I don't think it's an either/or, as I think she set it up. I agree that we need to think more about the social good, and the social value of higher education, but we shouldn't do that at the expense of the individual good.

Jeff Selingo:

Now, maybe the measures for community colleges are different, right? Because we know that the outcomes now that we measure community colleges by are not great, in terms of graduation rates, and so forth. Most students are going to community colleges, because they either want to transfer to a four-year institution, or they want to get degrees that matter in the marketplace.

Jeff Selingo:

So, rather than judge these institutions, largely on the same ones that we judge four-year institutions on, I would like to see some sort of measure about, what are we trying to get out of this two-year education? If we're trying to get a two-year degree that has power in the marketplace, that has currency in the marketplace, well, then, that's their ROI.

Jeff Selingo:

Or if you want to transfer, that's your ROI. But again, I think this goes probably more to the conversation about the tools than anything else. So Michael, what about you? I'm actually more curious to get your own definition on value, because I know that this is a topic you spent a lot of time on, dating back to you creating a quality value index for institutions over a decade ago, and more recently, co-founding the Education Quality Outcome Standards Board.

Michael Horn:

Yeah, Jeff. Well first, upfront, I'll just say, everything you said, I agree with. I think they're really well taken points and things that people ought to pay attention to. I was taken with both Karen and Ed's answers, because I think they recognize fundamentally, though, that value can only be defined relative to the progress that an individual, or, I suppose, a stakeholder, in the case of thinking about the value to a community of a college, is seeking from the experience.

Michael Horn:

I wrote about this recently for Forbes, and talked about how all of our metrics around college outcomes are really what I would call supply side metrics. They look at things from the perspective of institutions, and what's easier to measure. In my view, that's in opposition to what I'd call demand side metrics, the outcomes to what you were talking about, desired by each individual learner. And I think demand side metrics are really hard, I think, to capture.

Michael Horn:

Because, from a student perspective, you have to start with understanding first, why is someone enrolling in a college? And then, build from there. Are they looking to step it up, Like Ed was, and get a great job out of the experience? Are they looking just to get away from their current reality?

Michael Horn:

Maybe they don't really know what the outcome that they're seeking is, in some cases, and success, I think looks really different, in those instances, and individual stories rather than averages actually matter a great deal. But I guess I'd come back to where you highlighted. I'd say none of these conversations, though, matter if something is unaffordable from the get-go, and it's going to throw someone into something that they simply can't afford.

Michael Horn:

I do think the ROI, the return on investment conversation, is a really critical part of this value conversation. Because, as Karen said, I think it's an important picture that we're doing a better job of filling in, is table stakes for the bigger conversation, but we need to address it.

Michael Horn:

As we talked about in the episode, Jeff, with Beth Akers and Josh Mitchell, for the student that wants to go into the arts, and is in position to afford a big price tag for little economic return, that's fine. But if you can't afford that, then that's not fine. And I think rather than have colleges shirk that reality, they ought to be able to say, "Okay, how do we make the programs that do have less of an economic benefit far more affordable to those individuals?"

Michael Horn:

Because it's not that the arts are unimportant, or that there aren't other parts of value that one would get out of them. But just that if they're unaffordable from the get-go, then they're unattainable, and the conversation, to some degree, is moot.

Michael Horn:

Now a problem, of course, is that the data today we're getting about these programs, they calculate an on average cost in the denominator. As you know, students may pay dramatically different prices for college. An ROI calculator, or something that bases off the averages, as opposed to what you as an individual will actually pay, I do think creates some awfully misleading results, if you will, for certain schools or certain students.

Michael Horn:

Now, that actually leads me, you though into my next question, which gets at these tools, which is, I thought both Karen and Ed raised really good points about why there's a lot of data that shows that students, and low-income students, in particular, don't actually really use the data sets that are out there. Because they aren't really relevant to the colleges that they're comparing, or that they're thinking about in their decision.

Michael Horn:

And you alluded to it, Jeff, so I'm curious about your thoughts. You talked to a lot of students, many of whom, I'm sure you've found, who have said to you, "These tools are utterly unusable, or useless." Are we seeing changes in your mind to that now? Or what's it going to take?

Jeff Selingo:

Yeah, I mean, they're absolutely right on that, that these tools aren't very intuitive, or engaging, that they're geared toward many cases, policy makers, or others who are really well familiar with a higher education, particularly institutions themselves. And I'm also really surprised, Michael, on how many people don't know about them.

Jeff Selingo:

I probably bring up the College Scorecard at every talk I give, and people always say, "What was that? Could you repeat that again? Could you tell us where to find it?" And these are at high schools with resources, with big college counseling offices. And they still don't know about these resources, that there's as a federal government website.

Jeff Selingo:

So I can't even imagine, at high schools with even fewer resources, what they're actually using or what they know. When it comes to these tools, as I said earlier, I wish that there was more of the nudge, of a recommendation. As students are finding colleges on ROI, as they're looking up earnings by major, by institution, as they're looking up different sizes of institutions, whether they're doing on the College Scorecard or elsewhere, I wish that there would be a little bit of a nudge to say, "Did you consider these other institutions?"

Jeff Selingo:

Now, I realize that institutions don't want that to happen. I mean, there was enough of a fight about the College Scorecard, that they're not going to want anything that drives students away from a college, beyond the data. In some cases, the data alone will drive students away from a college.

Jeff Selingo:

But I think that what we do need, though, is something that gets a little bit at this recommendation piece. Because that's what students really need. Something needs to help them make sense of a universe with thousands of institutions, and that's really confusing to them.

Jeff Selingo:

Even if they're going to college within a hundred miles of home, as many to students do, we're still talking, perhaps, dozens of colleges, some that might be good choices for them, but some that might be bad choices.

Jeff Selingo:

But it also gets to something that you said, and that Ed said, and that Karen said. What do you want out of your higher education experience? Because right now, all of these tools treat students as kind of a one size fits all entity.

Jeff Selingo:

It's interesting, I bank with Wells Fargo. And I mentioned that they're not a sponsor, or anything, but I was on their website recently, and they have this thing called the Intuitive Investor Questionnaire. If I wanted them to invest my money, they first asked me, what are my goals? What are my risk tolerance for different things? How often do I want to use this money?

Jeff Selingo:

There's a lot of AI type of questions in there, that get at my use of money, my risk tolerance, and so forth. And I would love a tool that did the same thing with students, right?

Jeff Selingo:

What are you looking for? Again, if you're going to go to a two-year college, do you want to transfer, or do you want an associate's degree that's going to have some sort of currency in the job market? These are the questions, before students ever see anything ROI displayed to them, I wish that we would ask these questions first, so that we know what to display to them, put them in context.

Jeff Selingo:

Then there was something else, that they said, that really resonated with me. And this was about, we treat this as a national database, when most students do go to college within a hundred miles of home. So they're not comparing, for example, Caltech with MIT. They're really comparing regional public colleges, with maybe community colleges, and maybe a private or two.

Jeff Selingo:

So it would be better, I think, if these tools operate it more exclusively on the local or statewide level, and then looked at ROI at that point, as well, because many of these students might want to go to school within a hundred miles, and they might want to stay close to home, to work afterwards.

Jeff Selingo:

Getting a bachelor's degree in, say, my hometown in Northeastern Pennsylvania, and then staying there? You could have a pretty good ROI on that, given the low cost of living there, and the fact that very few people there have bachelor's degrees.

Jeff Selingo:

It actually looks better, for example, if you stay there, than if you were to go somewhere else, and go to New York City, or Washington or Boston, where that ROI might not be as good.

Michael Horn:

Yeah. And Jeff, you actually did a good job in your book of chronicling one student, as I recall, I think in Pennsylvania, if memory serves correctly, who all of a sudden, got down to the limit, and made this decision sort of bizarrely, right, brought in a new choice, if I remember the story correctly.

Michael Horn:

Two thoughts toward that. One, briefly, I do wonder what the role of perhaps social capital could be in this? And what I mean by that is, a lot of people's search choices are informed by the people around them they know that when to a certain school, or their counselor.

Michael Horn:

I wonder if some of these recommendation engines might pair you up with someone who could maybe give a couple different ideas of something, to put a more of a human face on it? Because some of these choices are not deeply numbers driven, analytical, but to your point, they realize, "Gee, Michael might benefit from knowing there's these two other schools in the region. Let's connect him with an alum who went there, or looked at it, or whatever," so just a thought on that.

Michael Horn:

And then, the second point around comparisons, and what you're really comparing a school or program to? It's one of the reasons when we built the EQOS Framework, the Education Quality Outcome Standards Board Framework. We really wanted to do it from a value added perspective, so asking, "What's the increase in earnings, relative to what a student from a particular background and circumstance would likely earn, if they didn't enroll in this program?"

Michael Horn:

For some students, that would help tease out what our friend Ryan Craig always likes to say, which is, "A lot of the kids who go to Yale and Harvard would be just fine, and the earnings they see are really about selection bias, not the quality of the school, per se."

Michael Horn:

But, at the same time, you and I both know, that for a low-income student who does go to an Ivy league school, the experience is transformational for them, and their family, from an earnings perspective. I just think, factoring that value added perspective, the counterfactual, and then, comparing like schools to like schools, I think, would be a real important contribution to evolving these tools, Jeff.

Jeff Selingo:

All right, Michael, we're going to leave the conversation there, and tackle more on this value equation on a future episode. But for now, we have a listener question from Elizabeth C., On LinkedIn, and she asked this, which is really pertinent, I think, to this conversation around value.

Jeff Selingo:

She says, "When so much of the Best Of guides include things like expenditures on athletic centers, and other accoutrements, how can we dig deeper into the value of what students learn?" And she emphasizes that word, "learn."

Jeff Selingo:

In other words, how can we rank the value of learning students receive? We've talked about this for so long in the US, about those institutions that are actually taking students from one place coming in, and by the time they graduate, they actually could showcase their learning. But what do you think of that?

Michael Horn:

I have two totally different takes, Jeff. And then I'm curious, yours. The cynical take on me is, in some degree, it doesn't matter. There's a black box called college, you enter it, and if it produces great outcomes because of the social networks that you get into, the experiences you have, the mentors you have, the learning, maybe you was transformational, maybe it wasn't. But if it produces that ROI, and that value, broadly speaking, that you wanted out of it, I'm not sure that it matters to some degree. That's the cynical take on it.

Michael Horn:

The second side of it is this, which is, it's really tricky to compare learning outcomes across programs, for a whole host of reasons. The EQOS Framework that we designed does have an area for learning outcomes. It basically says to schools, that if you claim that your value is tied to learning outcomes, put forth an objective measure, and do a pre- and post-rate assessment, essentially, or portfolio of work, or something like that, to verifiably and reliably showcase the learning that your students have.

Michael Horn:

That could be, if it's an entrance exam into a professional field, like doctors, nurses, healthcare workers, the bar exam, et cetera, then there is an objective standard, that you can look at pass rates, for example. We ought to look at that and see, how did students do?

Michael Horn:

Alternatively, I think schools could do a better job of creating objective assessments themselves, that they think are indicative of the learning outcomes that they produce. Is citizenship something that they claim would be an outcome of your program?

Michael Horn:

Certain schools would say, "Yes, measure voting, for example," but I'd like to see schools, and preferably, bands of schools with like programs, come together and create some common measures, so that we could have this conversation more robustly, so it's not defined in a top down way, from say, the state or federal government, which I think could get dangerous really quickly.

Jeff Selingo:

Yeah. I think, Michael, I don't really have much to add to that great answer to Elizabeth's question, except that we really measure things, and this goes back to the whole ROI, we measure things that we have easy access to the data for. Doesn't mean we should, but we do in higher ed, because we have that data available to us.

Jeff Selingo:

The things that we should measure, we don't, because we don't have access to data. And this is a hard one to answer, as a result of that, because we don't have easy access to the data that we need, in order to measure student learning. We'll leave it at that, and thanks to Elizabeth for asking that question on LinkedIn.

Jeff Selingo:

Please send us your questions. If we have one, and if yours appears on the episode, you'll get this great Future U service Tumblr sent to you, so please keep those questions coming in. And until the next episode of Future U, you stay safe, and we'll see you in a couple weeks.