Hours of talk and abundant financial and other resources have been devoted to diversity and inclusion on campus, but there is less-clear evidence that colleges and universities are checking to see how those initiatives are paying off.
Well-intentioned leaders believe they are doing enough by making diversity in recruitment part of the mission and establishing offices to promote inclusion, but according to Samuel Museus, a professor of education studies at the University of California, San Diego, and the author of a key report on measuring those efforts, they aren't looking deeply enough at their effectiveness.
"Institutions struggle a lot in these areas. Leaders at colleges and universities often think there is a quick fix to the problem," and that drives them to evaluate initiatives superficially, he said. "They can create much healthier environments if they approach this work as an ongoing struggle to understand how to create an education system that is responsive to their current student population — a population that is continuously changing."
Finding a better model
Assessments too often are scattered and not comprehensive, Museus said, only giving colleges a "partial picture of reality."
He developed a process for evaluating efforts at diversity and inclusion that he said are built on a well accepted but inadequate historical model from the 1980s. The Culturally Engaging Campus Environment Model of College Success helps colleges establish goals and assess progress.
"Most universities, including ours, use climate surveys, [which are] statistical measurements of recruitment and retention gains, to quantitatively measure the effectiveness of diversity equity and inclusion efforts," said Nicole Hodges Persley, associate dean for diversity, equity and inclusion (DEI) in the College of Liberal Arts and Sciences at the University of Kansas. "I appreciate quantitative data, but effective measurement of success also must analyze data with qualitative approaches that see people as people and not statistics."
Establishing DEI as a priority may be easy, but meeting goals is harder. And measuring success appears to be an area where higher education still is finding its way.
Cornell University, for example, took significant steps to evaluate its efforts more than five years ago and now tracks such efforts with "diversity dashboards" that continually examine four key elements of DEI: composition, engagement, inclusion and achievement. Last spring, Inside Higher Ed reported about how community colleges, too, are examining their efforts. It highlighted Sinclair Community College in Ohio, which undertook a diversity audit with a consultant to gauge hiring, enrollment and support for minorities as well as community relations.
Improving diversity through recruitment
Farah Imtiaz, an analyst with the educational consulting firm EAB that specializes in recruitment, said increasingly sophisticated data is available to colleges about their efforts to broaden their incoming class in terms of race, income, gender and gender identity, and even geographic region. Imtiaz said EAB has a new platform that can help institutions examine and develop their diversity recruitment efforts.
"I'd be surprised, however, if most colleges are measuring their efforts with very elaborate data," she said, noting that they more likely look at general historical numbers and percentages for various groups rather than at factors such as whether they might find successful low-income Hispanic students at a high school nearby where they previously haven't devoted many resources.
Jennifer Ziegenfus, associate director for recruitment at Towson University, in Maryland, said the university has data showing it consistently has improved diversity in a number of areas. She adds that such information is easy for colleges to examine, and most do. However, she added, her office doesn't spend too much time on such information but rather establishes diversity as a goal, examines those efforts and structures critically, and often considers anecdotal, information, especially about how their message is coming across to prospects.
"We have spent a lot more time asking students why they are attending and interested," she said.
Towson found, for instance, that when it evaluated its largest growth from a nearby county with a high minority and low-income population, a message about Towson being a good fit for those students was resonating. That, combined with data suggesting such students might want to be closer to home, has made the university more excited about prospects in that area.
"They can create much healthier environments if they approach this work as an ongoing struggle to understand how to create an education system that is responsive to their current student population — a population that is continuously changing."
Samuel Museus
Professor of education studies, University of California, San Diego
Rebecca Zwick, a professor emeritus in the Department of Education at the University of California, Santa Barbara, and author of the book "Who Gets In? Strategies for Fair and Effective College Admissions," agrees with that approach. She said some schools have found it beneficial to look at non-academic qualities in prospects, although they can be difficult to measure.
EAB analyst Gelsey Mehl said colleges and universities need to consider other layers of information as they look at admissions. She warns that state bans on affirmative action may limit setting targets for racial diversity, but that self-examination is acceptable.
First, they should consider what they mean by diversity, she said. It is often assumed diversity relates to race or gender, but it also can include older students, veterans, international students and those from rural areas.
Donnell Wiggins, assistant vice president of new markets for admissions at the University of Dayton, agrees. "It's also about students with learning disabilities, students who are from the LGBTQ community or students who are low income," he said. "We have to consider how we are doing in a wider array of areas."
Colleges also should consider how their enrollment numbers compare to previous years and to similar schools as well as how demographics may change over time, using good data and well-conceived projections about the future trends. In addition, they should look at application and yield rates for underrepresented groups and data on aid.
"Student success is also part of the equation," Mehl said. "Colleges should look at metrics like first-year retention and graduation rates for underrepresented populations and their change over time. It's not just about enrolling these students — we also want them to graduate. She added that schools should benchmark underrepresented groups against the general student body or more "advantaged" groups.
Wiggins said Dayton, a Catholic university that is aggressively seeking more diversity, looks at county and school data to see how the demographics there are changing and how its draw from a region or school is keeping up.
"How many applicants have we had from a certain county — and is our enrollment from there up or down based on what is happening there? Where are we being more successful as we look at that data?" he said.
Maintaining diversity through retention
The issue of success is critical in discussions about inclusion. That is, whether students and faculty members from various groups feel welcomed, supported and heard.
Zwick said recent research shows the college enrollment rate for low-income high school graduates has increased in recent decades but that the percentage with bachelor's degrees has changed very little.
"Everything in between admission and graduation is important to consider as well," she said. "How well are students from underrepresented groups doing academically, socially and financially? How satisfied are they? Are they receiving the services and resources they need?"
Museus' framework is intended to carefully examine those factors, dividing nine measures into two categories:
- Cultural relevance: How well a campus environment represents factors such as students' backgrounds, if the institution provides opportunities for students to connect with similar students and whether they can learn about their culture on campus.
- Cultural responsiveness: How well the institution supports diverse students' needs, makes them visible and accessible, and whether those students feel they can connect with faculty members and get their help. (A set of surveys using the framework is available from the National Institute for Transformation and Equity at Indiana University.)
Most importantly, Museus said, higher education officials must honestly examine whether such efforts are a priority.
"Campus leaders often think that they can create an inclusive and equitable campus if they tack these goals onto a long list of other objectives that they already hold," he said. "The problem is that these other values, such as acquiring more resources and prestige, often take priority over inclusion or equity efforts, which become a secondary superficial priority at best and forgotten by many members of the institution at worst."
