CHICAGO — Every few years new technology emerges that promises to improve student success and make it easier for college officials to do their jobs. 

Some tools have delivered on that promise, but many face blowback when they don't quickly fix the problems they were designed to solve.

Nudging, for instance, has helped some institutions improve retention and reduce summer melt, but recent studies have shown the digital alerts are not so effective at scale. Other popular technologies, such as predictive analytics, must be used carefully to avoid limiting students' options by putting them on easier paths rather than giving them more support. 

To learn what leaders in higher ed's technology sector see as the positives and pitfalls of these tools, we asked attendees of Educause's annual conference, held in Chicago last week, two questions: What problem in higher ed has ed tech yet to solve? And what ed tech solution is overhyped? 

Here's what they had to say. 

These responses have been edited and condensed.

What problem in higher education has technology not solved that you think there is an ed tech solution for?

Ron Yanosky, research director, EAB

I've heard some chief information officers (CIOs) say they feel emergent AI technologies, like machine learning and robotic process automation, have improved substantially in the last five years or so. But there are cultural expectations about interaction. People like talking to human agents, and so if they get the impression they are talking to an automated agent, even if they are getting faster and more accurate answers, it still feels like it's not as good of a service. 

That may be changing. I'm hearing CIOs say that the newest students are more familiar with and more tolerant of that technology or even have a preference for it, and we're starting to see institutions experiment with things like automated chatbots. So there's a perception, at least among some CIOs, that this is technically ready but the change management and the culture surrounding it isn't quite there yet.

Kevin Pollock, outgoing president, Montgomery County Community College

I try to look at it from a higher point of view. One thing that needs to be solved is having tech at the front of the table as ideas are being crafted. Everyone thinks tech is a support mechanism. Tech needs to be involved at the very beginning of the discussions so they can be creative and come up with ideas. It's a problem that by the time tech gets involved sometimes, they're just asked to come up with pieces. 

Mike Silagadze, co-founder and chief executive officer, Top Hat

Lots of companies have offered universities analytics packages that suck in data from different sources and use fancy artificial intelligence (AI) to help drive better student retention outcomes. The ultimate measure of success is whether universities continue to use and invest in those systems, and the answer is no.

There was a lot of hype around this maybe four or five years ago. They've since become much more in-house initiatives, from my assessments. I think the solution is a more end-user focused intervention, at the faculty or student level, rather than giving the university a bunch of dashboards and analytics.

Bryan Alexander, founder and principal, Bryan Alexander Consulting

We need to reduce the price of higher education in a way where we can still have excellent higher education. That's been part of the promise for educational technology, and we haven't been able to do it.

Jill Leafstedt, associate vice provost for innovation and faculty development, California State University Channel Islands

I don't think we have fully achieved what we need with openness. The web has allowed the creative commons to explode and allowed us to share all of our materials. I don't think our idea of copyright, publishing and knowledge creation has quite caught up to where the possibilities lie. Ed tech could play a role in easing and transforming that publishing process a bit. 

What education technology has been overhyped?

John Baker, president and CEO, D2L

There's a ton of promise in the field of machine learning and AI, but we've got to focus on making the educational system more human. It's not just about automation and using prediction engines as judgment machines. We should be using them to free up time to help people get back to more classical educational experiences that we envisioned a thousand years ago: desire, passion, pursuit, excitement. Be a better nurse, be a better actor, be a better researcher, be a better musician, be a better entrepreneur, explore big questions in our life. These are the things that matter, so getting caught up on technology for technology's sake is the thing that I would say is overhyped.

Jared Stein, vice president of higher education strategy, Instructure

One of the problems higher ed still struggles with is helping students plan for their future coursework. There have been a lot of solutions but I haven't seen any that really enable a student to say, "Four years from now, when I finish my degree, these are the courses I've taken, in this order, to get there." That's a combination of not only having institutional information about the future but also having planning tools students can actually use. Getting all that information centralized is hard. Once that happens, it needs really brilliant technology to make it easy for students to understand where they are, where they need to go, and how to get there.

Mark Milliron, co-founder and chief learning officer, Civitas Learning

Any of these technology solutions can be overhyped. Ten years ago there was a lot of hype around learning analytics, but there's been a realization that it's a lot harder to do learning analytics well. Predictive analytics was probably one that was hyped for a long time. People are realizing no, it's the way you do predictive analytics that really matters. Right now everyone's moving everything to the cloud. Everything can be overhyped if you do it in the wrong way.