Category: Evaluation

Design, Evaluation, Quality

Designing for a Smaller Future

Missy Alexander

The news about projected high school graduates in New England is not encouraging. Here in Connecticut the projected drop in high school graduates over the next ten years is 12%. We are all still reeling from the COVID drop and the impact of the last ten years of declines in high school graduates (@ 9%). In a nation obsessed with continuous growth, we are now in a conundrum. How do we balance our budgets in the face of continuous declines? Even if those projections turn out to be overly pessimistic, it seems it is time to design for less.

We often start this conversation as “how do we do more with less?” This is not a good idea. We can’t do more than we’re doing. In fact, we are already doing too much. In the lovely period where student populations kept growing, we allowed ourselves to add programs, courses, departments, and schools as ideas arose. This was exhilarating, to be sure. The freedom to just keep adding bolstered inventiveness and creativity (both wonderful things), with little concern for a future with fewer students. But trying to sustain all that we have created is not a path to sustainability. Nor is it a path to excellence. It simply strains resources so that nothing is properly supported. No, we should not try to do more with less. We should try to imagine our way to something that is smaller but still infused with inventiveness and creativity.

As soon as I suggest that we need to be smaller, our natural response is: “what will we stop doing?” This is both a reasonable question and a necessary reality. We have to stop doing some things because the spread of what most universities are doing is unsustainable without continuous growth. All institutions know which academic programs are thriving and which are not. They also know which co-curricular programs are thriving and which are not. All of this is pretty easy to see. This knowledge can point us in a direction for getting smaller, but it doesn’t tell us where we want to go. I think we need to start with a different question. Instead of asking about what we should cut, it might be more productive to consider what we want to achieve.

It is at this point, that I always think about missions. This is ours:

Western Connecticut State University changes lives by providing all students with a high quality education that fosters their growth as individuals, scholars, professionals, and leaders in a global society.

This statement is similar to those at peer institutions in the region and as such it does reflect some about how we see ourselves. But to make it truly useful we need to be a bit more specific. Here are some ways we might do so.

Let’s start with our desire to provide a high quality education. What are the essential elements of a high quality education? Universities were not always this sprawling and they did not always have so many choices of programs, courses, and majors. Since there were obviously some high quality experiences a century ago, it is clear we can achieve quality education with fewer options. But we need to figure out what the essential elements should be. This should be clearly defined in the general education curriculum, and in the balance of programs offered. How can we define that essentialness and then design from there? Can we do it without just defaulting to our favorite lists, but instead map it to our learning goals for our students? Can we consider some evidence-based practices? Can we approach this process as designing for quality, instead of cutting for financial reasons?

The same can be said of the structure of our majors. I’ll show my age when I note that the entirety of a liberal arts major used to be around 30 credits. This slim approach to the major seems to have disappeared over the last 30 years, as we all added more specificity to our programs, trying to reflect the breadth of ideas and respond to new skills or other developments. Well, I have no hope of a 30 credit liberal arts major, but I think it might be time to rein things in. We all have program learning outcomes: how might we achieve those outcomes with fewer requirements? Can we redesign courses to that end? Will this allow us to produce schedules that deliver the promise of a well-rounded major, without overwhelming our students? Can we approach this as designing for learning, instead of reducing choices for efficiency?

As for our co-curricular programs, might we consider how they help students grow as individuals, professionals and leaders in a global society? Our habit is to let our co-curricular programs emerge with our students’ interests each year. Some of this is to the good, but attendance and participation levels tell us that we could be doing better. Is it possible to have fewer programs, but get them focused on specific university goals? Could this foster greater coordination and collaboration between Academic Affairs and Student Affairs? Can we see this as supporting our mission to change lives, instead of over-direction?

The questions I am asking are not specific to WCSU. Every regional comprehensive in New England is facing this demographic shift. Every regional comprehensive will need to figure this out in a way that leads to a sustainable future focused on great education. As we all adjust to the projected demographics and the process of designing ourselves for a smaller future, we might be well-served by focusing on the idea of design. Instead of focusing on what we currently offer and how it should be reduced, we should focus on what we want to achieve and design to that end. This will give us the map to how to be excellent, even as we get smaller.

It will also make clear what we don’t need to do anymore. That will be a result, to be sure, and one that is uncomfortable. But starting with the goals in mind could create excitement about what we are building. It might inspire our imaginations, leading us to something altogether new and interesting. Most of all, it might help us see the power of what we are building instead of just feeling the losses of what we are cutting. This might just be a productive strategy for imaging a smaller, more sustainable future.

Evaluation, Quality

Assessment is fun?

Missy Alexander

In 2006, when I was an assistant professor on the tenure track, I wrote an essay that was published in the Chronicle of Higher Education. Those were the early(ish) days of the assessment movement in higher education and I was feeling the pain. The essay was really meant as an homage to the faculty in my doctoral program who had taught me to love playing with ideas. The loss I was feeling was what I described in that essay as the loss of the chance to wander. Specifying my learning outcomes seemed to rein in that wandering to the detriment of a good Socratic learning experience.

Well, as luck would have it, I wrote that little essay at the same time that the Spellings Report was released. My essay was attached to the back of a special section devoted to the work of the Commission on the Future of Higher Education and the Chronicle gave my essay the title, “Taking All the Fun Out of Education.” I received a lot of feedback, mostly from other faculty feeling frustrated with the too many rules that assessment seemed to be creating. I also heard from my boss and found myself in charge of assessment in my discipline and then my school shortly thereafter. That’s what complaining will get you.

Since that time, I have moved into administrative roles and the largest part of my job is assessment. When I served as the Dean of Arts & Sciences I was responsible for making sure that every program had an assessment plan and that the plan was followed. As Provost, I’ve expanded my focus from the assessment of programs to the assessment of general education, the development and assessment of university outcomes, and the appropriate measures of our academic success programs. I read numerous program review reports every year, serve on visiting teams for university accreditation, and coordinate the writing of our institutional self-study. It is assessment all day every day.

Today I embrace assessment in ways that I did not in 2006. There is value in setting goals at the course level, in the major, for the whole degree, or for a specific program, if those goals are not overly complicated. When faculty and program administrators take the time to define those goals, they invite us to consider the best strategies for achieving them. The goals are an invitation to start conversations about teaching strategies and the outcomes we value most as a university. They are an invitation to share ideas with our colleagues and to engage the robust literature about innovative and inclusive pedagogies. They are also a clear path to articulating our value to students, families, and the larger publics we serve.

I see the value and even the positive impact that assessment can have when our goals are not overly prescriptive. We do need to make room for some wandering, some inspiration, and some innovation. If we set very narrowly defined goals, we will stop all of that good stuff from happening. Still, there are some things that students must know. Chemists should know enough to keep laboratory work safe and productive. Nurses must know how treatments interact to protect the health of the patient. Musicians need to understand the scales that underpin the compositions they are creating. Historians need to understand the process of vetting information as they place it in a context for describing connections between events. Yes, there are some basics that might well be measured in traditional exams with right and wrong answers. This stuff is important.

But there are lots of places where our goals are broader and perhaps even more important. These goals are about the perspectives each discipline can offer, forms of reasoning to be cultivated, cultural awareness to be explored, the ability to communicate effectively, and the ability to weave new information and experiences into a defensible worldview. These goals need to be assessed, because they are the heart of an undergraduate degree. They help us describe the value we add to the lives of our students in terms that we recognize as important and meaningful. Assessment for the broader goals is more nuanced than an exam, but it doesn’t have to be overwhelming. The key is focusing on a few important examples, not on everything.

But you know all of this already. The point I am trying to make is that using our assessments to refine our strategies can actually be rewarding. Seeing the impact of a change in curriculum or pedagogy or other interventions can be thrilling, especially when the results are improved outcomes. Letting that strategy go when it isn’t yielding results is also rewarding. It settles a question and helps us move on. The critical thing is not just the doing of assessment, it is using our results.

I’m not grumbling about assessment anymore (although I still want to protect time to wander through ideas). We shouldn’t overdo it, because then it will overwhelm us, and it will lose its value. But we must be sure to fully reflect on the results. We need to carve out time for the conversations that should ensue in departments, on committees, and with the full university after each assessment occurs. It is in those conversations that we will find paths to improvement. It is in those conversations that we will continue to develop our visions for great learning experiences. It is in those conversations that we might have some fun.

Evaluation, Growth Mindset

Staring at the Data

Missy Alexander

Let me give it to you straight – WCSU is struggling to improve our retention rates. As long as I have been at this university (since 2012), we have been trying to figure out why so many students leave prior to degree completion and why so many leave after the first year. We have tried numerous strategies, each time hanging our hopes on a new effort, hoping to crack the code. Here is the list, since 2012:

We had a limited FY program (not required, so only partial enrollment). Part of that program in my first two years here included cohorting of classes for FY student (they took three together). Though the idea was a good one, it did not have the desired effect. Without full community backing, the impression among the students was that they were being punished in some way. Among the students involved it had a (very small) negative impact on retention. We stopped doing that.

We now have a full FY program (required as part of our general education curriculum). Departments have developed courses that act as extended orientations to the discipline, which seems to have improved our students time to degree completion (that number has been moving up from 48.9% to 53% starting with the 2012 cohort). This, combined with published four-year plans and a general education curriculum that has fewer hidden requirements than the prior version, all seem to be moving us in the right direction on graduation rates.

Unfortunately, the FY program and the related efforts did not improve our retention rates which hovered around 73% for years, with one up-year – we hit 75.5% in 2019, only to crash to 69.9% in 2020. That is definitely a COVID impact. We’ll see how recovery goes.

I’ve been paying attention to our data, and the number one predictor of student retention at WCSU is High School GPA. Students with an 85% or higher high school GPA tend to be retained in the 80% or higher range (often as high as 85%, which is a great number for a regional public comprehensive university). That number drops immediately at 84% high school GPA (down 6-10%) and with a pretty linear correlation with each point dropped. A perfectly nice B/B- high schooler is at a 10% or higher risk of being lost in the first year than the solid B/B+ or higher. This data isn’t just WCSU, it is a national trend. This is data we can act upon.

And we have. We’ve launched a peer mentor program to address students in that B/B- category. We’re hoping just a little support from their friends will help them stay. A group of folks from tutoring, the library faculty, the first-year coordinator, academic advising, orientation, and our alternate admissions team gathered together to make a plan. They agreed on a training plan for the peer mentors and recruited a diverse group of students to do this outreach. They are now analyzing what worked and where to improve for next year, and the team is expanding. We won’t know the impact on retention until the fall, but this work has been exciting, and it feels good to take action and measure results.

With this program launched, there is more data to consider, and it is time we do so. The director of institutional research has been running some queries on a few areas variables that seem to be related to retention. The two reports he just sent my way were startlingly clear. High school GPA is predictive across majors (not just as a general indicator of losing a student): the higher percentage of students in the major with an 85% high school GPA or above, the higher the retention rate. Not surprising, based on what we knew before this point, but here is where it gets interesting. There is a striking exception to this relationship, and it occurs in a major that is organized as a very close cohort. The students who started in that cohort, and were not fully successful in their original major, stayed at WCSU even when they had to change their major.

What can we do in light of this data? Perhaps a deeper dive into the reasons why some degrees are attracting more students with lower High School GPAs and making a more specific series of supports in those majors? Maybe. Or maybe we should think about how to create cohorts in programs where they aren’t naturally occurring. Maybe this is the glue that will help the students succeed. With all of the national data the links student persistence and transparent pathways through degrees, cohorts might help us focus those pathways even further. For a majority commuter campus, cohorts might also help our students connect with peers and feel connected to the community. Cohorts provide exciting possibilities that I’m keen to explore with my colleagues.

The second piece of data was about math. Math success is not a new problem and our math department has tried numerous strategies to try to improve student outcomes. They are working hard, incorporating new technologies and trying new approaches. Nevertheless, we have a persistent problem.

A review of five years of data shows us the following:

  • A shocking number of students are placed into Intermediate Math (credit bearing, pre-general education math): Nearly 50%. This is true for students with above an 85% HS GPA and those below it.
  • Students who earn a D, F, or withdraw from any math class in their first year are retained at a rate between 63-68%. Those who happened to fail Intermediate Math with embedded remediation are retained at a rate below 50%.
  • Students who fail any math class in their first year are 20% more likely to have a high school GPA below 85% than those who pass.
  • Students who pass any math class (including the one with embedded remediation) are retained at about an 82-84% rate.
  • The retention rate for students who take no math class in their first year, drops back down to a 70%.

Given all of this, we might want to re-think our math strategies. We need to dig deeper into the patterns of who is ending up with the D, F, or W in those first courses, to see if we have a few other predictors besides HS GPA. We appear to need better placement strategies than those we are currently using. We might even need a prep-course, prior to placement testing. We probably need a differentiated delivery of our curriculum based on a combination of variables, not just our placement tests. We may need some kind of intersession process to keep students from failing or withdrawing in the first place. We’ve tried variations on each of these in the past, but like the FY program done half-way, it hasn’t worked fully. It is time to focus and develop a plan for better math outcomes. Those outcomes are so directly related to our retention rates that it is imperative that we do so.

Yes, I’m staring at the data and trying to imagine next steps. I know I have a community of colleagues who want more for our students. I know I have a community of colleagues who are eager to find better paths. I’m hoping my colleagues will ask even better questions of our data, and embrace trying new things as a result. I hope those new things are coordinated, measured, and refined as we continue to learn more about our students. I hope all of this, because one thing is certain, staring at the data is not enough.

Engagement, Evaluation

Learning from Students

Missy Alexander

For the last ten years I have been a full-time administrator. In that time, I’ve focused on student learning outcomes and university effectiveness. I’ve obsessed over better pathways through WCSU, hoping to eliminate the unintentional barriers to graduation and policies that are too heavy handed, punishing all students for the poor behavior of the few. I regularly review all the data I can gather about who is succeeding and who is not, trying to address gaps and make things better. Some of those efforts have been effective, improving our overall outcomes; some do not seem to have made a difference. Nevertheless, I forge ahead in that continuous improvement cycle, because it is my job and because I care.

This semester, due to a series of events (read COVID), I am back in the classroom. Adding one course to my insane workload might seem crazy, but it turns out to be the very best part of my week. I am teaching Public Speaking (something I can manage to keep up with, since so much of the feedback is in the classroom), and truly enjoying the interactions with the students. They are as I remember, equal parts interested and ambivalent about their education. Some are always early to class, others often late. Everyone starts the morning looking at their phones. It is my job to get them to look up.

This is a very active class, with a lot of what I call “pop-ups” to help students fight the pervasive fear of public speaking. During most classes, everyone gets up in front of the class to tell us something. You can learn a lot about your students from popups. They reveal attitudes, interests, and experiences that help me see what they are experiencing in the class and in their lives. This is also a First-Year class focused on orientation to college, so a lot of the prepared speeches focus on things at the university. Last Friday the students presented their first informative speeches and I learned a lot about the student experience at WCSU.

Lesson 1: Our study spaces matter. It is not surprising that many students focused their informative speeches on physical spaces. It is a very open-ended assignment – tell us about something at WCSU- so several students identified locations to describe. Those who did emphasized those places where they can sit down and get some work done. I was happy to hear their tales of using our library, computer Labs, quiet lounges and not so quiet spaces to get through the day. Developing these kinds of spaces has been part of our campus master plan and the facilities team has done a great job of finding spaces in every building for students to land. Our library faculty and staff have completely reimagined the library as a campus hub, with academic supports (tutoring, research, writing center) and a bagel shop. This one assignment tells me that our efforts were worth it.

But it isn’t just that they described the spaces, they described their days. They told tales that were familiar to me because I was a commuter student many years ago. With classes spread out throughout the day, and the inefficiency of going home or traveling between our two campuses, our spaces are essential for managing gaps between classes. Having those spaces near help (library) and faculty (science building in particular) was seen as a big bonus. Having access to computers (all over campus) helps them do assignments that are a pain on their mobile devices (even laptops). And being able to find a quiet space to study or a more social space that might help them meet other students was revealed to be essential.

Lesson 2: Our students are interested in co-curricular activities as part of their undergraduate experience. As a majority commuter campus, we sometimes worry about the students who stop in for class and just go home. Yet, this was not what the students in my class focused on. There are athletes (commuter and residential) who described the demands of their practices and games and how they juggle those demands. As first year students, the athletes faced a big transition from high school sports and college. This transition was described as both intimidating and rewarding. Other students talked about being part of our arts programs and hoped to lure some other students to the performances. This group seemed to have a built-in buddy system with their ensembles, exhibitions, and performances. Both of these groups of students appear to be thriving already because they have well-defined communities at WCSU, filled with both curricular and co-curricular activity.

But our offerings are not suiting all of the students’ needs. For several, who are not in those well-defined cohorts, our clubs are falling short. Every campus likes to brag that students can start any club they’d like, and that is sort of true, but it is not something that a first-year student is inclined to do. Finding something of interest is important for these students so that they do connect with others and with the campus experience outside of the classroom. It was clear that our communication about this is falling short. I must admit I flinched as I heard tales of broken links, and missing details about who is involved or when a club might meet. In addition, the meeting times for these student-run organizations absolutely dissuade our commuter students from participation. They would have to return to campus after 8:00 pm, when they have already been to class, hung around between classes, and perhaps even gone to a part-time job. Even young people don’t really want to do that.

So, we have work to do here. One student suggested we survey students about their interests: I think we might need to do this every year. We also need to carve out some time slots during the day with no classes scheduled so that we can invite more to join in these activities. These are details about our campus that I suspected to be true but hearing it from the students directly, really brought it into focus. We need to help them participate if we want them to thrive.

Lesson 3: Given half a chance, the natural inclination of our students is to be supportive. This is particularly true in a class where everyone has to stand up in front of the room and deliver a speech. We all applaud, of course, that’s just good manners, but the supportiveness comes out in other ways. As we summarized the successes and areas for improvement after our first prepared speeches, students observed growth in their peers already. One noted that everyone’s voice was stronger and more controlled than the first pop-up, another observed that the topics were interesting, and the speakers were prepared. Suggestions for improvement focused on degrees, not absolutes–try to look around the room a bit more, make more eye contact, and try not to pace. These were offered as gentle encouragement. No one felt the need to be negative or harsh in those pointers.

This supportiveness is also expressed in their desire not to offend me as they apologize for lateness or absences or messing up a due date on an assignment. Surely they want my forgiveness (no points off), but I feel that there is also a desire not to appear rude or dismissive of the work we are doing together. In this FY class, I want to encourage that behavior; I want them to feel that I am supportive of them, too. I think carefully about my responses, hoping to support each student while encouraging improvement.

Most of what I have learned so far confirms the data that I regularly review, but teaching gives me a great opportunity to move away from my spreadsheets and see things first-hand again. Being in the classroom brings the trend lines to life and in some cases, makes clear some patterns that those lines don’t fully reveal. I am not sure I will be able to teach another course anytime soon, but I am grateful for this opportunity to learn from our students. The lessons they provide are powerful, indeed.

Evaluation, Quality

The Follow Through

Missy Alexander

Here is a question that I am frequently asked: Why would you want to be in administration? It started with my first truly administrative role, assistant dean, and it persists even now that I have served as provost for nearly six years. As a person whose career began as an adjunct faculty member, then tenure-track to tenured faculty line, it is not lost on me that there are losses when one leaves the classroom. That dreams that led me to higher education were built around love of my discipline and the desire to help students see its value. Teaching is hard work, often frustrating, often rewarding, but it carries with it a clarity of purpose–teach the students in front of you. Living that purpose is exhilarating.

So, why move to administration? Well, for me it was about an ever-widening circle of concerns about how students were experiencing their education. One of my earliest questions was about whether or not students were getting the most out of the totality of their degrees, instead of just focusing on the major. I worried about the connections students were not making between those required humanities or social sciences or science courses and their major. Once I opened that can of worms, my attention moved away from my discipline and toward education as a holistic. Thus, an administrator was born.

What does that holistic perspective mean now that I am a provost? It means I continuously examine data about who we serve, who is thriving, who is not, what students are learning, where our programs are strong and where they need support, what new ideas about teaching are emerging and how to engage faculty with those ideas, and of course, since WCSU is in New England, what to do about enrollment. There’s more. There are questions about equity for everyone (students, faculty, staff). There are questions about processes and organizational structures, and whether they are doing what we want them to do. There are questions about the balance of scholarship, teaching, service, for faculty and appropriate support for professional development for everyone. There is no shortage of things to think about when you are trying to imagine an effective and rewarding whole.

Unsurprisingly, I do a lot of reading about higher education developments and trends. Indeed, this Sunday, as I settled in to review the news and enjoy my morning coffee, I found my attention drawn to a publication from the Chronicle of Higher Education, called The Truth about Student Success. I know, why ruin a perfectly nice Sunday? But I am worried about outcomes and so I downloaded the document and read it through. When I was done, all I could think was, but we’ve done all that already!

Except we haven’t quite. Despite my best efforts to foster an environment where ideas are welcome, strategies for improvement are implemented, and then results are examined, I think I am falling short on the part where we learn from it all. It reminds me of my early days in administration when I realized that higher education is very good at starting (adding) things, but terrible at finishing (subtracting) things. Even worse, we are often missing the part where we examine results and act on them, you know, closing the loop.

Over the last twenty years, higher education as a whole has developed some reasonable habits around the use of assessment to improve curriculum. Everyone has learning outcomes now and assessment plans to trigger reviews of the results. Some plans are better than others, and some programs are more committed to the meaning of those outcomes than others, but overall, folks are trying to learn from their efforts at assessment. At WCSU, I can see the impact of this work on curriculum and to some degree on teaching strategies. This has room to grow, and the sharing of this information is spotty, but it is going on.

We are (I am) less successful at systematic use of the data about the rest of what we do. For example, has the implementation of Degree Works improved academic advising? Has asking about advising practices in annual reports resulted in any changes in strategies at the department level? Are the pre-major pathways (meta-majors) reducing the time to graduation and the accumulation of excess credits? When faculty have participated in teaching institutes, has it changed their teaching strategies? Has it improved outcomes? Has the transition to embedded remediation reduced the number of students stuck in foundational courses? When we see that some courses have very high withdrawal or failure rates, are we acting on that information? There is so much more, but this is the main idea.

As fast as I run, I can’t seem to stay on top of all of this. I have not even managed to implement a good data dashboard to try to keep people in the loop on these things. I hope to complete one this semester, but in the meantime, things are filtering through Deans to Department Chairs to Faculty (maybe), listed in my weekly announcements (sometimes), announced at our University Senate meetings (when time allows), and listed in annual reports (usually). I have to do better.

Without consistent examination of information by the whole community, all of those good things we are doing will just be in pockets (silos). Departments (academic and otherwise) will continue to try new things, but we’ll never see the full impact. We risk not learning from each other and duplicating efforts that would be better if coordinated across areas. We risk abandoning strategies too soon or simply forgetting they are underway. We risk under-investing in things that show signs of working. Most of all, we squander the value of a shared effort to be better, and that is a fundamental waste of talent and resources.

So, as I finished my coffee and that darned report on The Truth about Student Success, I realize that there is no more pressing initiative than establishing good processes for gathering, analyzing, and distributing the information we already have. There is nothing new to do but that. We’re doing all of the other things that everyone else is doing. If we get this part right, we might be able to re-double our efforts on things that are working and stop doing the things that are not. That’s the follow through, folks. We need to learn from what we do.

Examining our processes and making sure that a data dashboard gets done this semester is one more thing on my endless list of duties, of course, and I wonder how I’ll get it done. But I have to because there are no magic bullets to discover; there are only evaluations of what we have already done and plans for next steps. The data dashboard is on me, but I hope that the result is for everyone. I’m hoping with better follow through many more members of our community will work together to improve the whole of the university experience.