Evaluation, Growth Mindset

Staring at the Data

Let me give it to you straight – WCSU is struggling to improve our retention rates. As long as I have been at this university (since 2012), we have been trying to figure out why so many students leave prior to degree completion and why so many leave after the first year. We have tried numerous strategies, each time hanging our hopes on a new effort, hoping to crack the code. Here is the list, since 2012:

We had a limited FY program (not required, so only partial enrollment). Part of that program in my first two years here included cohorting of classes for FY student (they took three together). Though the idea was a good one, it did not have the desired effect. Without full community backing, the impression among the students was that they were being punished in some way. Among the students involved it had a (very small) negative impact on retention. We stopped doing that.

We now have a full FY program (required as part of our general education curriculum). Departments have developed courses that act as extended orientations to the discipline, which seems to have improved our students time to degree completion (that number has been moving up from 48.9% to 53% starting with the 2012 cohort). This, combined with published four-year plans and a general education curriculum that has fewer hidden requirements than the prior version, all seem to be moving us in the right direction on graduation rates.

Unfortunately, the FY program and the related efforts did not improve our retention rates which hovered around 73% for years, with one up-year – we hit 75.5% in 2019, only to crash to 69.9% in 2020. That is definitely a COVID impact. We’ll see how recovery goes.

I’ve been paying attention to our data, and the number one predictor of student retention at WCSU is High School GPA. Students with an 85% or higher high school GPA tend to be retained in the 80% or higher range (often as high as 85%, which is a great number for a regional public comprehensive university). That number drops immediately at 84% high school GPA (down 6-10%) and with a pretty linear correlation with each point dropped. A perfectly nice B/B- high schooler is at a 10% or higher risk of being lost in the first year than the solid B/B+ or higher. This data isn’t just WCSU, it is a national trend. This is data we can act upon.

And we have. We’ve launched a peer mentor program to address students in that B/B- category. We’re hoping just a little support from their friends will help them stay. A group of folks from tutoring, the library faculty, the first-year coordinator, academic advising, orientation, and our alternate admissions team gathered together to make a plan. They agreed on a training plan for the peer mentors and recruited a diverse group of students to do this outreach. They are now analyzing what worked and where to improve for next year, and the team is expanding. We won’t know the impact on retention until the fall, but this work has been exciting, and it feels good to take action and measure results.

With this program launched, there is more data to consider, and it is time we do so. The director of institutional research has been running some queries on a few areas variables that seem to be related to retention. The two reports he just sent my way were startlingly clear. High school GPA is predictive across majors (not just as a general indicator of losing a student): the higher percentage of students in the major with an 85% high school GPA or above, the higher the retention rate. Not surprising, based on what we knew before this point, but here is where it gets interesting. There is a striking exception to this relationship, and it occurs in a major that is organized as a very close cohort. The students who started in that cohort, and were not fully successful in their original major, stayed at WCSU even when they had to change their major.

What can we do in light of this data? Perhaps a deeper dive into the reasons why some degrees are attracting more students with lower High School GPAs and making a more specific series of supports in those majors? Maybe. Or maybe we should think about how to create cohorts in programs where they aren’t naturally occurring. Maybe this is the glue that will help the students succeed. With all of the national data the links student persistence and transparent pathways through degrees, cohorts might help us focus those pathways even further. For a majority commuter campus, cohorts might also help our students connect with peers and feel connected to the community. Cohorts provide exciting possibilities that I’m keen to explore with my colleagues.

The second piece of data was about math. Math success is not a new problem and our math department has tried numerous strategies to try to improve student outcomes. They are working hard, incorporating new technologies and trying new approaches. Nevertheless, we have a persistent problem.

A review of five years of data shows us the following:

  • A shocking number of students are placed into Intermediate Math (credit bearing, pre-general education math): Nearly 50%. This is true for students with above an 85% HS GPA and those below it.
  • Students who earn a D, F, or withdraw from any math class in their first year are retained at a rate between 63-68%. Those who happened to fail Intermediate Math with embedded remediation are retained at a rate below 50%.
  • Students who fail any math class in their first year are 20% more likely to have a high school GPA below 85% than those who pass.
  • Students who pass any math class (including the one with embedded remediation) are retained at about an 82-84% rate.
  • The retention rate for students who take no math class in their first year, drops back down to a 70%.

Given all of this, we might want to re-think our math strategies. We need to dig deeper into the patterns of who is ending up with the D, F, or W in those first courses, to see if we have a few other predictors besides HS GPA. We appear to need better placement strategies than those we are currently using. We might even need a prep-course, prior to placement testing. We probably need a differentiated delivery of our curriculum based on a combination of variables, not just our placement tests. We may need some kind of intersession process to keep students from failing or withdrawing in the first place. We’ve tried variations on each of these in the past, but like the FY program done half-way, it hasn’t worked fully. It is time to focus and develop a plan for better math outcomes. Those outcomes are so directly related to our retention rates that it is imperative that we do so.

Yes, I’m staring at the data and trying to imagine next steps. I know I have a community of colleagues who want more for our students. I know I have a community of colleagues who are eager to find better paths. I’m hoping my colleagues will ask even better questions of our data, and embrace trying new things as a result. I hope those new things are coordinated, measured, and refined as we continue to learn more about our students. I hope all of this, because one thing is certain, staring at the data is not enough.

Engagement, Higher Education, Innovative Pedagogies

From Evolution to Revolution

Over the last year a group of dedicated faculty and staff, all members of our standing assessment committee, have been working to define our university outcomes. This effort is meant to help us see the big picture of the goals of WCSU. University outcomes ask us to consider the connections between majors, general education, minors, electives, co-curricular programs, etc., and their collective impact on our students. When established, they will also provide a path to determining if we are achieving them. This has been a somewhat daunting task, but the committee members have worked hard, and we are nearly there.

When we started this conversation two years ago, I supplied the team with records of all of the established learning outcomes in every major at the university. In addition, I suggested that our general education curriculum and the university’s mission, vision, and values were great places to look for clues as to how to begin. Taking that information together, the things we value as a university started to come into focus. After a few rounds of analysis, forays into drafting language that sprang from those sources, and one focused retreat, the committee emerged with a proposal that is a good reflection of who we are. This backward mapping was very effective as a process, and after incorporating the edits suggested by the broader campus community, this list will stand as an excellent start for defining our shared goals. Bravo.

University outcomes encourage us to look at how things weave together. We are often so busy focusing on our specific tasks in majors or academic support programs or athletics that we don’t look at the whole. Establishing shared goals focuses our attention on how all areas interact. They suggest a path to collaboration that is exciting. Having a standing committee with representatives from faculty, staff, and administration has been a great place to move this conversation forward. This is an important step for WCSU, one that I hope will inspire great conversations at our university.

But I must admit that as we approach the finish line (I hope), I am feeling a little restless. You see, even as that very good work has been taking place, I have been immersed in conversations about the ways that we are failing to reach all of our students. Three different groups (committees, leadership teams, program leaders) have just reached out to me about retention. Each one was trying to discern the reasons why students leave and what next steps we might take to reach them. The answers to their questions are both simple and complex; simple in that I know that the greatest predictor of losing our students is actually their high school GPA, complex in that after accounting for that predictor there are many other subtle factors that lead students out the door. We have taken important steps to address the main predictor; there is more work to do on the rest.

At the same time that those earnest and important questions emerged, I ended up in two wonderful conversations about engaging students in the first year. WCSU already as a first-year course, but these conversations led to an observation about the totality of that first-year experience. In one, we were discussing ways to connect students to our community, in another we were talking about learning experiences that first year students should have. These were exciting conversations and the ideas we explored reflect so well on my colleagues. They are feeling the gaps in the experiences our students are having and trying to innovate and respond.

But I worry, because whenever we get into these discussions about better ways to support the students at a high risk of leaving, or about creating learning experiences that might be more engaging for this generation of learners, we always get bogged down in questions of time, effort, and resources. These are important considerations, but they often derail us before we begin. It is clear we are feeling the need to change. I don’t want those good impulses to get derailed. I am just as exhausted by the juggle as my colleagues are, and I am well aware of our limited resources, so it would be easiest to just pause with the university outcomes and wait a minute. But I find myself feeling that there is no time to waste. I default to my usual perspective–let’s focus on productive changes, while we conspire to do less. Ironically, doing less will take an immense effort. I think the effort might be worth it.

This effort is a lot to ask for, I know. But the questions raised by faculty and staff and students on a daily basis, tell me that this is not a moment to tweak what we do; it is time for real change. So, just as we finish this task of establishing our shared goals, I am thinking of next steps. This time we shouldn’t be backward mapping to what we already do. Instead, we should give full reconsideration to the question of how the learning experiences we design (curricular and co-curricular) interact to engage our students. We should be thinking about how those experiences can help students seize control of their lives and empower them to create the world they want to live in. We should draw on the full range of expertise on our campus to create meaningful connections between ideas, instead of separating them by departments and divisions. We should work to undo the biases built into federal and state regulations that privilege the student who can survive five classes a semester and rebuild our programs to outsmart those rules. We should remember to consult the vast body of research on instructional design throughout. And so much more.

So, why am I restless about those university outcomes? Well, I’m not really. They are a great place to start to focus our efforts and examine our decisions in light of a common set of goals. Those goals are not contradictory to the next project I have just outlined. They will help us evolve as a university, connecting people and ideas that we have struggled to connect in the past. I hope these shared goals will foster conversations at every level of our university, so we can make good decisions about what we are investing in and the experiences we are designing to achieve them. They can propel us forward, even as they reflect the past.

But establishing these outcomes is not enough because we don’t just need to evolve, we need a full-scale revolution. No time to pause… ready, set, go.

Accountability, Change, Higher Education

The Limits of Ad Hoc Committees

In 2012, when I first came to Western Connecticut State University, one of my new colleagues asked me if I was more of a spreadsheet person or an idea person. I happily responded, ideas, and then found my life fully immersed in spreadsheets. The truth is that I am both. I have ideas every single day, but I need to ground them in data, considering the context and cost of an initiative, and planning to measure the impact. As far as I can tell one cannot lead without having some ideas and then vetting them; it is the job.

As I reflect on the work we have done over the last ten years, I see that a lot of good has come out of the back and forth between data and ideas. Our revised general education curriculum seems to have had a positive impact on our graduation rates. I can’t measure causality on this one, but I think the implementation of the First Year Navigation course as part of the gen-ed, coupled with greater transparency in our requirements must be part of the reason why those numbers have improved. I can say for sure that after it was implemented the number of requests for last minute waivers or course substitutions because students had missed a hidden rule in the requirements decreased. There is still room for improvement, but I think the change has overall been a win for student success. This work was completed through the work of our Committee on General Education, working through governance and responding to input. Not everyone was happy, but decisions were made.

Driven by enrollment data and the not so great news about declining high school graduates in New England, I have worked with colleagues to support the development of new graduate degree programs. These programs have been much more informed by regional workforce needs (nursing, addiction counseling, education, healthcare leadership) than our older paths to curriculum development. Indeed, we have had to face facts about enrollment data in some of those older degree programs and we made the decision to close a few. Whether these new programs will be enough to fill the gaps in recruitment at the undergraduate level remains to be seen, but the signs are promising, and I am monitoring outcomes. The curricular changes followed our normal governance processes, with a little extra support in terms of projected demand for new programs and a commitment to looking at our own enrollment trends for faltering programs. Not everyone was happy, but decisions were made.

Close scrutiny of our retention rates has led to the development of our new peer mentor program. After years of asking ourselves who we were losing, and in some cases, just feeling overwhelmed by the many potential factors, a look at the patterns in our retention data gave us clear direction. As the chief evaluator of our outcomes data, I instigated this conversation. Starting first with a standing university committee, I thought the path would be relatively smooth. Unfortunately, it was not, and after being rejected at that level, I moved to an ad hoc committee. The good news is some talented faculty and staff worked together to move this one forward. That bad news is it took three years to implement. There was progress, and we are measuring outcomes, but I was unsuccessful in communicating the urgency of the situation. Still, decisions were made.

As a strong believer in shared governance, I do my very best to move initiatives forward through our normal university processes. At WCSU, we have very positive governance structures in place, structures that I am immensely proud of, because they recognize the collaboration that must take place between faculty, staff, students, and administration. Most of our committees have representation from all of those constituencies on them. This encourages free discussion and the engagement of ideas from all areas. Most of the time this works very well, if somewhat ploddingly. Nevertheless, there are moments when an idea does not really fit in our normal structures and I generally choose to ask for an ad hoc committee to be appointed (sometimes by me, sometimes through Senate leadership), to explore those ideas. Unfortunately, these committees do not seem to reach the point where decisions can be made.

Over the last few years, I have relied on ad hoc committees to try to help me sort through several initiatives or questions. Of those several, only one has managed to truly move an idea forward. It has been a lesson in leadership for me. To be clear, I have utterly failed to impress upon those involved, the importance of the initiatives for the university’s future. It is likely that I was less than clear in the goals as well. I take the blame for the lack of clarity, but I am perplexed as to what to do next.

It isn’t that I expected those committees to return reports that looked exactly like what I thought they would when we started the conversation. If that were the case, it would not be an ad hoc committee exploring a question, but an implementation team. The point of the committees was to look at the starting material/question and then consider a variety of ways to address those questions. In this spirit I met with each group to outline what I thought the pertinent questions about the topic were and then opened things up for the questions and ideas. In each case, the committees asked for clarifications, which I tried to provide, and then they went to work. I stepped back and let the group’s wisdom take hold.

Unfortunately, in nearly every case, people seemed to either be unable to resolve debates within the committee, or they veered off in an unanticipated direction that completely transformed the original charge. Oh well. People did their best. Obviously I was not clear enough. That’s life.

Except some of these committees were formed to address urgent questions, questions that could have an impact on enrollment, or on campus climate, or on the general direction of the university, as we adjust to enrollment challenges and recover from a pandemic. Oh well, just doesn’t cut it. I have failed to lead.

This puts me in a quandary. You see, I don’t just say I embrace shared governance, I mean it. I know the limits of my imagination and I value the dialogue that our processes support. But I have clearly reached the limits of ad hoc committees because they are not leading to action. We need to take action. It is urgent.

I need a new path. I have to figure out how to move urgent things forward, things that have the potential to transform or bolster our campus, so that we might thrive in the face of that demographic cliff we are all staring at. Not all of our next steps will fit neatly into our defined structures so I can’t just default to the usual paths. I still value the input of the many, in all that we do, but I am worried about pace and I am worried about distractions. Decisions have to be made. It is time to regroup.

Higher Education, Innovative Pedagogies

The Ecosystem of Higher Education

According to Brittanica.com, an ecosystem is “the complex of living organisms, their physical environment, and all their relationships in a particular unit of space.” This concept reminds us to look at how things interact with and influence each other rather than focusing on isolated instances of things. A convenient metaphor for interactions within commercial spaces, this term has been applied to contexts outside of the biological, things like healthcare, technology, housing, and so on. Today, I am thinking about higher education through the ecosystem lens.

Several years ago, I was at a legislative breakfast – a semi-annual ritual where our university hosts our local representatives to give them updates on all that is going on at WCSU. These are often pleasant affairs where we get to know each other and talk about our challenges, but also our strengths. At one such meeting, I was asked a pointed question out our university outcomes. The College Navigator tool had just become widely available, and so the focus on comparisons between schools and their retention and graduation rates had come into sharp focus. It was an interesting moment because it was clear that the tool itself did not yet give a context to those numbers, a context that should have demonstrated the differing expectations for those numbers depending on the type of school. I endeavored to explain.

While every college and university strives to support every student to degree completion, there are striking differences in types of campuses, programs, and student needs at each. All of those differences impact our outcomes. It is no surprise that elite campuses, who only admit the most prepared students, have very high retention and graduation rates. If they did not, it would be cause for concern. As admissions standards open to a more inclusive group of students, those numbers change. When you add things like the proportion of residential vs. commuter students, high need vs. middle income students, state appropriations sufficient to support reasonable student to advisor and student to faculty ratios, those numbers change again. Despite the goal of 100% degree completion all of these factors make a big difference in those outcomes.

Now, it is not the case that a campus has no agency. Prioritizing a focus on student success strategies can make a difference. Investing in opportunities for faculty to engage new research about teaching can make a difference. Focusing on fundraising to help support those tremendously important last dollar student grants – grants that can keep a student from stopping out for lack of a few hundred dollars – can make a difference. If a campus can manage to do all of these things, it will likely rise to the top of the list for student outcomes among its peer group (taking into account all of those other variables).

But on that day eight or nine years ago, I was actually discussing something more akin to that ecosystem idea. In Connecticut that ecosystem includes UCONN (the research university that many outside of CT think is a private university), the Connecticut State Universities (regional comprehensives – 2 large, 2 small), Community Colleges throughout the state, Charter Oak College (the public online college), and a significant number of private colleges and universities (Sacred Heart, Fairfield, University of Hartford, Quinnipiac, University of New Haven, Yale, Trinity, and there are more). I was arguing, at that time, that we had different jobs to do, different students to serve, and we needed to be understood from that perspective. Don’t evaluate colleges and universities on one data point, I said, you want us to be different so that all students in the state have options. You need to consider how we work together. That was then.

Now we are all staring at that long-warned demographic cliff, and in a state as small as Connecticut, our crowded ecosystem has reached a critical moment. With so many of us competing for the same students with similar programs and accreditations, we are out of balance, and something will have to give.

But wait there’s more. Although most of us offer some online programs, large online providers from out of state are here, and their impact is already being felt. We have also seen the growth in popularity of Coursera, Google, and Amazon education programs, giving strength to the argument that there are many ways to prepare graduates for the jobs available in the region. That crowded landscape, coupled with the not so quiet questions being asked about the value of our traditional models, is causing panic (and it should), but perhaps it might inspire invention and adaptation instead.

As I think about our overbuilt higher education ecosystem, not just in CT, but in all of the Northeast, the natural impulse is to think about campus differentiation. Should we try to apportion out who will offer what? That question has been asked for years and outside of a few specialties, it is largely impossible to achieve. Universities must have a broad range of programs that interact with each other to create a quality liberal arts degree. We might haggle over a few specific degrees, but overall, there really isn’t room for much differentiation. This is a path that will not yield much change.

But I am wondering if this is an opportunity to adapt to this crowded landscape in a different way. Instead of focusing on the programs we offer, maybe we can specialize in approaches to teaching and learning. I’m thinking about a much more defined campus experience, curricular and co-curricular that are organized around a consistent teaching and learning model. It might be thought through based on those conditions I described at the outset -the types of students we serve and the types of campuses we support. Focusing on teaching and learning, instead of programs might help us find our niches, without losing the breadth of subject matter that we so value. It might allow us to improve our outcomes and be more specific about the kinds of support our campuses need to achieve these ends. It might help us articulate our specific value within this crowded world of higher education.

Taking this approach might help us reposition our questions about how many programs we can support, to how we might build a true educational identity that draws in an appropriate audience of students who have an excellent chance of success. I don’t know if this strategy could work, but what I do know is that something is going to have to change. We are at a tipping point in this ecosystem, so I am thinking it through.

Engagement, Evaluation

Learning from Students

For the last ten years I have been a full-time administrator. In that time, I’ve focused on student learning outcomes and university effectiveness. I’ve obsessed over better pathways through WCSU, hoping to eliminate the unintentional barriers to graduation and policies that are too heavy handed, punishing all students for the poor behavior of the few. I regularly review all the data I can gather about who is succeeding and who is not, trying to address gaps and make things better. Some of those efforts have been effective, improving our overall outcomes; some do not seem to have made a difference. Nevertheless, I forge ahead in that continuous improvement cycle, because it is my job and because I care.

This semester, due to a series of events (read COVID), I am back in the classroom. Adding one course to my insane workload might seem crazy, but it turns out to be the very best part of my week. I am teaching Public Speaking (something I can manage to keep up with, since so much of the feedback is in the classroom), and truly enjoying the interactions with the students. They are as I remember, equal parts interested and ambivalent about their education. Some are always early to class, others often late. Everyone starts the morning looking at their phones. It is my job to get them to look up.

This is a very active class, with a lot of what I call “pop-ups” to help students fight the pervasive fear of public speaking. During most classes, everyone gets up in front of the class to tell us something. You can learn a lot about your students from popups. They reveal attitudes, interests, and experiences that help me see what they are experiencing in the class and in their lives. This is also a First-Year class focused on orientation to college, so a lot of the prepared speeches focus on things at the university. Last Friday the students presented their first informative speeches and I learned a lot about the student experience at WCSU.

Lesson 1: Our study spaces matter. It is not surprising that many students focused their informative speeches on physical spaces. It is a very open-ended assignment – tell us about something at WCSU- so several students identified locations to describe. Those who did emphasized those places where they can sit down and get some work done. I was happy to hear their tales of using our library, computer Labs, quiet lounges and not so quiet spaces to get through the day. Developing these kinds of spaces has been part of our campus master plan and the facilities team has done a great job of finding spaces in every building for students to land. Our library faculty and staff have completely reimagined the library as a campus hub, with academic supports (tutoring, research, writing center) and a bagel shop. This one assignment tells me that our efforts were worth it.

But it isn’t just that they described the spaces, they described their days. They told tales that were familiar to me because I was a commuter student many years ago. With classes spread out throughout the day, and the inefficiency of going home or traveling between our two campuses, our spaces are essential for managing gaps between classes. Having those spaces near help (library) and faculty (science building in particular) was seen as a big bonus. Having access to computers (all over campus) helps them do assignments that are a pain on their mobile devices (even laptops). And being able to find a quiet space to study or a more social space that might help them meet other students was revealed to be essential.

Lesson 2: Our students are interested in co-curricular activities as part of their undergraduate experience. As a majority commuter campus, we sometimes worry about the students who stop in for class and just go home. Yet, this was not what the students in my class focused on. There are athletes (commuter and residential) who described the demands of their practices and games and how they juggle those demands. As first year students, the athletes faced a big transition from high school sports and college. This transition was described as both intimidating and rewarding. Other students talked about being part of our arts programs and hoped to lure some other students to the performances. This group seemed to have a built-in buddy system with their ensembles, exhibitions, and performances. Both of these groups of students appear to be thriving already because they have well-defined communities at WCSU, filled with both curricular and co-curricular activity.

But our offerings are not suiting all of the students’ needs. For several, who are not in those well-defined cohorts, our clubs are falling short. Every campus likes to brag that students can start any club they’d like, and that is sort of true, but it is not something that a first-year student is inclined to do. Finding something of interest is important for these students so that they do connect with others and with the campus experience outside of the classroom. It was clear that our communication about this is falling short. I must admit I flinched as I heard tales of broken links, and missing details about who is involved or when a club might meet. In addition, the meeting times for these student-run organizations absolutely dissuade our commuter students from participation. They would have to return to campus after 8:00 pm, when they have already been to class, hung around between classes, and perhaps even gone to a part-time job. Even young people don’t really want to do that.

So, we have work to do here. One student suggested we survey students about their interests: I think we might need to do this every year. We also need to carve out some time slots during the day with no classes scheduled so that we can invite more to join in these activities. These are details about our campus that I suspected to be true but hearing it from the students directly, really brought it into focus. We need to help them participate if we want them to thrive.

Lesson 3: Given half a chance, the natural inclination of our students is to be supportive. This is particularly true in a class where everyone has to stand up in front of the room and deliver a speech. We all applaud, of course, that’s just good manners, but the supportiveness comes out in other ways. As we summarized the successes and areas for improvement after our first prepared speeches, students observed growth in their peers already. One noted that everyone’s voice was stronger and more controlled than the first pop-up, another observed that the topics were interesting, and the speakers were prepared. Suggestions for improvement focused on degrees, not absolutes–try to look around the room a bit more, make more eye contact, and try not to pace. These were offered as gentle encouragement. No one felt the need to be negative or harsh in those pointers.

This supportiveness is also expressed in their desire not to offend me as they apologize for lateness or absences or messing up a due date on an assignment. Surely they want my forgiveness (no points off), but I feel that there is also a desire not to appear rude or dismissive of the work we are doing together. In this FY class, I want to encourage that behavior; I want them to feel that I am supportive of them, too. I think carefully about my responses, hoping to support each student while encouraging improvement.

Most of what I have learned so far confirms the data that I regularly review, but teaching gives me a great opportunity to move away from my spreadsheets and see things first-hand again. Being in the classroom brings the trend lines to life and in some cases, makes clear some patterns that those lines don’t fully reveal. I am not sure I will be able to teach another course anytime soon, but I am grateful for this opportunity to learn from our students. The lessons they provide are powerful, indeed.