Welcome to Episode 148 of the Think UDL podcast: Students Tell Us What Works in Statistics with Jen McNally and Laura Callis. Jen McNally is a Professor and the Math Area Coordinator in the Department of Science and Mathematics at Curry College. Laura Callis is an Associate Professor in the Department of Science and Mathematics. Jen is the PI, or Principal Investigator, and Laura is the Co-PI of the DISCUS-IS (Discourse to Improve Student Conceptual Understanding of Statistics in Inclusive Settings) project which we will be exploring today. Jen and Laura have found that students are often the best collaborators to understand what UDL interventions and applications work best in their particular settings. Their project is particularly focussed in statistics and today’s conversation will explore how to help students learn and even have a little fun in statistics classes using UDL. From flipping pennies to dolphin tricks, this episode has it all. You can also find the resources associated with today’s conversation on the thinkUDL.org website.
Resources
Reach Jen McNally on LinkedIn and Laura Callis on LinkedIn
If you are interested to learn more, go to www.inclusivestatistics.com where Laura and Jen will be posting updates and people can sign up to learn more.
Transcript
46:59
SUMMARY KEYWORDS
Universal Design for Learning, learner variability, statistics class, student engagement, inclusive settings, simulation-based inference, neurodivergent students, instructional practices, student support, Curry College, PAL program, student interviews, cold calling, authentic learning, professional development.
SPEAKERS
Jen McNally, Laura Callis, Lillian Nave
Lillian Nave 00:02
Welcome to Think UDL, the universal design for learning podcast where we hear from the people who are designing and implementing strategies with learner variability in mind. I’m your host, Lillian nave, and I’m interested in not just what you’re teaching, learning, guiding and facilitating, but how you design and implement it, and why it even matters.
Lillian Nave 00:39
Welcome to Episode 148 of the think UDL podcast, students tell us what works in statistics with Jen McNally and Laura Kallis. Jen McNally is a professor and the math area coordinator in the department of science and mathematics at Curry College. Laura Kallis is an associate professor in the Department of science and mathematics, and Jen is the PI or principal investigator. Laura is the co PI of the discus is discourse to improve student conceptual understanding of statistics in inclusive settings, project which we will be exploring today, Jen and Laura have found that students are often the best collaborators to understand what UDL interventions and applications work best in their particular setting. Their project is particularly focused in statistics, and today’s conversation will explore how to help students learn and even have a little fun in statistics classes using UDL from flipping pennies to dolphin tricks, this episode has it all. You can also find the resources associated with today’s conversation on the think udl.org website and thank you for listening to the think UDL podcast. I’d like to welcome both of my guests. Thank you for joining me, Jen and Laura on the podcast today.
Jen McNally 02:13
It’s great opportunity to be here. Thanks for having us.
Laura Callis 02:15
Yeah, thank you so much for having us.
Lillian Nave 02:17
Thank you. So I want to jump right into my first question before we get into the great research, anytime we get a chance to do UDL research in higher ed, I’m so excited, but first I will start with Laura to ask my my first question, which is, what makes you a different kind of learner?
Laura Callis 02:36
Yeah, I love this question, because I think so many professors kind of reflect on themselves as learners when they’re preparing to work with their work with their students, but they might actually be a different kind of learner than the students who are in front of them. So I really like learning about things that are systems or connections or have relationships. So things like math and physics really resonated with me well. But if I had to memorize a body of knowledge like something in history or biology, where there’s just lots and lots of facts. That’s a little bit harder for me. And I’ve done like I love self help books, so I’ve done all of the different, you know, Myers, Briggs and everything like that. So for me, I realized that the kind of strengths that I have kind of fit in with the system, the existing educational system that exists in K 12 settings. So I’m pretty independent. I learned by reading and writing. It’s mostly what you do in school. I’m not super great at audio processing, but I can always turn that into notes, and that makes it more understandable for me. And you’re expected to take notes, right? And even though I’m not very organized with my stuff, like, if you could see my desk, there’s tissues and masks and papers everywhere. I learned how to be organized with my time, and so I could kind of hide the things that you know the school system would expect of me. So I learned different hacks for remembering my homework. I would staple all my lab reports into my notebooks so that I couldn’t lose them, and my keys are always tied to my body somehow. So even though I have some traits that, you know, you see kids sort of get yelled at a lot in schools, I was able to hide them. And then my existing strengths kind of fit in with with the existing school system and so but that doesn’t mean that the existing system is like, right, right. And I think a lot of professors sometimes feel like, well, I got through, so they should get right. So one example of that is, you know, I am very motivated by pleasing others or by doing the right thing. Like, I’m very conscientious, so I don’t need to know why I have to learn something. I just the teacher said to so I go and do it so. But when I learned about ADHD, their brain doesn’t get fired up in the same way mine would. They need to be interested in something and it doesn’t like it’s not necessarily a better thing that I’m willing to learn about whatever people tell me to maybe it would be better if I was actually intrinsic. Motivated in the things that I’m learning. So I think that’s true in a lot of different things that you see in the existing kind of school system is that it’s built for a particular kind of learner. I happen to be that kind of learner, or I could pass as that kind of learner, but that might not necessarily be the students who are sitting in front
Lillian Nave 05:18
of me. Gotcha. Wow, I appreciate how you matched your strengths with what you saw. And you know what? What is our system, our K 12 system in the United States especially, and realize that that’s not everybody, and of course, it’s not who we get in higher ed either all the time.
Laura Callis 05:36
So there’s nothing intrinsically great about that system. You know that we have to hold on to it too, right?
Lillian Nave 05:42
Yeah, it’s just is. It’s not better or worse. It’s just that’s the system. So what are we going to do? Oh, actually, we have lots of ideas about what we can do. But first question again, Jen, what makes you a different kind of learner?
Jen McNally 05:58
So I’ll be honest, I haven’t really deeply reflected on my own learner profile in the same way that Laura has, but it allowed me to do some self searching. So I’m so glad you asked the question. So a lot like Laura, I did fit in with the current educational system where I grew up. In fact, I fit in so well that I was in the gifted program in my little town in my city, and I really did, like Laura, thrive on pleasing the people around me, doing all my work, following all the directions, acing every test. And I had worked out like the system of how to do school so well, and I really excelled in all the things that I did when it came time to actually choose a major in college, I didn’t know what to do because I was just seeking a challenge. I didn’t really care about what the content itself was. And I’ll admit now, like, I really have, have real strong perfectionist tendencies that I’m really trying hard to overcome because of this. Like, you know, I fit so well in the system, and I could, you know, get that motivation from being a good doobie in the system as an adult when I’m trying to learn new skills, like, true learning is hard for me to actually achieve. Like, I like to have authentic challenges. Like, sure, I have an 800 day streak on Duolingo, but I couldn’t even ask the waitress to ask for separate checks when we were in a restaurant in Puerto Rico. Like, it was functionally irrelevant that I had an 800 day streak. So it reminds me that students can really perform well, even if they haven’t learned anything at all. And that’s, you know, a little bit sobering to educators, because we think that we can test for knowledge, and when we see that these results come out that they actually mean something. So it’s really only when I was an emerging adult that I started to recognize that I was really nave to the privileges that I had, you know, having that good match between the educational system and my learning style, and so my way of learning and performing was just highly valued in the system that I was in. So now I really question a lot about like, how many people were overlooked, how many exceptionally bright people there are out there that didn’t have that same match that I did, and didn’t have the opportunities like laid out for them in the same way that I did. So as an instructor, UDL is really this way for me to have a framework, like a perspective, or a way of, like, interacting with my students and their performance. Like, is the student really doing well because they understand the concepts well? Or, you know, are they just able to replicate the procedures that I’ve done and it looks like they’ve learned something? And then, like on the other side of that too, is somebody not doing well because of the way I’ve asked a question or a way I framed the activity or the assessment that I’m doing. So I think that if I had teachers that were, you know, more steeped and really lived UDL, maybe they would have seen through my roots exactly.
Lillian Nave 09:00
You know, I had a similar kind of moment and understanding how I was very much a people pleaser. And, like, I really liked getting that positive reinforcement, like to get the a right to be a high achiever. And then when you’re out of the school system and you’re like, I did. I did laundry all by myself. Where’s my a, you know, where’s Harry card for making dinner for the family for 28 months straight? You know, there’s no, there’s no a for that, right? And so I realized how conditioned, right I was about what was I doing that for? And I really appreciate that you kind of brought up that strategic learning versus deep learning, where you have the 800 day streak, but you couldn’t functionally use it, right? So what are we really testing for, and are we really getting at what we want our students to know? Are they really understanding it, or do we. Have a lot of somewhat faulty methods, or just not quite directing or meeting with the students where they are to find out what they really are learning. So again, it just makes me question how we always taught, and that’s why I love Universal Design for Learning, because there’s so many more ways than the way where I want an A for doing my laundry, and I just don’t get it. Nobody says that I’ve done a good job. All right. So you guys are at Curry College, which I have heard quite a lot about, and it’s really a fantastic, great curriculum and a great college that I want to, want lots of people to hear about. So one of the reasons why I wanted to have you on but you’re also doing amazing things there. So I’ll ask Laura this part about can you tell me about Curry College? Who are your students? Who do you serve there? What’s your context?
Laura Callis 10:50
Yeah, so we are on a beautiful, 130 acre campus. We’re just seven miles from downtown Boston. It’s a small private college and primarily serving undergraduate so we do have some graduate programs. Mostly we have majors in liberal arts and pre professional programs. We have a large nursing program both the undergraduate and graduate level. We also have education and business and Criminal Justice at the graduate and undergraduate level. We are a less elective institution in that we welcome lots and lots and lots and lots of folks. So our acceptance rate is 88% so what that means is we have a great deal of variability in our classrooms. So we’re a small college, so we don’t have like, you know, honors tier and this tier and that tier, everybody’s kind of together. And so you might have students who have taken, for instance, in statistics, AP statistics, but didn’t take the exam, mixing with students who haven’t taken a math class since their sophomore year. And that’s kind of the exciting, interesting challenge of working here. We’re unapologetically student centered. So we have small classes. I don’t think any of them are larger than 30, right? We spend a lot of time with our students. We develop strong relationships with our students. We have individualized student support in terms of other services too. And we’re a teaching first institution, and we’re always continuously improving our pedagogy. So we have a faculty center that many of us rotate throughout being the coordinator of. So there’s a history of people working on their practice and giving back to the community in terms of what they’ve learned about in their own practice, across not just within our own institution, but really across all different disciplines. So it’s kind of everything is kind of seeped in, in teaching and learning here,
Lillian Nave 12:28
awesome. I love the 88% acceptance rate. You take a wide variety of students, and that is exactly where we need Universal Design for Learning is when we’re dealing with a variety of students, which, of course, is in every single class. But I think you’re seeing, yeah, a huge strength, the huge variety of strengths that your students bring. So in addition to that, you also have something called the PAL program, P, a, l, what is that?
Jen McNally 12:57
Jen, sure, the PAL program is our program for the Advancement of Learning, and it was established in 1970 here at Curry College by one of our professors, Dr Gertrude Webb. So in 1970 it was even before Americans with Disabilities Act, and our focus on like really making sure that we had access to education and post secondary education for students who are neurodivergent. So pal serves students with diagnosed learning disabilities, executive functioning challenges, andor ADHD, so we know that a lot of those are correlated to each other, right? And students in the program, they attend our regular heterogeneous undergraduate courses with students who are in the program and students who are not as well as having credit bearing one on one small group or individualized support with pal faculty. So in those meetings and that ongoing coaching, pal faculty build students metacognitive skills, their strategies to mitigate executive functioning challenges, and they promote students ability to really self advocate for themselves, and, you know, perform well in a variety of educational and social settings. It is a fee based program. About 20% of our students are enrolled in the Pell program and at Curry College, but because we have this reputation for really supporting a neurodiverse student population, we attract a whole lot more than 20% of our students as being neurodivergent. So we estimate that at Curry students who self identify as having a learning or attention difference is somewhere around 40% so having this program really contributes to our campus wide perspective of the neurodiversity perspective compared to deficit model. So people here talk about their cognitive differences, really self aware of them, and there isn’t as much stigma around neuro divergence as you might find at other places.
Lillian Nave 14:58
I love it, and what a great community. D, I’ve just, I’ve heard of really wonderful things out of Curry College for a while. So I’m really glad to see specifics too. And one of those specifics is something called discusses. Discusses, it’s D, I S, C, U, S, dash, i s, and Laura, I need you to explain that for me. Yeah.
Laura Callis 15:21
So discusses is a National Science Foundation supported program, and it stands for discourse to improve students conceptual understanding of statistics and inclusive settings. And it grew out of our department’s initiative to improve our introductory statistics class. This is a course that almost all students take at Curry to satisfy their general education requirements, but it’s also a prerequisite for many majors like psychology and science. And our project is looking to identify and explicate instructional practices that are supportive of what we call students in the margins. So students the margins are students who have traditionally been excluded from their research literature. So either they’re they’re hidden, or they’re just, you know, not present. It might be include students who are neurodivergent. In our case, it’s going to include a lot of those students because of our campus, but it also includes students who are from marginalized backgrounds, or students who have gaps in their mathematics backgrounds, or students who just aren’t really represented in the quantitative fields. We are placing an emphasis on discourse, on building on students ideas, and we want to be able to provide greater context through examples of student thinking and key statistical ideas. So we’ve got this kind of two pronged approach of like, what kind of instructional practices are helping our students be successful? Because we have lots of data that shows that there, they are successful at the same rate as neurotypical students and white students. But then also, we’re trying to we know that when instructors are doing ambitious curricula, so inquiry based curricula or problem based learning, these kinds of ambitious strategies, they have to know a lot about student thinking in order to do it effectively. And so we’re looking to identify how students think about different statistical ideas and how they talk about them, particularly students who have been ignored in the research literature and maybe their conceptions might not be following in line with everyone else. So we believe that by giving instructors these resources to understand student thinking, they’re going to be ahead of the game and be able to enact a high quality curriculum in statistics at a higher level. The way that we’ve done this is we have been going into classrooms of instructors who were successfully Closing the Achievement a gap between all these different groups of students to see what they’re doing. We do video simulated recall interviews. So we sit down with a professor and we re watch their class together, and we talked about, like, oh, you know, I saw you do this thing, you know. Can you tell me a little bit about why you decided to do that? We’ve been interviewing students too. It’s really important for us to get the student perspective on what they think is working. So we’ve been talking with them about the instructional practices that they found supportive in their statistics classes. And then we also have a survey and some other kind of measures as well. And then we’ve been interviewing students talking about different statistical investigations to figure out, okay, what do they think a p value means, or how would they go about designing a study? So we’ve got lots lots of data from those different approaches. Great.
Lillian Nave 18:19
So, yeah, that’s a fantastic so you’ve got a lot, and I want to know exactly how you’re doing it, or how you’re applying it. So the next one is, so which course did you just decide to apply these UDL interventions in why you chose one particular course for this project? And that I’ll start with Jen on that
Jen McNally 18:38
one. Sure. So the course that we focus on is math 1150, it’s our statistics one. It’s an algebra based introductory statistics course. And like Laura said, it satisfies our general education quantitative requirement, and it’s a prerequisite course for a lot of our quantitative fields that have their own specialized quantitative courses in the upper levels. So it serves probably about 90% of all of our students. So it’s a high impact course. It’s one that’s going to have, like, a lot of traction, based on, you know, how many students we can reach with the project. And for most students, they take it in their first year. So we know how important that first year is, especially for students who may be trying to decide whether college is the right place for them. We want them to have really positive experiences and ones that are tailored to, you know what works well for them? And it would be a surprise for many students that a math course did work well for them. So we have all of these key factors that we were thinking about in selecting statistics to be the course that we focused on. And we know that, you know, math can be a roadblock for students, both to STEM disciplines, but to getting a college degree in general. And so we wanted to make sure that we had a wide impact by choosing a course that could otherwise derail students, but also one that was important for students to become educated citizens. So. So by our metrics, our course was doing okay. We were covering our content that was in most stats one classes, students were passing at a rate that was pretty good, especially since we did track like DFW rates in first year courses. We wanted to make sure that our course itself wasn’t a roadblock, and it wasn’t. But we kept hearing from our colleagues in other disciplines that they wanted us to cover more content. They wanted us to do more in the course, and they were concerned about students not retaining what we taught and then being able to transfer it on to other, the more upper level, quantitative, discipline specific courses that they were taking. So it was a lot of pressure on us, right? And we were trying to figure out what was best for our students. And we tried a lot of different things. We tried co requisite support. We tried flipped classroom. We did auto graded homework that linked right back to the resources and the textbook, to try to provide some just in time support when we weren’t in the classroom with students. But then we really started looking at a whole different approach to teaching statistics. We looked at simulation based inference, we’ll call it SBI, at times, and because we thought that by adopting SBI, we might be able to cover more material, what we found out was that it had a lot of other benefits as well. So I’ll just run through like, the basic premise of what a simulation based inference approach is. So what it does is it really de emphasizes by hand calculations and in the reliance of statistics courses on theory based approaches, and instead, it tries to make statistical inquiry a little bit more intuitive or accessible to students. So for instance, there’s an example that explores whether two dolphins, they’re named Doris and buzz, whether they can communicate with each other. So the researcher, Dr Bastian, trained the dolphins to be able to push the button on the left if they saw a steady light in their tank, and the button on the right if they saw it flashing. So they separated out the two dolphins so that only Doris could see the light and only buzz could push the buttons. And when they conducted 16 trials like this, Buzz pushed the correct button 15 times.
Lillian Nave 22:08
Wow, this is some smart
Jen McNally 22:12
or lucky, right? That’s what
Lillian Nave 22:14
we have to figure out. Could be lucky, yeah, yeah.
Jen McNally 22:17
So the question really is, was he just guessing, or is there evidence that in some way, Doris was communicating they’re smart enough, and they were able to figure out which button to push when buzz was on his own with those buttons. So intuitively, you’re probably thinking like 15 out of 16. That’s like 93% he would have aced that test, right? It’s pretty remarkable, but we use this as an example to introduce the statistical inquiry process like how do we actually decide whether we have enough evidence to show that our results are significant or not? And so in class, we hand out 16 coins to all the students and a sticky note, and we unpack a simulation. So we’re building this model with students through what they already know about flipping coins through what they already sense about this being a remarkable event, to try to say, Okay, what do statisticians do when they’re looking at these data, and what do they compare it to, in order to decide it’s remarkable enough to say that it’s statistically discernible, statistically significant. So all the students flip their coins on the sticky note. They write the number of heads that they get, and we as a class decide that heads will indicate that buzz guessed correctly, right. And flipping all these coins is one trial in which Buzz is just guessing right. There’s nothing happening, right? And so we take all of the little sticky notes and put them on a communal poster at the front of the room and build what we call a chance model. This is what it would have looked like. This is the variability that we can expect. So most of their sticky notes have sevens, eights, nines. We have a couple of fours and twelves in there, like sometimes, just by chance, we get very few heads or very many heads. But I’ve never had a student come up with a sticky note that had a 15 on it, just like Buzz did, right? And so it really helps our students formalize their intuition and build a more robust concept about what statistical inquiry is. And so we find that there is evidence that buzz was not just guessing, because we didn’t get results like that when we built the chance model. And so it’s that type of method that we use that’s simulation based. We eventually transition over to computer based simulations after students have some experience with what we’re actually doing with these numbers here, so that they can build on that intuition that they already come to the table with and formalize it a little bit
Laura Callis 24:40
more. Yeah. So there’s a lot that goes into this curriculum, but we kind of, you can sort of see how it’s much more concrete and much more based off students intuition, versus having to memorize, like a bunch of formulas and color in the right part of a graph, and, you know, remember all of these different conditions when you can use this formula or the. Formula. And then, in addition, it’s also, we’re really focused on looking at one data set throughout the entire class period and working through the statistical investigation cycle. Instead of, here’s a rule, here’s 10 problems to practice that rule on. So so in that way, we’re developing statistical thinking and not just a collection of facts and procedures. So it was working for our students when we looked at the pilot versus just an active learning classroom. Student, Jen was running the pilot, I was running the active learning classroom with a traditional curriculum, we saw that we were able to close the gap between white students and non white students, and between students with learning differences and students without. In Jen’s class and with the new curriculum, and in my class, I was actually doing a disservice to my students. So they were, some of them were unlearning. So that kind of, you know, small data set. It was only maybe 50 students, but that kind of was like, we we need to stop. We need to move to this new, new
Lillian Nave 26:02
way. Yeah, we can’t do this anymore, right? Yeah, the risk
Laura Callis 26:05
is too high. Like, even if it was a fluke, the risk is too high. And so we adopted the entire program for all the different classes. We kind of trained some of our professors on it, and we’ve been doing it since. We have a paper coming out soon that shows again, when we use this curriculum, there’s no difference between neurotypical and neuro divergent students. We opened up the definition so that students could help self identify a little bit. So so far, things have been going pretty good when we look at the big picture.
Lillian Nave 26:34
Wow, that this sounds like so much fun, too, and I am not like a statistics fan at all so, but this sounds like a lot more fun, and I would actually probably like statistics if this is the kind of class that I was in. Wow. Okay, so now we know why. What a great time to do it, and what a great class. So then, what did you learn? So you’ve put this into place. You had your study, your student interviews, and I am interested, what did the students have to say, and what did you find out about what supportive practices they indicated were helpful, and how they were actually doing that learning?
Jen McNally 27:17
Jen, yeah, so we learned that there’s no one UDL aligned way to teach simulation based inference in introductory statistics. There’s no magic template, sorry, there’s no rule book on what this looks like that we could copy across course sections. But what we did find was that we had a pretty diverse group of instructors. We gave them a pretty short timeline, and what we found was that pretty much anybody could do this well if they had the intention and the motivation to do it. So we have math educators teaching statistics classes, along with folks whose formal training was in business or pure math. We have part time adjuncts who we trained up. We have full time faculty. We have folks who have never thought about educational methods or anything like that, they’ve entered teaching after retiring from a career in business or law. So we found that it’s going to look different in each one of those cases, but the common denominator amongst it all is that people use UDL practices when they have a curriculum like this, and it is impacting their students. We’re finding no differences between the students in section to section, regardless of the instructor or the instructor type. So really, despite all those differences in the backgrounds, we’re all pretty generally equivalent, which is good news for statistics, especially, I think because we have across the country, statistics is one of those courses that’s taught by pretty much anyone. So you have people in disciplines teaching statistics, you have a lot of adjuncts teaching statistics. So this shows that you can impact UDL and you can have UDL practices even if you don’t have a career faculty member. But what UDL looked like was really different in every classroom, and with the way students talked about the classrooms that they were in and what they identified as impactful,
Laura Callis 29:09
yeah, so I can give a few examples. So for instance, I’m sure, like all good math instructors address language and symbols and kind of decoding the notation, and if you look on the UDL rubric, that’s like checkpoint 2.1 and 2.2 but in our classes, the students were experiencing this in a lot of different ways, and they would tell us the stories about how this was done when we asked them what was helpful in your class, and they talked about vocabulary or notation. So some of them said that the instructor didn’t just do it once. Every single time there was some kind of symbol on the board, they unpacked it. And they unpacked it in a general way, like, you know, this squiggle line, mu, you we say it mu, and it means the mean of a population. And in this case, we’re talking about the mean of whatever the particular thing they were studying, like the mean amount of money people are spending on Christmas presents. And. So they did that every single time. It wasn’t just like a one off, and then we expect students to be able to generalize from a one off. But other things that students did, or the instructors did, was they created, like a packet for a student created glossary, and they would tell the students, okay, take out your glossary, and the students would write down when they were introducing a new idea. And the students found it really valuable, because they didn’t have to go searching through their notes. They could their notes. They could just pull it out. And it wasn’t like looking on Google, where Google is going to give you a high level definition that was going to be specific to the example that they had done in class, and so it’s going to help recall all of their memories about that symbol. And then another professor had a kind of like a word wall, almost like an elementary school, they had a giant post it, and would keep adding the different symbols as they went. And so when students were working in class, they could look up at the poster and not being you not have to use all of their working memory to decode the symbols. Think about the context of the problem. Think about mathematically what they’re going to do. How am I going to do it with the technology they could look up and have that reference there. So even though these instructors were doing it all in different ways, they were all intentionally every class session, giving their students different kinds of resources. I could talk about other ways that we saw like, this is one idea, but it actually was, and, you know, lived in different ways too. If you’d like to hear more
Lillian Nave 31:17
I’m interested in, yeah, give us one more. One more you can give us.
Laura Callis 31:24
Okay, sounds good. So another one that, you know interests me a lot is this idea of optimizing relevance and value authenticity. So if you’re on the rubric, that’s checkpoint 7.2 and a lot of students were talking about this when we asked them, like, what was helpful in your class, they talked about the different data sets in the context that we used. So they said they were interesting, and we’re like, Okay, well, what is, what does that mean to you? Because we’re old, we don’t know what do you mean? Yeah, it’s about iPods.
Lillian Nave 31:57
You’re so old.
Laura Callis 32:00
So what they they actually differ. What students think interesting is. So that was important to kind of realize. So for some of the students, it was about learning about the world. So there’s some examples about animal behavior, like the dolphins, there’s ones about dogs, there’s ones about bugs even, and then bug behavior. And so they felt like they were learning about real things. One of the investigations was about Vietnam War draft, and the student talked about how that was interesting because it was about about the world around her, but for other students, it meant something much closer to home, so something that they would find useful or within their experience. So one student was talking about the price of used cars, because that was something that he actually had relevant experience or could imagine himself doing. But then for others, it was more about curiosity. So it wasn’t necessarily even the topic so much as it the unexpectedness of it. And so, for instance, one of the investigations was on different types of music. And if listening to different types of music leads to youth delinquency, and because they didn’t know like they could see it going both ways, they wanted to find out the answer. So it didn’t even have to necessarily relate to their own personal lives. It was just this kind of curiosity that was piqued so and then another student was just amazed that the professor knew enough about the class and the students that they pivoted their lesson so they didn’t even use something on a textbook. They the students are always talking about parking on our campus. Yes, really hard for commuters. Yeah. So the professor was engaged enough with the students to realize this was an issue, and said, Okay, do you guys want to do our investigation on parking? And they went outside and they collected real data about parking spaces and how many were used up, and she used that to teach the content, and the student couldn’t stop talking about it because it made her feel seen and listened to. And that statistics was was something that you would actually use in your life, and not necessarily just in your profession, but use in your your day to day life. So can mean many, many different things to different students.
Lillian Nave 34:05
Yeah, that’s so great. The Of course, I think that’s really good data is that not everybody’s interested in the same things, because that tells us learner variability, right? And it did make me think of something that I love watching one of it’s a series called Love on the spectrum, which is about dating for young people who are on the autism spectrum. And one of the best quotations from that show was when one of the young women said on one of these dates, said, That’s very interesting, but I’m just not interested. And I thought, that’s perfect. That’s very interesting, but I’m just not interested. You know, that’s a really nice thing. That’s really great. I’m sure it’s very interesting, but I’m not interested. And I thought, Oh, that is so succinct. It’s like that. Anyway, one of my favorite things that I’ve ever heard is. Yeah, that’s very interesting. I’m just not interested. So that’s, you know, every person in the class at some point, you know, this is interesting. I’m just not very interested in this one right now.
Laura Callis 35:12
Yeah, and the students would tell us, they said, Why do you really like this one? But exactly,
Lillian Nave 35:17
yeah, that’s great, and it does make me think about that authenticity and the relevance. What, why should we be learning this in the first place? And it makes me go back to Jen’s like Duolingo, and can I learn about how to ask about splitting the check? That would be really helpful. And I don’t need to know what color your bicycle is, right? So what surprised you? Then, from the data, what did you learn from that? And Jen, we can start with you.
Jen McNally 35:50
Yeah, I was just really surprised at the range of ways that we could see UDL actually happening in classrooms. So I knew what my teaching looked like, and I was really familiar with Laura’s but to think about walking into somebody else’s classroom, maybe somebody trained in pure math, or one of our adjunct instructors who came from business and just started teaching statistics for fun, you know, I was really curious, like, could they really do a good job of listening to what their students needs? Were removing as many barriers as they could. And what I’m really convinced by is that, yeah, UDL really can be employed by anyone. It takes, you know, I think it took the curriculum to spur us to do the thing well, but I think that we even without that, if we had all come together as a community and just shared the impact that our attention to students needs had would have done a lot to make sure that we weren’t really meeting students needs and we were aligning our practices in a way that we were designing the learning environment around the needs of our students.
Laura Callis 36:58
Yeah, I think, um, Jen, just to build off of what you were saying, because I found this in other research too, is that, like, a car is not a boat, right? Unless you’re talking about the duck boat, I guess you’re in Boston, but so like, you can have all the great intentions for your students in the world, and you can know about UDL, but if you don’t have a curriculum that allows you to do that to the fullest extent of your abilities, you’re kind of limited, like you’re not going to develop brand new problem sets that are suddenly going to be able to help you live UDL in the way that you want to, but at the same time, like a car or a boat, they don’t drive themselves right. So you need the right tool for the right train, but you need a driver, and if you like, we’ve we’ve seen because we did, we have taught stats asynchronously with this curriculum, and it does about as fine as the traditional curriculum does, but it doesn’t do what we can do in the classroom. We they students really need that interaction between their peers, and they need the interaction with the professor in real time, not just like, you know, asynchronously. So it’s almost like you’re it’s not just about the instructor, and it’s not just about the curriculum. If you’re trying to make change, you really have to kind of put the two pieces together. And I think I kind of underestimated what a curriculum could do. And I know other people out there underestimate the impact of instructors. They just think the solution is to change, change all the textbooks, and then, and then everything will be solved. So kind of the two things together. I keep talking about things that
Lillian Nave 38:28
surprised me, yeah, and the context like, it’s so context specific. So I’m that’s why I wanted to start with. The context there at Curry College is that you have all these small classes, you have a wide variety of students. And I attended your talk at the AAC youth conference, and I remember one of those things that surprised me was that students said something I did not expect students would say about cold calling. Yeah, what happened there? Either one of you can answer that one, yeah.
Laura Callis 39:05
So they This surprised me too, because it started when I asked students, they said that the professor really, really cared about them. I was like, Okay, we keep hearing this care idea come up. Like, what does that mean? Because obviously I feel like I care about you, but how do you know that, and what does that mean to you? Because I’m not sure it’s necessarily coming through. And they said, Well, he calls on us, even if we don’t raise our hand. And I was like, but I did not think that would be but I kept coming up like other students would say, well, he cold calls, and they call it cold calling. I wouldn’t call it cold calling because the professors will do little tricks, like they will give you can pass one, or you can ask a friend, or they’ll call in an entire table. So they’ll say, oh, Megan’s table. But Megan doesn’t have to be the one to answer. It could be Tiffany. We could answer for her. So. There’s all these little subtle outs, and they also will okay, if so everybody’s passing them, that means I have to go back and revisit the idea as an instructor. So it was a kind of safe cold calling. It wasn’t like in law school, where they’re like, Okay, you, you, you, you don’t know the answer. You’re out. You know, it was very safe, safe place. But it’s the students that it made them pay more attention, and they felt like the professor cared about what they were thinking. So it was really kind of surprising. And I think Jen has said this probably better than I can say it, but like, don’t talk yourself out of a strategy, kind of like, you know, cold calling seems horrible. Like, I’m not going to do that to my students, but you don’t know, you don’t know what their experience of it is going to be.
Lillian Nave 40:44
It sounds so much to us about intent, because it’s really like that law school. I mean, I can think of movies, you know, where it’s, where it’s as if the professor is trying to embarrass somebody, right? And that’s, and you can tell, and it’s, you know, you’re, you’re trying to see, oh, that person’s not paying attention. I’m going to make them pay for that. But no, this was very different. It sounds like it was. I really want to know, as the professor, I really want to know if, if my students are getting it and and I, and I want to give them the chance to work it through, to think it through, to have some help, you know, because this is the time that we have, it’s really important, let’s get a chance that we can to go to go through it. So it’s that ethos of care seems to be very much part of Curry College. And what was part of this project, it seems to me, least that’s how I read it.
Jen McNally 41:38
Yeah, definitely. And I think that building on that idea of care, it’s really about empathy that the instructors are showing for their students, not just the I care about your emotional state right now, but the true empathy about like, I understand where you are as a learner right now, and I understand where I want you to be, and I want to help you become more authoritative over the content. Like, I want you to become an authority along with me and be able to do the things that I can do. And so it’s not a checklist. You wouldn’t see a UDL checklist that says, Use cold calling with your students, but you would have this perspective or this this attitude towards students that I know where I want you to be. I know where you are right now, and I’m going to provide an environment where we’re safe to do the things that we’re we need to do in order to become more expert in this.
Laura Callis 42:27
Yeah, even some of the professors would talk about that, that because there are social anxiety students with social anxiety in our classrooms, and we do a lot of group work, and we ask people to speak up and whole class discussion. And our instructor even talked about that that, you know, once you leave Carter, you’re going to be asked to do these things. So let’s jump together now. And he explicitly talked with a student about this while it’s still safe, and we’ll figure out, like, a way to do it maybe, maybe I, you know, come by and tell you, I’m going to call on you, so I give you a little bit of a warning or something. So, but let’s find out a way that we can do this while we’re still in this, in, you know, safer environment. So it’s kind of this empathy, of like, I know this is hard, but I believe in you and we’re going to find a way to do it anyway.
Lillian Nave 43:13
That’s so great. So there were a couple surprises in there. That’s great. Yeah, yeah. Well, what’s what’s next? So what’s next in the research? What’s next with this class? Where do you see it going from here, from this great start, yeah.
Laura Callis 43:28
So we want to share our results. So next up, we are pulling together and editing all of our video footage to develop a repository of professional development resources so that instructors can go on and learn more about different practices that might support them, and see examples of students talking about their thinking so they can better prepare for for their classes. But we’re also looking to expand our community. So as we reach out, we’ve been running some webinars. So if people want to reach out to us and join one of our webinars that would be great. We’re looking toward maybe developing mentorship so that we can help professors in other colleges learn how to do the kind of research on their own practice and their own students, so that they can learn what it is works in their context, for their students, and kind of how we were able to do that, see if they are able to adopt some of those methods too. What else? Jen, what else is next? We’ve got big plans.
Jen McNally 44:27
So, yeah, we just love expanding our community. So if you’ve heard something that resonates with you, or that sparks a curiosity, we hope that you’ll reach out and we’ll definitely stay in touch and be able to share the resources and also the approaches that we’re using that are really student centered and focused on really over sampling, so that we get the students have traditionally been left behind out of the from the research to be amplified in our work.
Lillian Nave 44:54
Great. Yeah, and we will link on the resources tab for this episode. So to the podcast, how to reach out to both of you, yeah, and some more resources about what you have been doing so folks can learn a lot more. I know you already have a good bit that you shared with the folks that were at the presentation at the conference. So there’s a lot to to get from this that people can emulate and maybe use it in their own statistics classes and see how it might work in their context as well. So thank you so much, both of you for joining me today. It was a real pleasure to talk to you about what you’re doing for your students, and I am very excited that we get to share this today, so thank you both Jen and Laura.
45:43
Thanks. Thank you.
Lillian Nave 45:47
TU148
Mon, Sep 29, 2025 7:52PM • 46:59
SUMMARY KEYWORDS
Universal Design for Learning, learner variability, statistics class, student engagement, inclusive settings, simulation-based inference, neurodivergent students, instructional practices, student support, Curry College, PAL program, student interviews, cold calling, authentic learning, professional development.
SPEAKERS
Jen McNally, Laura Callis, Lillian Nave
Lillian Nave 00:02
Welcome to think UDL, the universal design for learning podcast where we hear from the people who are designing and implementing strategies with learner variability in mind. I’m your host, Lillian nave, and I’m interested in not just what you’re teaching, learning, guiding and facilitating, but how you design and implement it, and why it even matters.
Lillian Nave 00:39
Welcome to Episode 148 of the think UDL podcast, students tell us what works in statistics with Jen McNally and Laura Kallis. Jen McNally is a professor and the math area coordinator in the department of science and mathematics at Curry College. Laura Kallis is an associate professor in the Department of science and mathematics, and Jen is the PI or principal investigator. Laura is the co PI of the discus is discourse to improve student conceptual understanding of statistics in inclusive settings, project which we will be exploring today, Jen and Laura have found that students are often the best collaborators to understand what UDL interventions and applications work best in their particular setting. Their project is particularly focused in statistics, and today’s conversation will explore how to help students learn and even have a little fun in statistics classes using UDL from flipping pennies to dolphin tricks, this episode has it all. You can also find the resources associated with today’s conversation on the think udl.org website and thank you for listening to the think UDL podcast. I’d like to welcome both of my guests. Thank you for joining me, Jen and Laura on the podcast today.
Jen McNally 02:13
It’s great opportunity to be here. Thanks for having us.
Laura Callis 02:15
Yeah, thank you so much for having us.
Lillian Nave 02:17
Thank you. So I want to jump right into my first question before we get into the great research, anytime we get a chance to do UDL research in higher ed, I’m so excited, but first I will start with Laura to ask my my first question, which is, what makes you a different kind of learner?
Laura Callis 02:36
Yeah, I love this question, because I think so many professors kind of reflect on themselves as learners when they’re preparing to work with their work with their students, but they might actually be a different kind of learner than the students who are in front of them. So I really like learning about things that are systems or connections or have relationships. So things like math and physics really resonated with me well. But if I had to memorize a body of knowledge like something in history or biology, where there’s just lots and lots of facts. That’s a little bit harder for me. And I’ve done like I love self help books, so I’ve done all of the different, you know, Myers, Briggs and everything like that. So for me, I realized that the kind of strengths that I have kind of fit in with the system, the existing educational system that exists in K 12 settings. So I’m pretty independent. I learned by reading and writing. It’s mostly what you do in school. I’m not super great at audio processing, but I can always turn that into notes, and that makes it more understandable for me. And you’re expected to take notes, right? And even though I’m not very organized with my stuff, like, if you could see my desk, there’s tissues and masks and papers everywhere. I learned how to be organized with my time, and so I could kind of hide the things that you know the school system would expect of me. So I learned different hacks for remembering my homework. I would staple all my lab reports into my notebooks so that I couldn’t lose them, and my keys are always tied to my body somehow. So even though I have some traits that, you know, you see kids sort of get yelled at a lot in schools, I was able to hide them. And then my existing strengths kind of fit in with with the existing school system and so but that doesn’t mean that the existing system is like, right, right. And I think a lot of professors sometimes feel like, well, I got through, so they should get right. So one example of that is, you know, I am very motivated by pleasing others or by doing the right thing. Like, I’m very conscientious, so I don’t need to know why I have to learn something. I just the teacher said to so I go and do it so. But when I learned about ADHD, their brain doesn’t get fired up in the same way mine would. They need to be interested in something and it doesn’t like it’s not necessarily a better thing that I’m willing to learn about whatever people tell me to maybe it would be better if I was actually intrinsic. Motivated in the things that I’m learning. So I think that’s true in a lot of different things that you see in the existing kind of school system is that it’s built for a particular kind of learner. I happen to be that kind of learner, or I could pass as that kind of learner, but that might not necessarily be the students who are sitting in front
Lillian Nave 05:18
of me. Gotcha. Wow, I appreciate how you matched your strengths with what you saw. And you know what? What is our system, our K 12 system in the United States especially, and realize that that’s not everybody, and of course, it’s not who we get in higher ed either all the time.
Laura Callis 05:36
So there’s nothing intrinsically great about that system. You know that we have to hold on to it too, right?
Lillian Nave 05:42
Yeah, it’s just is. It’s not better or worse. It’s just that’s the system. So what are we going to do? Oh, actually, we have lots of ideas about what we can do. But first question again, Jen, what makes you a different kind of learner?
Jen McNally 05:58
So I’ll be honest, I haven’t really deeply reflected on my own learner profile in the same way that Laura has, but it allowed me to do some self searching. So I’m so glad you asked the question. So a lot like Laura, I did fit in with the current educational system where I grew up. In fact, I fit in so well that I was in the gifted program in my little town in my city, and I really did, like Laura, thrive on pleasing the people around me, doing all my work, following all the directions, acing every test. And I had worked out like the system of how to do school so well, and I really excelled in all the things that I did when it came time to actually choose a major in college, I didn’t know what to do because I was just seeking a challenge. I didn’t really care about what the content itself was. And I’ll admit now, like, I really have, have real strong perfectionist tendencies that I’m really trying hard to overcome because of this. Like, you know, I fit so well in the system, and I could, you know, get that motivation from being a good doobie in the system as an adult when I’m trying to learn new skills, like, true learning is hard for me to actually achieve. Like, I like to have authentic challenges. Like, sure, I have an 800 day streak on Duolingo, but I couldn’t even ask the waitress to ask for separate checks when we were in a restaurant in Puerto Rico. Like, it was functionally irrelevant that I had an 800 day streak. So it reminds me that students can really perform well, even if they haven’t learned anything at all. And that’s, you know, a little bit sobering to educators, because we think that we can test for knowledge, and when we see that these results come out that they actually mean something. So it’s really only when I was an emerging adult that I started to recognize that I was really nave to the privileges that I had, you know, having that good match between the educational system and my learning style, and so my way of learning and performing was just highly valued in the system that I was in. So now I really question a lot about like, how many people were overlooked, how many exceptionally bright people there are out there that didn’t have that same match that I did, and didn’t have the opportunities like laid out for them in the same way that I did. So as an instructor, UDL is really this way for me to have a framework, like a perspective, or a way of, like, interacting with my students and their performance. Like, is the student really doing well because they understand the concepts well? Or, you know, are they just able to replicate the procedures that I’ve done and it looks like they’ve learned something? And then, like on the other side of that too, is somebody not doing well because of the way I’ve asked a question or a way I framed the activity or the assessment that I’m doing. So I think that if I had teachers that were, you know, more steeped and really lived UDL, maybe they would have seen through my roots exactly.
Lillian Nave 09:00
You know, I had a similar kind of moment and understanding how I was very much a people pleaser. And, like, I really liked getting that positive reinforcement, like to get the a right to be a high achiever. And then when you’re out of the school system and you’re like, I did. I did laundry all by myself. Where’s my a, you know, where’s Harry card for making dinner for the family for 28 months straight? You know, there’s no, there’s no a for that, right? And so I realized how conditioned, right I was about what was I doing that for? And I really appreciate that you kind of brought up that strategic learning versus deep learning, where you have the 800 day streak, but you couldn’t functionally use it, right? So what are we really testing for, and are we really getting at what we want our students to know? Are they really understanding it, or do we. Have a lot of somewhat faulty methods, or just not quite directing or meeting with the students where they are to find out what they really are learning. So again, it just makes me question how we always taught, and that’s why I love Universal Design for Learning, because there’s so many more ways than the way where I want an A for doing my laundry, and I just don’t get it. Nobody says that I’ve done a good job. All right. So you guys are at Curry College, which I have heard quite a lot about, and it’s really a fantastic, great curriculum and a great college that I want to, want lots of people to hear about. So one of the reasons why I wanted to have you on but you’re also doing amazing things there. So I’ll ask Laura this part about can you tell me about Curry College? Who are your students? Who do you serve there? What’s your context?
Laura Callis 10:50
Yeah, so we are on a beautiful, 130 acre campus. We’re just seven miles from downtown Boston. It’s a small private college and primarily serving undergraduate so we do have some graduate programs. Mostly we have majors in liberal arts and pre professional programs. We have a large nursing program both the undergraduate and graduate level. We also have education and business and Criminal Justice at the graduate and undergraduate level. We are a less elective institution in that we welcome lots and lots and lots and lots of folks. So our acceptance rate is 88% so what that means is we have a great deal of variability in our classrooms. So we’re a small college, so we don’t have like, you know, honors tier and this tier and that tier, everybody’s kind of together. And so you might have students who have taken, for instance, in statistics, AP statistics, but didn’t take the exam, mixing with students who haven’t taken a math class since their sophomore year. And that’s kind of the exciting, interesting challenge of working here. We’re unapologetically student centered. So we have small classes. I don’t think any of them are larger than 30, right? We spend a lot of time with our students. We develop strong relationships with our students. We have individualized student support in terms of other services too. And we’re a teaching first institution, and we’re always continuously improving our pedagogy. So we have a faculty center that many of us rotate throughout being the coordinator of. So there’s a history of people working on their practice and giving back to the community in terms of what they’ve learned about in their own practice, across not just within our own institution, but really across all different disciplines. So it’s kind of everything is kind of seeped in, in teaching and learning here,
Lillian Nave 12:28
awesome. I love the 88% acceptance rate. You take a wide variety of students, and that is exactly where we need Universal Design for Learning is when we’re dealing with a variety of students, which, of course, is in every single class. But I think you’re seeing, yeah, a huge strength, the huge variety of strengths that your students bring. So in addition to that, you also have something called the PAL program, P, a, l, what is that?
Jen McNally 12:57
Jen, sure, the PAL program is our program for the Advancement of Learning, and it was established in 1970 here at Curry College by one of our professors, Dr Gertrude Webb. So in 1970 it was even before Americans with Disabilities Act, and our focus on like really making sure that we had access to education and post secondary education for students who are neurodivergent. So pal serves students with diagnosed learning disabilities, executive functioning challenges, andor ADHD, so we know that a lot of those are correlated to each other, right? And students in the program, they attend our regular heterogeneous undergraduate courses with students who are in the program and students who are not as well as having credit bearing one on one small group or individualized support with pal faculty. So in those meetings and that ongoing coaching, pal faculty build students metacognitive skills, their strategies to mitigate executive functioning challenges, and they promote students ability to really self advocate for themselves, and, you know, perform well in a variety of educational and social settings. It is a fee based program. About 20% of our students are enrolled in the Pell program and at Curry College, but because we have this reputation for really supporting a neurodiverse student population, we attract a whole lot more than 20% of our students as being neurodivergent. So we estimate that at Curry students who self identify as having a learning or attention difference is somewhere around 40% so having this program really contributes to our campus wide perspective of the neurodiversity perspective compared to deficit model. So people here talk about their cognitive differences, really self aware of them, and there isn’t as much stigma around neuro divergence as you might find at other places.
Lillian Nave 14:58
I love it, and what a great community. D, I’ve just, I’ve heard of really wonderful things out of Curry College for a while. So I’m really glad to see specifics too. And one of those specifics is something called discusses. Discusses, it’s D, I S, C, U, S, dash, i s, and Laura, I need you to explain that for me. Yeah.
Laura Callis 15:21
So discusses is a National Science Foundation supported program, and it stands for discourse to improve students conceptual understanding of statistics and inclusive settings. And it grew out of our department’s initiative to improve our introductory statistics class. This is a course that almost all students take at Curry to satisfy their general education requirements, but it’s also a prerequisite for many majors like psychology and science. And our project is looking to identify and explicate instructional practices that are supportive of what we call students in the margins. So students the margins are students who have traditionally been excluded from their research literature. So either they’re they’re hidden, or they’re just, you know, not present. It might be include students who are neurodivergent. In our case, it’s going to include a lot of those students because of our campus, but it also includes students who are from marginalized backgrounds, or students who have gaps in their mathematics backgrounds, or students who just aren’t really represented in the quantitative fields. We are placing an emphasis on discourse, on building on students ideas, and we want to be able to provide greater context through examples of student thinking and key statistical ideas. So we’ve got this kind of two pronged approach of like, what kind of instructional practices are helping our students be successful? Because we have lots of data that shows that there, they are successful at the same rate as neurotypical students and white students. But then also, we’re trying to we know that when instructors are doing ambitious curricula, so inquiry based curricula or problem based learning, these kinds of ambitious strategies, they have to know a lot about student thinking in order to do it effectively. And so we’re looking to identify how students think about different statistical ideas and how they talk about them, particularly students who have been ignored in the research literature and maybe their conceptions might not be following in line with everyone else. So we believe that by giving instructors these resources to understand student thinking, they’re going to be ahead of the game and be able to enact a high quality curriculum in statistics at a higher level. The way that we’ve done this is we have been going into classrooms of instructors who were successfully Closing the Achievement a gap between all these different groups of students to see what they’re doing. We do video simulated recall interviews. So we sit down with a professor and we re watch their class together, and we talked about, like, oh, you know, I saw you do this thing, you know. Can you tell me a little bit about why you decided to do that? We’ve been interviewing students too. It’s really important for us to get the student perspective on what they think is working. So we’ve been talking with them about the instructional practices that they found supportive in their statistics classes. And then we also have a survey and some other kind of measures as well. And then we’ve been interviewing students talking about different statistical investigations to figure out, okay, what do they think a p value means, or how would they go about designing a study? So we’ve got lots lots of data from those different approaches. Great.
Lillian Nave 18:19
So, yeah, that’s a fantastic so you’ve got a lot, and I want to know exactly how you’re doing it, or how you’re applying it. So the next one is, so which course did you just decide to apply these UDL interventions in why you chose one particular course for this project? And that I’ll start with Jen on that
Jen McNally 18:38
one. Sure. So the course that we focus on is math 1150, it’s our statistics one. It’s an algebra based introductory statistics course. And like Laura said, it satisfies our general education quantitative requirement, and it’s a prerequisite course for a lot of our quantitative fields that have their own specialized quantitative courses in the upper levels. So it serves probably about 90% of all of our students. So it’s a high impact course. It’s one that’s going to have, like, a lot of traction, based on, you know, how many students we can reach with the project. And for most students, they take it in their first year. So we know how important that first year is, especially for students who may be trying to decide whether college is the right place for them. We want them to have really positive experiences and ones that are tailored to, you know what works well for them? And it would be a surprise for many students that a math course did work well for them. So we have all of these key factors that we were thinking about in selecting statistics to be the course that we focused on. And we know that, you know, math can be a roadblock for students, both to STEM disciplines, but to getting a college degree in general. And so we wanted to make sure that we had a wide impact by choosing a course that could otherwise derail students, but also one that was important for students to become educated citizens. So. So by our metrics, our course was doing okay. We were covering our content that was in most stats one classes, students were passing at a rate that was pretty good, especially since we did track like DFW rates in first year courses. We wanted to make sure that our course itself wasn’t a roadblock, and it wasn’t. But we kept hearing from our colleagues in other disciplines that they wanted us to cover more content. They wanted us to do more in the course, and they were concerned about students not retaining what we taught and then being able to transfer it on to other, the more upper level, quantitative, discipline specific courses that they were taking. So it was a lot of pressure on us, right? And we were trying to figure out what was best for our students. And we tried a lot of different things. We tried co requisite support. We tried flipped classroom. We did auto graded homework that linked right back to the resources and the textbook, to try to provide some just in time support when we weren’t in the classroom with students. But then we really started looking at a whole different approach to teaching statistics. We looked at simulation based inference, we’ll call it SBI, at times, and because we thought that by adopting SBI, we might be able to cover more material, what we found out was that it had a lot of other benefits as well. So I’ll just run through like, the basic premise of what a simulation based inference approach is. So what it does is it really de emphasizes by hand calculations and in the reliance of statistics courses on theory based approaches, and instead, it tries to make statistical inquiry a little bit more intuitive or accessible to students. So for instance, there’s an example that explores whether two dolphins, they’re named Doris and buzz, whether they can communicate with each other. So the researcher, Dr Bastian, trained the dolphins to be able to push the button on the left if they saw a steady light in their tank, and the button on the right if they saw it flashing. So they separated out the two dolphins so that only Doris could see the light and only buzz could push the buttons. And when they conducted 16 trials like this, Buzz pushed the correct button 15 times.
Lillian Nave 22:08
Wow, this is some smart
Jen McNally 22:12
or lucky, right? That’s what
Lillian Nave 22:14
we have to figure out. Could be lucky, yeah, yeah.
Jen McNally 22:17
So the question really is, was he just guessing, or is there evidence that in some way, Doris was communicating they’re smart enough, and they were able to figure out which button to push when buzz was on his own with those buttons. So intuitively, you’re probably thinking like 15 out of 16. That’s like 93% he would have aced that test, right? It’s pretty remarkable, but we use this as an example to introduce the statistical inquiry process like how do we actually decide whether we have enough evidence to show that our results are significant or not? And so in class, we hand out 16 coins to all the students and a sticky note, and we unpack a simulation. So we’re building this model with students through what they already know about flipping coins through what they already sense about this being a remarkable event, to try to say, Okay, what do statisticians do when they’re looking at these data, and what do they compare it to, in order to decide it’s remarkable enough to say that it’s statistically discernible, statistically significant. So all the students flip their coins on the sticky note. They write the number of heads that they get, and we as a class decide that heads will indicate that buzz guessed correctly, right. And flipping all these coins is one trial in which Buzz is just guessing right. There’s nothing happening, right? And so we take all of the little sticky notes and put them on a communal poster at the front of the room and build what we call a chance model. This is what it would have looked like. This is the variability that we can expect. So most of their sticky notes have sevens, eights, nines. We have a couple of fours and twelves in there, like sometimes, just by chance, we get very few heads or very many heads. But I’ve never had a student come up with a sticky note that had a 15 on it, just like Buzz did, right? And so it really helps our students formalize their intuition and build a more robust concept about what statistical inquiry is. And so we find that there is evidence that buzz was not just guessing, because we didn’t get results like that when we built the chance model. And so it’s that type of method that we use that’s simulation based. We eventually transition over to computer based simulations after students have some experience with what we’re actually doing with these numbers here, so that they can build on that intuition that they already come to the table with and formalize it a little bit
Laura Callis 24:40
more. Yeah. So there’s a lot that goes into this curriculum, but we kind of, you can sort of see how it’s much more concrete and much more based off students intuition, versus having to memorize, like a bunch of formulas and color in the right part of a graph, and, you know, remember all of these different conditions when you can use this formula or the. Formula. And then, in addition, it’s also, we’re really focused on looking at one data set throughout the entire class period and working through the statistical investigation cycle. Instead of, here’s a rule, here’s 10 problems to practice that rule on. So so in that way, we’re developing statistical thinking and not just a collection of facts and procedures. So it was working for our students when we looked at the pilot versus just an active learning classroom. Student, Jen was running the pilot, I was running the active learning classroom with a traditional curriculum, we saw that we were able to close the gap between white students and non white students, and between students with learning differences and students without. In Jen’s class and with the new curriculum, and in my class, I was actually doing a disservice to my students. So they were, some of them were unlearning. So that kind of, you know, small data set. It was only maybe 50 students, but that kind of was like, we we need to stop. We need to move to this new, new
Lillian Nave 26:02
way. Yeah, we can’t do this anymore, right? Yeah, the risk
Laura Callis 26:05
is too high. Like, even if it was a fluke, the risk is too high. And so we adopted the entire program for all the different classes. We kind of trained some of our professors on it, and we’ve been doing it since. We have a paper coming out soon that shows again, when we use this curriculum, there’s no difference between neurotypical and neuro divergent students. We opened up the definition so that students could help self identify a little bit. So so far, things have been going pretty good when we look at the big picture.
Lillian Nave 26:34
Wow, that this sounds like so much fun, too, and I am not like a statistics fan at all so, but this sounds like a lot more fun, and I would actually probably like statistics if this is the kind of class that I was in. Wow. Okay, so now we know why. What a great time to do it, and what a great class. So then, what did you learn? So you’ve put this into place. You had your study, your student interviews, and I am interested, what did the students have to say, and what did you find out about what supportive practices they indicated were helpful, and how they were actually doing that learning?
Jen McNally 27:17
Jen, yeah, so we learned that there’s no one UDL aligned way to teach simulation based inference in introductory statistics. There’s no magic template, sorry, there’s no rule book on what this looks like that we could copy across course sections. But what we did find was that we had a pretty diverse group of instructors. We gave them a pretty short timeline, and what we found was that pretty much anybody could do this well if they had the intention and the motivation to do it. So we have math educators teaching statistics classes, along with folks whose formal training was in business or pure math. We have part time adjuncts who we trained up. We have full time faculty. We have folks who have never thought about educational methods or anything like that, they’ve entered teaching after retiring from a career in business or law. So we found that it’s going to look different in each one of those cases, but the common denominator amongst it all is that people use UDL practices when they have a curriculum like this, and it is impacting their students. We’re finding no differences between the students in section to section, regardless of the instructor or the instructor type. So really, despite all those differences in the backgrounds, we’re all pretty generally equivalent, which is good news for statistics, especially, I think because we have across the country, statistics is one of those courses that’s taught by pretty much anyone. So you have people in disciplines teaching statistics, you have a lot of adjuncts teaching statistics. So this shows that you can impact UDL and you can have UDL practices even if you don’t have a career faculty member. But what UDL looked like was really different in every classroom, and with the way students talked about the classrooms that they were in and what they identified as impactful,
Laura Callis 29:09
yeah, so I can give a few examples. So for instance, I’m sure, like all good math instructors address language and symbols and kind of decoding the notation, and if you look on the UDL rubric, that’s like checkpoint 2.1 and 2.2 but in our classes, the students were experiencing this in a lot of different ways, and they would tell us the stories about how this was done when we asked them what was helpful in your class, and they talked about vocabulary or notation. So some of them said that the instructor didn’t just do it once. Every single time there was some kind of symbol on the board, they unpacked it. And they unpacked it in a general way, like, you know, this squiggle line, mu, you we say it mu, and it means the mean of a population. And in this case, we’re talking about the mean of whatever the particular thing they were studying, like the mean amount of money people are spending on Christmas presents. And. So they did that every single time. It wasn’t just like a one off, and then we expect students to be able to generalize from a one off. But other things that students did, or the instructors did, was they created, like a packet for a student created glossary, and they would tell the students, okay, take out your glossary, and the students would write down when they were introducing a new idea. And the students found it really valuable, because they didn’t have to go searching through their notes. They could their notes. They could just pull it out. And it wasn’t like looking on Google, where Google is going to give you a high level definition that was going to be specific to the example that they had done in class, and so it’s going to help recall all of their memories about that symbol. And then another professor had a kind of like a word wall, almost like an elementary school, they had a giant post it, and would keep adding the different symbols as they went. And so when students were working in class, they could look up at the poster and not being you not have to use all of their working memory to decode the symbols. Think about the context of the problem. Think about mathematically what they’re going to do. How am I going to do it with the technology they could look up and have that reference there. So even though these instructors were doing it all in different ways, they were all intentionally every class session, giving their students different kinds of resources. I could talk about other ways that we saw like, this is one idea, but it actually was, and, you know, lived in different ways too. If you’d like to hear more
Lillian Nave 31:17
I’m interested in, yeah, give us one more. One more you can give us.
Laura Callis 31:24
Okay, sounds good. So another one that, you know interests me a lot is this idea of optimizing relevance and value authenticity. So if you’re on the rubric, that’s checkpoint 7.2 and a lot of students were talking about this when we asked them, like, what was helpful in your class, they talked about the different data sets in the context that we used. So they said they were interesting, and we’re like, Okay, well, what is, what does that mean to you? Because we’re old, we don’t know what do you mean? Yeah, it’s about iPods.
Lillian Nave 31:57
You’re so old.
Laura Callis 32:00
So what they they actually differ. What students think interesting is. So that was important to kind of realize. So for some of the students, it was about learning about the world. So there’s some examples about animal behavior, like the dolphins, there’s ones about dogs, there’s ones about bugs even, and then bug behavior. And so they felt like they were learning about real things. One of the investigations was about Vietnam War draft, and the student talked about how that was interesting because it was about about the world around her, but for other students, it meant something much closer to home, so something that they would find useful or within their experience. So one student was talking about the price of used cars, because that was something that he actually had relevant experience or could imagine himself doing. But then for others, it was more about curiosity. So it wasn’t necessarily even the topic so much as it the unexpectedness of it. And so, for instance, one of the investigations was on different types of music. And if listening to different types of music leads to youth delinquency, and because they didn’t know like they could see it going both ways, they wanted to find out the answer. So it didn’t even have to necessarily relate to their own personal lives. It was just this kind of curiosity that was piqued so and then another student was just amazed that the professor knew enough about the class and the students that they pivoted their lesson so they didn’t even use something on a textbook. They the students are always talking about parking on our campus. Yes, really hard for commuters. Yeah. So the professor was engaged enough with the students to realize this was an issue, and said, Okay, do you guys want to do our investigation on parking? And they went outside and they collected real data about parking spaces and how many were used up, and she used that to teach the content, and the student couldn’t stop talking about it because it made her feel seen and listened to. And that statistics was was something that you would actually use in your life, and not necessarily just in your profession, but use in your your day to day life. So can mean many, many different things to different students.
Lillian Nave 34:05
Yeah, that’s so great. The Of course, I think that’s really good data is that not everybody’s interested in the same things, because that tells us learner variability, right? And it did make me think of something that I love watching one of it’s a series called Love on the spectrum, which is about dating for young people who are on the autism spectrum. And one of the best quotations from that show was when one of the young women said on one of these dates, said, That’s very interesting, but I’m just not interested. And I thought, that’s perfect. That’s very interesting, but I’m just not interested. You know, that’s a really nice thing. That’s really great. I’m sure it’s very interesting, but I’m not interested. And I thought, Oh, that is so succinct. It’s like that. Anyway, one of my favorite things that I’ve ever heard is. Yeah, that’s very interesting. I’m just not interested. So that’s, you know, every person in the class at some point, you know, this is interesting. I’m just not very interested in this one right now.
Laura Callis 35:12
Yeah, and the students would tell us, they said, Why do you really like this one? But exactly,
Lillian Nave 35:17
yeah, that’s great, and it does make me think about that authenticity and the relevance. What, why should we be learning this in the first place? And it makes me go back to Jen’s like Duolingo, and can I learn about how to ask about splitting the check? That would be really helpful. And I don’t need to know what color your bicycle is, right? So what surprised you? Then, from the data, what did you learn from that? And Jen, we can start with you.
Jen McNally 35:50
Yeah, I was just really surprised at the range of ways that we could see UDL actually happening in classrooms. So I knew what my teaching looked like, and I was really familiar with Laura’s but to think about walking into somebody else’s classroom, maybe somebody trained in pure math, or one of our adjunct instructors who came from business and just started teaching statistics for fun, you know, I was really curious, like, could they really do a good job of listening to what their students needs? Were removing as many barriers as they could. And what I’m really convinced by is that, yeah, UDL really can be employed by anyone. It takes, you know, I think it took the curriculum to spur us to do the thing well, but I think that we even without that, if we had all come together as a community and just shared the impact that our attention to students needs had would have done a lot to make sure that we weren’t really meeting students needs and we were aligning our practices in a way that we were designing the learning environment around the needs of our students.
Laura Callis 36:58
Yeah, I think, um, Jen, just to build off of what you were saying, because I found this in other research too, is that, like, a car is not a boat, right? Unless you’re talking about the duck boat, I guess you’re in Boston, but so like, you can have all the great intentions for your students in the world, and you can know about UDL, but if you don’t have a curriculum that allows you to do that to the fullest extent of your abilities, you’re kind of limited, like you’re not going to develop brand new problem sets that are suddenly going to be able to help you live UDL in the way that you want to, but at the same time, like a car or a boat, they don’t drive themselves right. So you need the right tool for the right train, but you need a driver, and if you like, we’ve we’ve seen because we did, we have taught stats asynchronously with this curriculum, and it does about as fine as the traditional curriculum does, but it doesn’t do what we can do in the classroom. We they students really need that interaction between their peers, and they need the interaction with the professor in real time, not just like, you know, asynchronously. So it’s almost like you’re it’s not just about the instructor, and it’s not just about the curriculum. If you’re trying to make change, you really have to kind of put the two pieces together. And I think I kind of underestimated what a curriculum could do. And I know other people out there underestimate the impact of instructors. They just think the solution is to change, change all the textbooks, and then, and then everything will be solved. So kind of the two things together. I keep talking about things that
Lillian Nave 38:28
surprised me, yeah, and the context like, it’s so context specific. So I’m that’s why I wanted to start with. The context there at Curry College is that you have all these small classes, you have a wide variety of students. And I attended your talk at the AAC youth conference, and I remember one of those things that surprised me was that students said something I did not expect students would say about cold calling. Yeah, what happened there? Either one of you can answer that one, yeah.
Laura Callis 39:05
So they This surprised me too, because it started when I asked students, they said that the professor really, really cared about them. I was like, Okay, we keep hearing this care idea come up. Like, what does that mean? Because obviously I feel like I care about you, but how do you know that, and what does that mean to you? Because I’m not sure it’s necessarily coming through. And they said, Well, he calls on us, even if we don’t raise our hand. And I was like, but I did not think that would be but I kept coming up like other students would say, well, he cold calls, and they call it cold calling. I wouldn’t call it cold calling because the professors will do little tricks, like they will give you can pass one, or you can ask a friend, or they’ll call in an entire table. So they’ll say, oh, Megan’s table. But Megan doesn’t have to be the one to answer. It could be Tiffany. We could answer for her. So. There’s all these little subtle outs, and they also will okay, if so everybody’s passing them, that means I have to go back and revisit the idea as an instructor. So it was a kind of safe cold calling. It wasn’t like in law school, where they’re like, Okay, you, you, you, you don’t know the answer. You’re out. You know, it was very safe, safe place. But it’s the students that it made them pay more attention, and they felt like the professor cared about what they were thinking. So it was really kind of surprising. And I think Jen has said this probably better than I can say it, but like, don’t talk yourself out of a strategy, kind of like, you know, cold calling seems horrible. Like, I’m not going to do that to my students, but you don’t know, you don’t know what their experience of it is going to be.
Lillian Nave 40:44
It sounds so much to us about intent, because it’s really like that law school. I mean, I can think of movies, you know, where it’s, where it’s as if the professor is trying to embarrass somebody, right? And that’s, and you can tell, and it’s, you know, you’re, you’re trying to see, oh, that person’s not paying attention. I’m going to make them pay for that. But no, this was very different. It sounds like it was. I really want to know, as the professor, I really want to know if, if my students are getting it and and I, and I want to give them the chance to work it through, to think it through, to have some help, you know, because this is the time that we have, it’s really important, let’s get a chance that we can to go to go through it. So it’s that ethos of care seems to be very much part of Curry College. And what was part of this project, it seems to me, least that’s how I read it.
Jen McNally 41:38
Yeah, definitely. And I think that building on that idea of care, it’s really about empathy that the instructors are showing for their students, not just the I care about your emotional state right now, but the true empathy about like, I understand where you are as a learner right now, and I understand where I want you to be, and I want to help you become more authoritative over the content. Like, I want you to become an authority along with me and be able to do the things that I can do. And so it’s not a checklist. You wouldn’t see a UDL checklist that says, Use cold calling with your students, but you would have this perspective or this this attitude towards students that I know where I want you to be. I know where you are right now, and I’m going to provide an environment where we’re safe to do the things that we’re we need to do in order to become more expert in this.
Laura Callis 42:27
Yeah, even some of the professors would talk about that, that because there are social anxiety students with social anxiety in our classrooms, and we do a lot of group work, and we ask people to speak up and whole class discussion. And our instructor even talked about that that, you know, once you leave Carter, you’re going to be asked to do these things. So let’s jump together now. And he explicitly talked with a student about this while it’s still safe, and we’ll figure out, like, a way to do it maybe, maybe I, you know, come by and tell you, I’m going to call on you, so I give you a little bit of a warning or something. So, but let’s find out a way that we can do this while we’re still in this, in, you know, safer environment. So it’s kind of this empathy, of like, I know this is hard, but I believe in you and we’re going to find a way to do it anyway.
Lillian Nave 43:13
That’s so great. So there were a couple surprises in there. That’s great. Yeah, yeah. Well, what’s what’s next? So what’s next in the research? What’s next with this class? Where do you see it going from here, from this great start, yeah.
Laura Callis 43:28
So we want to share our results. So next up, we are pulling together and editing all of our video footage to develop a repository of professional development resources so that instructors can go on and learn more about different practices that might support them, and see examples of students talking about their thinking so they can better prepare for for their classes. But we’re also looking to expand our community. So as we reach out, we’ve been running some webinars. So if people want to reach out to us and join one of our webinars that would be great. We’re looking toward maybe developing mentorship so that we can help professors in other colleges learn how to do the kind of research on their own practice and their own students, so that they can learn what it is works in their context, for their students, and kind of how we were able to do that, see if they are able to adopt some of those methods too. What else? Jen, what else is next? We’ve got big plans.
Jen McNally 44:27
So, yeah, we just love expanding our community. So if you’ve heard something that resonates with you, or that sparks a curiosity, we hope that you’ll reach out and we’ll definitely stay in touch and be able to share the resources and also the approaches that we’re using that are really student centered and focused on really over sampling, so that we get the students have traditionally been left behind out of the from the research to be amplified in our work.
Lillian Nave 44:54
Great. Yeah, and we will link on the resources tab for this episode. So to the podcast, how to reach out to both of you, yeah, and some more resources about what you have been doing so folks can learn a lot more. I know you already have a good bit that you shared with the folks that were at the presentation at the conference. So there’s a lot to to get from this that people can emulate and maybe use it in their own statistics classes and see how it might work in their context as well. So thank you so much, both of you for joining me today. It was a real pleasure to talk to you about what you’re doing for your students, and I am very excited that we get to share this today, so thank you both Jen and Laura.
45:43
Thanks. Thank you.
Lillian Nave 45:47
Thank you for listening to this episode of The think UDL podcast. New episodes are posted on social media, on LinkedIn, Facebook, X and blue sky. You can find transcripts and resources pertaining to each episode on our website. Think u, d, l.org, the music in each episode is created by the Oddyssey quartet. Oddyssey is spelled with two D’s, by the way, comprised of Rex Shepard, David Pate, Bill Folwell and Jose Cochez. I’m your host, Lillian Nave, and I want to thank Appalachian State University for helping to support this podcast. And if you call it Appalachian, I’ll throw an apple at you. Thank you for joining. I’m your host, Lillian Nave, thanks for listening to the think UDL podcast.