Welcome to Episode 151 of the Think UDL podcast: Conversational Quizzes with Meghan Donnelly. Dr. Meghan Donnelly is an Assistant Professor of Anthropology at Austin College in Sherman, Texas. I had the good fortune to meet her in a UDL course for higher Ed educators and her final project dazzled me and left me wanting to know more about her use of conversational quizzes in her course. I also wanted to get the word out to others who may see this as a useful tool in their teaching toolbox. Of course this works in particular contexts such as in-person, small to medium enrollment courses, but it may be something that sparks a similar type of creative and collaborative assessment in your area. Meghan has also put together a fantastic resource document with instructions on how to replicate what she has done in her class, and you can find that on the Think UDL. org website. This conversation was so fun and creative and I think you’ll enjoy it.
Resources
Find out more about Meghan Donnelly or contact her via email at mdonnelly@austincollege.edu
Conversational Quizzes Resources Sheet prepared by Meghan Donnelly
Transcript
1:03:23
SUMMARY KEYWORDS
Universal Design for Learning, conversational quizzes, Hemingway app, oral quizzes, executive functioning, Google Docs, rubrics, student engagement, accessibility, UDL course, teaching strategies, student feedback, learning objectives, inclusive teaching.
SPEAKERS
Meghan Donnelly, Lillian Nave
Lillian Nave 00:02
Welcome to think UDL, the universal design for learning podcast where we hear from the people who are designing and implementing strategies with learner variability in mind. I’m your host, Lillian Nave, and I’m interested in not just what you’re teaching, learning, guiding and facilitating, but how you design and implement it, and why it even matters. Welcome to Episode 151 of the think UDL podcast, conversational quizzes with Meghan Donnelly. Dr Meghan Donnelly is an assistant professor of anthropology at Austin College in Sherman, Texas. I had the good fortune to meet her in a UDL course for higher ed educators, and her final project dazzled me and left me wanting to know more about her use of conversational quizzes in her course. I also wanted to get the word out to others who may see this as a useful tool in their teaching toolbox. Of course, this works in particular contexts, such as in person, small to medium enrollment courses, but it may be something that sparks a similar type of creative and collaborative assessment in your area. Meghan has also put together a fantastic resource document with instructions on how to replicate what she has done in her class, and you can find that on the think udl.org website linked to this episode. This conversation was so fun and creative, and I think you’ll enjoy it too. And I do want to thank you so much for listening to the think UDL podcast. And I’d like to welcome my guest. Welcome Meghan Donnelly to the think UDL podcast.
Meghan Donnelly 02:06
Thank you. I’m so glad to be here.
Lillian Nave 02:08
I am super excited to talk to you, because I sort of sought you out after you finished taking the cast UDL course for post secondary faculty and you had some really, really cool things that you started implementing. Like, right away you were just Johnny on the spot and started implementing things. And I thought it’s a really good case study for how to redesign a course with the UDL lens. And so that’s what I wanted to talk to you about today. And it’s just you’re very creative and inventive. And so I wanted to just let people know, like, what could be possible. I’ll start with asking about the intervention. So what were some of the interventions that you started to use while you were still just sort of learning about a little more in depth course on UDL right away, before you even redesigned what the next steps were going to be,
Meghan Donnelly 03:01
yeah, so I decided to take this course during the semester while teaching full time, and I knew that was going to be a crap shoot in terms of how it was going to play out. So the course was in February and March, and I literally started the semester a week before we started the course. So busy, yeah, so I knew that that was going to be a little tough, and my biggest concern actually was that I wasn’t going to be able to give the course my full attention. I wasn’t going to get as much out of it because of that. But it actually ended up being perfect, because it did really inspire me to, like, take whatever we were learning in the particular module we were doing and try to apply it to the classes. So, like, I used it, I did a bunch of different things. One of the things I thought was really cool that we learned about, that I just shared with some colleagues last week, was the Hemingway app. So hemingwayapp.com and for folks that aren’t familiar with it, you can just paste text into it. They have, like, a more advanced version of it that I think you have to pay for. But the basic version, you just paste text into it, and it tells you which of your sentences are hard to read, and it does some other stuff too, like it notes passive voice or weakeners, like adverbs that are weakeners. But pasting in text from, I started pasting in text from all of my instructions for in class activities and then for all of my assignments. And of course, like it shows you in red, all the ones that are really hard to read, yeah, and like my first one that I pasted in, like, basically the whole thing was red.
Lillian Nave 04:50
So you wanted to see how your students were really understanding what, like you had it in your head, and you thought you were real clear and telling them what to do. And then Hemingway was, like, me. Be Done, yeah.
Meghan Donnelly 05:01
So yeah, really thinking about that part of the of the course, thinking about accessibility, in terms of making content more readable and more understandable. And yeah, I like, you know, like many academics, I’m like, Yeah, I’m a great writer. So this is super clear, and everyone can understand this. And then a lot of it was not. And so actually, through using that, I pretty quickly started training myself what the problems were in my writing for students to be able to understand it, like longer sentences. And so then sometimes I didn’t even need to use it anymore. I would just cut a sentence in half or in threes and make it a lot clearer, or get rid of words that were in there that weren’t important. So I started using Hemingway app that was really helpful for just improving clarity for instructions, and then I did a bunch of other interventions. So I started using oral quizzes for the quizzes that I do in my classes that are like reading comprehension quizzes I did in one of my classes, a survey of executive functioning skills and then a discussion about that I totally redid, or like, basically started from scratch With The Guidelines for two of my courses, final projects. So that was something else that I, that I worked on during the course. So those are some of the things. There were a lot of other things, little things that I started to incorporate, but those were some of the bigger things that I did during the course.
Lillian Nave 06:37
Well, that makes a huge difference, and you actually make me think a lot about that dichotomy in our minds about how we write for which audience. And we tell our students all the time, who’s your audience, who’s your audience. How are you going to write to what audience? And then we often, as the academics, are writing to be get published in a paper, not writing towards our students who have never taken a sociology or anthropology course, right? Or an art history course, and we’re using all of these words, maybe even jargon, maybe even acronyms, and they’ve got a wall of text that is really esoteric to them. Oh, look, maybe I need to change my words. It’s really hard for them to to understand what we want them to do. And it is. It’s just, are we making it user friendly so that our students can end up, you know, having that vocabulary and working in the areas we want them to work in, but we also have to be speaking directly to them. That’s a that’s a huge, like, liminal threshold concept. Like, once you figure that out, you’re like, oh, I can never go back to being the esoteric writer I once was.
Meghan Donnelly 07:52
Yeah, you really can’t. And it’s something that I actually care a lot about in my academic writing, and that I talk to students about. You know, there are. There are. There are scholars who write in a way that is incredibly inaccessible, and unfortunately their brilliant ideas. Don’t you know? There are lots of people who would be able to read them, or would be able to enjoy them, or learn from them that don’t because they’re just they’re just filled with so much jargon and so many giant words and so many unclear sentences and so yeah, it’s something that I, you know, have thought a lot about in terms of of the work that I publish, but then somehow I was like, No, I think my writing is really clear for students, and 100% not, right, yeah.
Lillian Nave 08:40
And the the folks that I work with in our writing center, they say that the most, most of what they do is helping a student actually decipher what the assignment is. Not necessarily like, here’s how we’re going to write, here’s how we’re going to do this. No, no, it’s they come in with an assignment and they’re like, I don’t know what I’m supposed to do. What is this paper even about right? So again, lot of good intel from our writing assignment and writing center folks on that one. So the Hemingway app, it’s a freemium. You can get it for free, but if you really love it and need it for more things, you can pay. And I you know, it was actually through cast course that I got to know about it as well, and have been able to plug in a few things, and it’ll tell you, like, this is written at maybe a 14th grade, you know, or something or higher. And then if you’re talking to freshmen, you might need to change the wording so everybody can understand it. Yeah, absolutely. So Okay, the next question is, like, the whole reason why I was like, I’ve got to get you on the podcast, which was your multiple ways of doing quizzes now, because you started these oral options for quizzes, and I wanted to know if you could tell me more about how you use those oral options for quizzes and explain. How it went for you and how you kind of integrated that into other types of quizzes. Or, like, did everything now change to an oral quiz? Or, like, why did that go?
Meghan Donnelly 10:10
Yeah, so, okay, so just a little bit of background info in my classes. I do expect that students have done some pre work before they’ve come to class. And I know this is not, I mean, this is not unique to me, obviously, but I have heard students say like, no for this class, I have to do assigned reading. But we never talk about the assigned reading in class. And so those students are always like, this feels totally pointless that I have to do this reading. Yeah. Why am I even going to do that? That is not the case for my classes. So they always have, I mean, with a few exceptions, during the semester, but they have either a reading or a podcast to listen to, or like a couple short videos to watch, but they’re expected to have done that stuff before they get to class. And I just figured out over the years that it’s nice to have a little bit of a an accountability strategy to get right, to make it such that almost everybody, or everybody has done that work when they’ve when they’ve come to class, because we just have better my classes are really interactive, and so it’s not me standing up there and lecturing. So if most of the people have done the reading or listened to the podcast, we’re just going to have a much better discussion. The activities students are due are going to be much more fulfilling. And so I, years ago, started doing very low stakes quizzes. So there’s 10 quizzes throughout the course of the semester, each is worth 1% of their final grade, so it’s very low stakes, and each of the quizzes is just one question. Occasionally, I’ll have two questions if they’re like, connected to each other. And I was fine with that, you know, I thought, Okay, this is, this is, you know, a fine way to to measure whether they’re understanding things or to help them kind of retain key concepts they’ve gotten from the from the reading. But as I was taking the class and we got to the part about multiple means of being able to demonstrate one’s knowledge, or Multiple Means of Assessment, I realized that this was really, really limited, right? A really limited strategy, because all of the quizzes were written, all of the quizzes, and I would give students the option either to handwrite or type, right? So it didn’t have to be a handwritten quiz, didn’t have to be a typed quiz, but still, really limited. And so one day I was preparing, I was trying to think about what the quiz for the following week was going to be for my intro to sociocultural anthropology class, and I thought, Oh, I could do, I could try a new method of doing like an oral quiz. Okay, I’m sure there are more technologically savvy ways to do this than the way that I set it up, but I basically envisioned them doing it just using their cell phone or using a laptop or using a tablet and just using whatever, you know, built in recording app there was on on that device. And I will say, for my classes, the way I set this up was for them to do it in pairs. So basically like recording a conversational response to the quiz question. But I think this could be done. Could be done individually. My classes are all on ground, synchronous classes, so they were doing this in person. But I actually think it would work very well for an online course or an asynchronous course where they’re submitting work like that. And so the very first time that I did it, of course, I was really nervous, so I had a lot of like, I put a lot of time into thinking about like, how is this gonna work? How am I gonna like, you know, what are all the kind of steps of me having to explain this to them? And I knew for them that there would be some of them that would be have some high anxiety, because this would be really new for them, right? Because most professors are not doing this kind of setup and and there are students that like, it’s something new, and they freeze up and they’re terrified. That’s another reason that I thought doing it in pairs might be nice, so that if one of the people was really nervous, maybe the other person could help, kind of like, you know, just support around them. Yeah, ground and support them a little bit.
Lillian Nave 14:38
So you weren’t interviewing any students? No.
Meghan Donnelly 14:42
So they did this completely by themselves. So the way that I set it up, the instructions I gave them was Now I happen to be teaching in kind of the perfect room in the perfect building for this, I will say so I was teaching in a classroom that is in a really big. Like, it’s the natural sciences building, where there are just tons of little, like, nooks and crannies, where one can just go and sit right. Gotcha. So I said, you know, once I put you with your partner, go find somewhere to sit. I did tell them they could also go outside. They wanted to just not, like, too far afield from this. You got
Lillian Nave 15:20
to come back for the rest of the class to come back,
Meghan Donnelly 15:24
but I told them, find somewhere relatively quiet. I already, I think, because of my work as a an ethnographer, I have a good understanding of the capacities of most cell phone mics and those pens, and they’re actually pretty high quality. And even if there is background noise, they’ll capture what, what people are saying. So I just said relatively quiet. And, you know, these are this group of students. You know, they’re young people. And you know, I said, if you know both of you want to sit on the floor, you can sit on the floor, or, like, find a chair somewhere, or stand somewhere, or whatever you want to do. I had talked to a couple of colleagues, one of whom had done something similar in the past, and her advice was, make sure you give them a time limit, which makes sense, because for a written quiz, we would do that right. We would say, like, write two to three sentences. Or we wouldn’t just, you know, leave it open ended and have some people write a page and some people write one sentence. So I would have thought that the problem would be that students wouldn’t talk enough. And she said, No, I had it happen in the past. They just would like talk and talk, yeah, how
Lillian Nave 16:35
much they knew and they wanted to share, and we just weren’t asking the questions. Were
Meghan Donnelly 16:41
we Yes, yeah. So after I got that advice from her, I set a time limit. Now it depended on. I ended up replicating this in all of my classes. So it depended on the question. Also depended on sort of the level of the class, but I would basically say between two and four
Lillian Nave 16:57
minutes is the response for both of them, or per person
Meghan Donnelly 17:02
for both of them. And we did talk in advance that I was like, this shouldn’t, ideally, should not be two monologs. Ideally, this should be a dialog. Ideally it should be like, if I’m doing it with you, Lillian, you’re gonna say something about some aspect of the reading that I asked about, and I’m going to respond, and I’m going to say, Yeah, I thought that that was interesting, too. And I also thought that this other thing was interesting, right? Like really trying to build, like a verbal discussion board, yes? For really trying to get them to build, because I’ve noticed in my time teaching, even students who are very hesitant to do, to participate when it’s the whole class, right, to be the only person speaking at that moment in class 25 other people looking at you? Yeah, even those students, generally, in most cases, are just fine in small group discussions, talking and sharing with others, right? And so I was really trying to build on that, that that you see them talking to each other and and I’m like, okay, they can do that. They can, like, have that kind of conversation. And basically it just ends up being me listening in on the conversation that they’re having, working through these ideas. And the way that my quizzes are set up. They’re low stakes, and so I always tell them it’s okay if the answer isn’t right, I just want to know, did you listen to the thing? Did you read the thing? Did you actually make an effort to understand it? And I want you just to kind of talk through that. And so I gave them those instructions. I had I on the slides, I went through the instructions one by one, and then I had printed them out, because also, another thing that we were learning about in the course was working memory, and so I was nervous about sure I just gave them all these instructions. But this is brand new kind of thing I’m asking them to do, and like, they’ll have questions, yeah, and they’re leaving the classroom, so they’re going to be by themselves. So I printed out the instructions and gave them those instructions, and I then I projected the quiz question. The quiz question was also on the instructions, and I just gave them. I probably gave them two minutes to just think a little bit about their response. And some of them, without prompting, kind of jotted down some notes for themselves, of things like, oh, this might be something I want to talk about. I just figured they might need a minute to process before they got together with their with their partner, and then I sent them out to do it. Two pairs stayed in the classroom because it was a big classroom, and so it was okay to have a couple people still in the classroom. And the part I was actually most nervous about in terms of tech. Technology was them submitting the
Lillian Nave 20:01
quizzes, right? How did you do that? And by the way, how many students did you have in this class? There’s 30 in that particular class. So there were 15 different audios to listen to. Okay, yes. So there
Meghan Donnelly 20:13
were 15 that particular day. I had told them, if you have an Apple device, one of you has an Apple device, use the Apple device because I have an Apple device, and then you’ll be able to just air drop the audio file to me, and that might be the most seamless way of of doing it. But the others I actually had created, I created a Google Drive folder for them to, like, drop the files into. If they didn’t have an Apple device. That ended up being more complicated. So I really, yeah, it was like, this was they didn’t understand. They’d like, get to the and then they’d be like, how do I get the file into the folder? It just wasn’t that helpful. So actually, what I started doing after that was just saying, if you have an Apple device, you can air drop it to me. If not just use some other method to get it to me. You could share it with me. You can email it with me. And of course, I mean, most of them are pretty technologically savvy in that way, and had no problem getting it to me. And then I just did a quick check, do I have all of the files right? I just wanted to make sure, okay, everybody’s back in the classroom. Do I have all 15 files that I’m supposed to have? And I did, and it was fine. So it definitely took more time than, like, the short, written quizzes that I was used to. But it was so it was so worth it in terms of like they were really excited to do something that was they get bored. I mean, there are students who like consistency and who like routine and who like predictability, but in general, I think students get more excited when it’s something that is not the same old thing that’s always being asked to them, and they get nervous. It’s quite novel. Yeah, it’s novel, but, and they get nervous about it, but they’re like, Oh, that was cool. Like, I like, we did this new thing. Also, I think kind of the thing of, it’s always scary. I mean, I think for most people, there’s probably a small percentage of the population who never gets terrified about doing new things. But think most people get scared of doing new things, and then once you do it, it also is a great learning experience. You’re like, Oh, I thought that was going to be weird. It was actually fine, or it was weird, but it’s fine. And I did it and it’s over. I don’t have to do it every day. I don’t have to do it all the time, yeah, and then the thing that for me was the most. I did not expect this. So I always have some students when I’m doing the written quizzes who just consistently appear to have not done the reading or to have not listened to the thing they had to listen to or and truly in the past, that was always my impression, like, oh, X student is just not doing the reading, and I send them an email and be like, you know, you really got to do the reading. If you want to do well in the class, you’re not going to learn as much if you don’t do the reading. So that quiz was, I think, the fourth quiz that I had done in that class, and so I had a couple of students who had done poorly on the first three quizzes, and the impression I had formed of those students was, Oh, they’re not they’re not doing the reading, they’re not preparing before class. And then we get to the oral quiz, and all three of those students did great on the oral quiz. They clearly had, and this was for a reading that they had done. They clearly had done the reading, they understood the reading. They had things to say about the reading. They interacted really. You could tell that they felt more confident talking about the reading. And it really made me reflect that when we have this very limited way of measuring, yeah, just written in a timed environment. In a timed environment, very small timeframe to do it, yeah, like that was the barrier. The barrier was not that they weren’t doing the work the barrier was like this particular kind of assessment that they were really struggling with. And so that was pretty mind blowing to me, just listening to them and hearing their confidence and their knowledge and that they really could shine in this different kind of. Of format, and it really up ended this idea that I had that was about like, oh, that student is just not doing the work that they need to do in order to succeed in the class. And I will say for folks that have been teaching for a long time, and if you have quizzes like this that are not multiple choice quizzes that you have to go through. You know, when they’re written, you have to evaluate every single one of them. Man, was it so much nicer listening to 15 interesting conversations where I was, like, getting a window into their interactions with each other and their personalities and like, it was just, so it was really, it was so nice as a kind of break for me too, from reading written responses.
Lillian Nave 25:47
Yeah. So the novelty is good for us, both students and for us, isn’t it 100% Yeah. And you know, you were still providing a lot of structure, because the students knew that there would be an assessment, right? There’d be 10 of these. So it’s not like it’s like a pop quiz that, yes, you never know what’s going to happen. It’s they knew they were responsible for doing something. The cadence of the week was the same, but the novelty came in just the mechanism of that assessment. It was different. It was much more familial or just collegial, right? You can, you’re talking with a friend, and yeah, we’re just, it’s more like we get to eavesdrop on that conversation, rather than the student stressing. I’d say a little much more about the evaluation when they’re writing, yeah, it’s down. So yeah, you get a lot better output it seems Yes.
Meghan Donnelly 26:39
I mean it really. And I will say, if there is a concern about, like, oh, this student is not keeping up with the reading, I actually could tell much better in the spoken responses, because students, again, like they’ve been in school for a long time, they’ve been doing written responses for a long time, and students have, I think, just, it’s like a survival skill. Just have found kind of savvy ways to make it seem like they’ve done the work that they needed to do. It’s really hard to do that. I realized I hadn’t even thought about this. When you’re talking about it, it just becomes pretty clear when you don’t have any sort of grounding points for what you were expected to do. But those students, I mean truly, those students, at least in the in the context that I’m teaching in, tend to be the exception. Most students are coming to class prepared and having done the work. And so this just became, I think, a way to expand the field for for them to be able to express what they had, what they had learned, or what they knew in a different way, in one of the other so I ended up doing this three times each in all three of my classes. Okay,
Lillian Nave 27:57
so it wasn’t every quiz, no peppered in there. So you heard multiple different ways. So if some students are much better writers, they still had plenty of chance to showcase their writing,
Meghan Donnelly 28:08
that’s right? And I talked to my students about this, right? We talked about it explicitly in one of my classes. One of the students asked, this is a student who wants to be an educator herself? And she said, Oh, why did you like, why the oral quizzes, and I explained, you know, I’m taking this class on Universal Design for Learning, and really trying to make it more accessible by having different ways that students can express their their knowledge. And so that was very helpful for them to hear, right? Like, Oh, you know, not everybody does super well with this format. And again, like you say, the students who do do well with writing, then don’t have to be panicked that like, Oh no, we’re now everything is going to be oral and like, That’s terrifying. And I also started doing some ones that were visual. So I did this in two of my classes where I had them create I’m not using these terms properly. I hope people will not like I know that there’s a difference between a concept map and a mind map, but I don’t necessarily. I cannot articulate what that difference is but a visual representation of a concept from the reading right where they had to do something, and I told them that it could be like a concept map with the circles and the lines. Yeah, it could also be an illustration. I told them it could be anything that you can use. And so then they created those on the whiteboard. Everyone could see them, which is really nice. Now, of course, you’d have to be in a classroom where you have enough whiteboard, right? Man as a teacher, whiteboards are like my favorite thing to have just lots of them for them to work on. But they could do it on paper too. But it was nice on whiteboard because everyone could see them. And then in that class I had other I again, this was in groups that they were doing it in groups of like three students. Each I had other groups vote for that they would say, Oh, I think that group should explain their drawing to us, because that drawing is awesome. Yeah, I love doing that. I do. I love doing that because it’s it just, I think is so nice to see them lifting each other up in that way to be like, Oh, that’s really smart. I really want that person to, you know, explain to us what they did. Yeah, so it was great, and I will absolutely continue, continue to do it. In my intro class, I ended up actually making the second exam in the class, I ended up making part of the exam, an oral exam with me, so them actually coming in and me asking them the questions. But that felt like it paired really well that they had already been kind of practicing the skill in doing the oral quizzes, so they weren’t so unfamiliar with the skill of sort of talking through their their knowledge about something
Lillian Nave 31:02
great. Yeah, wow. I was so excited when you told me about that, and I really liked the way you went about it too, that it was the students interviewing each other, and that it was really just a conversation. Because if we don’t do that, then it’s only one person like you might say, Okay, what did you get from the reading? And then 2530 people are looking at one student, and there’s only one student who’s putting forth the effort there, and to me, it’s the one who’s doing the work, is the one who’s doing the learning. And so if we’re up there lecturing the whole time, we’re doing a lot of work, and to get the students involved, you’ve made it so that every single one of your students work on answering that question and and getting feedback, really from their peer, because they’re building on that. So I just thought it was a beautiful loved it wonderful idea. So, but you did so much more. So I wanted to ask a few more things, because you said you also changed you read, you revised your final project guidelines, and I wanted to know what happened when you did that and what changed.
Meghan Donnelly 32:08
Yeah, so, so these were actually the guidelines. I was really starting from scratch. I think this makes it a little easier, actually, that these two final projects, one was for a class I hadn’t I was teaching for the first time this semester, so I didn’t have final project guidelines. If any students are listening, your professors don’t make the final project guidelines like in January
Lillian Nave 32:33
for the first time they teach something, they may not have it figured all the way out by January 1 for the May 10 deadline. Oh, my God, may change. Oh, man, it’s up here in our heads. Yeah, it’s there, it’s there, but it’s might not be anywhere else, anybody else could see it.
Meghan Donnelly 32:53
That’s right, and ideally, it would be all done. But that is not always how things work. So that actually I will say that in this particular case, that ended up being a positive thing, because then the things I was learning in the UDL course, I realized that there were a lot of things that I could apply to the way that I created the guidelines for these two projects. So really, the two things from the course that were that that kind of inspired me to do it this way. You had shared the tilt checklist,
Lillian Nave 33:28
yes, transparency and learning and teaching Mary and Winkle Miss
Meghan Donnelly 33:32
Yes, you, you had shared that with our small group and and I realized I looked at it. The first question is like, does your do? Your guidelines have a clear purpose section that say what the purpose? And I was like, no, they don’t. And I literally have never made a set of guidelines that say this is why I’m asking you to do this assignment, and this is what you’re going to get out of doing this assignment. And sometimes I think as educators, we just think, Oh, isn’t it obvious?
Lillian Nave 34:07
Clear. It’s clear, obviously, of course, you know why you’re writing this paper. No, it is not. It is not.
Meghan Donnelly 34:14
I mean, kind of like what you’re sharing about students going to the writing center and not understanding well then obviously it’s not clear why they’re doing the assignment or how did so. So I was working with that checklist. And then another one of the facilitators from the course, Dr Whitney Griffin, had shared with us an example of a set of guidelines that she had used, and the way that she phrased it, if I remember right? Is, this is what I’m asking you to do. This is why I’m asking you to do it. This is how to do it. Is the like, you know, kind of way, the and the why being the the purpose, right? So, so I started with those two models. The other thing I really. Liked about the way that Whitney did hers was she used a Google doc to create the guidelines. And this semester, I actually created, I used Google Docs to create my syllabus. And I love, I mean, of course, this is not unique to Google Docs, but I love the way Google Docs, you can create the headings and there’s an outline on the left,
Lillian Nave 35:18
make it accessible, people. Oh, it just
Meghan Donnelly 35:21
for the first time. I realized this semester, students actually were fully reading most of my syllabus because they had those headings that they could use to navigate through the document. And I have, like, I have a section called resources for students. I have had it in my syllabus since I started teaching. I’m pretty sure this semester was the first time anybody ever looked at
Lillian Nave 35:48
because it was findable that you see it
Meghan Donnelly 35:50
was findable, so, but I hadn’t really thought about doing that for guidelines. I always had created guidelines, you know, as PDFs, and uploaded them to the to the learning management platform and emailed them to students, or printed them and given them to students. But so I decided, following Whitney’s model, to use a Google doc to do this. This, I will say now this is just anecdotal, because it’s just from these two classes. I think I only had two students in those two classes. That was 40 students total who didn’t follow the
Lillian Nave 36:25
instructions. Nice? Is that a marked improvement?
Meghan Donnelly 36:29
Yes, yeah. And so that indicated to me that they were more accessible. They were more readable. Again, I was using Hemingway to make sure that the these were much more readable. I, for the first time, created a clear purpose section that had the learning objectives for the assignment, and it really forced me. The other thing I like about the tilt checklist is it asks you to articulate to students how this particular how achieving this particular learning objective is going to serve them beyond the context of your specific class, right?
Lillian Nave 37:05
Like purposes or life application, you’re going to need to make decisions later on. Yeah, you’re going to need to figure out what you know, ductwork to buy when you’re at Lowe’s, you know? How are you going to figure that out, yes, at some point you’re going to need these skills,
Meghan Donnelly 37:24
yes. And so that was really helpful for me, because now, actually these were assignments that I already knew what the assignment was going to be, but actually think really in terms of thinking about backward design, really thinking about that part, and intentionally designing assignments that will be useful to them and that, and really thinking about in terms of things like self efficacy, if they feel like there is a higher purpose to this assignment that they are doing, probably their chances, their their likelihood of persisting and doing that assignment is going to be much greater, right? And and their and their commitment to kind of doing it well, not just to check a box to get it done, right, which is, I’m always trying to get them away from this sort of transactional idea of just like getting it in, checking the box, right, but really feeling like you’re growing from it and you’re getting something out of it. So I created that purpose section, then I created a specific instruction section. And the nice thing about this was I in one of my class, one of the two classes, I basically, like, shared a beta model with them of the instructions, and then said, what’s missing? Because I don’t know if you experience this, but I often forget important things from the instructions that I just think, oh yeah, this is so obvious. So they would ask questions like, oh, well, does it need to have a title, or does it need to, like, that kind of stuff, yeah. And because it was in a Google Doc, I could update it without having to do the annoying thing, uploading the new version, telling everybody this is the new version, don’t use the old version. Always worrying that somebody has the old version, right. So that was really nice that I could kind of do that and get their feedback. I also created glossary that, again, when we were learning in the course about making content more understandable, and I think that cast does a really nice job of modeling this in the way that the course is designed, of creating glossaries of terms that are important for them to understand. And so I created a glossary with things that I thought they might not understand. And sure enough, I got questions like, what does this mean? And I would explain, but I’d also say, hey, remember, there’s a glossary where you can look these terms up, so technical terms or terms that were like, really important for them to understand what they were doing in the assignment. And I added those on to the i. Yeah to the end. And so that worked really well. I Oh, and I, I don’t know if you were taught to create rubrics. I was never taught when I was learning to teach to create rubrics.
Lillian Nave 40:13
Most of us in grad school don’t get any sort of teaching, yeah, courses about how to teach the thing you know infinitely more about than anybody else on the planet. But it’s just go ahead. Everybody can teach, right? That is not true. It is your reader, dear listener,
Meghan Donnelly 40:33
easy to me, it is. It is especially because most people who go to graduate schools are not going to end up teaching at a research focused university where most of their job is research and very little of it is teaching. Most people just statistically based on the kinds of schools out there, right? Aren’t going to be they’re bigger. Yeah, more. There’s more places where your teaching is going to be the heart of your work as an academic, right? And so so I really don’t have any all the training I have in creating rubrics is is self trained. And I think for that reason, sometimes I have avoided them because I find it to be a really challenging task to create a good rubric, a high quality one that will help students understand the kind of bigger purpose of the assignment and be able to be on track for that. But thinking about transparency, it makes a lot of sense to me why a rubric is important for transparency. Students need to understand how they’re going to be evaluated before and while they’re creating the work that they’re that they’re doing. And so I for both of those assignment guidelines, I linked a rubric in them. That’s another nice thing about Google Docs, right? Or another online platform? Link to it. I will say the rubrics that I created are still probably like B level rubrics
Lillian Nave 42:13
you would if you had a rubric for your rubric. It might, it might get a, you know, need some improvement, yeah,
Meghan Donnelly 42:21
approaching competency, but not competent.
Lillian Nave 42:24
The rubric for this rubric.
Meghan Donnelly 42:29
So I have work to do on that, but I’m committed now, like my guidelines, now have a rubric section, and so I’m committed to there always being a rubric for assignments, and I know that I’ll get better at doing it as I as I lean into those being there. And then I was also able to link some other things, like for both of these, these were kind of more creative projects. So I could link examples of of projects that students could use as kind of a guide or a model for what they were doing, which was, I think, very helpful for them, yeah,
Lillian Nave 43:02
yeah. And I will say too, is that some people are pro rubric, and some people are not. And not everything has to have a rubric, but having but in the tilt method, the last part is the criteria for success, and one of those things can be a rubric. And I use them sometimes, but I don’t always. And so it could also be maybe what’s called a one point rubric, almost like a checklist, like, make sure that you’ve included this, this, this, this and this, or you have an annotated example of students work. Like, here’s what it should look like. Yours may look a little different, but look at the components that this have, and I’m going to show you, like, the first paragraph is, let’s shows the whole project how it’s going to be outlined, and then the second paragraph goes into the methods. And, you know, so there are lots of ways that the tilt method doesn’t necessarily say you have to have a rubric, and the why, what and how doesn’t it doesn’t have to. And a lot of people are like, either afraid of rubrics, like I was forever and didn’t or, first of all, actually didn’t know what they were and never seen one because I wasn’t in education at all. But something, something that tells the student like, what is successfully completing this, something that gives them a clue as to what is, because you’ve got students who’ve never been in an anthropology class, so they don’t know what competence looks like or excellence looks like. So let them, let them know. Let them have a clue, really. So what we’re doing, we’re handing out clues.
Meghan Donnelly 44:34
Yeah, absolutely. Yeah. I was talking to a colleague about this, who was saying that someone had critiqued her for providing students with examples of what, yeah, of what she was looking for in a particular assignment. And the critique was something like, if you give them these models, they’ll just be formulaic about creating something. That’s just like that model, and they won’t be creative, and they won’t be able to, you know, kind of produce something that is more original work. But I think, in terms of thinking about this question that you and I had talked about, you know, this difference between cognitive rigor and logistical rigor, if you’re not giving them any model to work with, you’re really putting them in a place where you’re you’re putting so many barriers on them that they’re never even going to get to a place where they can be creative, right, right? They’re going to be paralyzed, yeah.
Lillian Nave 45:34
What is it? Am I supposed to do an interpretive dance? Or am I supposed to, you know, write four paragraphs? I don’t, you know, I don’t know what you want. What do you want from
Meghan Donnelly 45:44
me? Yes, and at least where I teach, we have about in the in the incoming first year class this year, I think about a third, a little over a third of our students are first gen and so there is a big disparity between somebody who both of their parents went to college or maybe even have an advanced degree, and their parents have given them a lot of kind of independent training on, you know, if they have a doubt about something, they go to their mom, and their mom was like, Oh, this is How you do it, right? Whereas first gen students often their parents don’t have that kind of inside knowledge, and so there just is a big disparity for those students in being able to navigate unknowns and understand, you know, how to navigate an assignment that has very little by way of instructions or concrete, you know, examples, yeah,
Lillian Nave 46:44
and, and higher ed is just a whole different world. It’s a different culture. It’s like going into a different planet, you know. And so we cannot expect that students are just going to figure it all out. It’s not fair to them. It is. It’s like, welcome to this new country where we have a different language and we have a different, you know, monetary system. They’re called grades. But anyway, you know, and just expect that, oh, yeah, I’ll figure it out, or I’ve got it figured out. Some students have come from that country because they were born there, and then some students have not yes,
Meghan Donnelly 47:23
absolutely, and just thinking about even the thing, you know, thinking about questions of of executive functioning, and students that really struggle with that, you know, when I talk to my students, they say, really, it’s unusual, the situation that you’re in where, depending on wherever you’re Going to college, you have, you know, four different bosses every semester, or you have five different bosses every semester, and each one of those bosses has a totally different set of instructions for you. And some professors use the learning management platform, and some don’t, and some want everything on paper, and some want things emailed. And it’s a lot of different kinds of instructions and approaches for them to manage. And I think each of us as professors sometimes operate with the assumption that everyone else is doing things exactly the way we’re doing things. And why don’t you, why don’t you know to do this thing? Because Shouldn’t it be obvious to you? But it’s not. If another professor has said, Don’t do this, right? And then I’m saying, right? And this has come up a lot in conversations about generative AI, because so many different professors have different takes on it, right? You can use it in this way. Don’t use it at all. You know? How dare you use this? When some other professor has said, Sure, it’s fine to use that to outline, yeah, it’s just a lot, I think, for them to manage, and I think we need to be a little bit more empathetic and a little bit more patient about about that, and just be more transparent about what we want for our classes, how things should be done for our classes, and just provide them with those resources.
Lillian Nave 49:01
Yeah, and I know you had a one of the things that you did, kind of inspired by the UDL course that you were taking with us with cast, was you have this executive functions discussion with your students. So what did, what did you learn from them, and what did they learn from that? I know you already, you just spoke about starting that conversation,
Meghan Donnelly 49:19
yeah. So I realized, I have to say, one of the classes that I taught this semester, you know, when you think about teaching regrets that you’re like, Ooh, I really don’t love the way that I did this, right? And one of the things I came to reflect on a lot this semester was that I’m always nervous in my classes to have, like, just a few high stakes assignments, and so for, you know, a couple of years now, I’ve been designing my classes so that they’re smaller assignments and they’re scaffolded toward the final project. But I realized this semester that. I was having more students, it seemed to me this particular semester, whatever group of students I got who were struggling to get the smaller assignments done, and thus were falling behind. And it was having this kind of snowball effect, because it’s super stressful when this happens where you didn’t do assignment one, and then here comes assignment two, and you still haven’t done assignment one, and then you don’t do assignment two, and then it really creates, you know, just a lot of stress. And so I really wanted to understand at that point intuitively, and then I think also at this, you know, I was taking the UDL course, so I was like, Ooh, I think some of this is just the way I designed. This course is making it really tough for the particular kinds of students I have. And we have many students who come from low income backgrounds, who are who are working. I have students who work full time. I have students who, you know, have multiple part time jobs. I also have students, many of our students are from Texas, and many of them are from, like, the North Texas region, and so many of them have family obligations, like they have to take their grandma to the doctor every week or they, you know, have family obligations. And so I was realizing already that there was something going on where they just it just wasn’t enough time between assignments for them to be able to manage all that stuff and get this done. But I wanted to understand if some of it was executive functioning challenges that they were having challenges with planning and organizing how they were going to get this work done, and questions of things like even just like remembering the assignment in advance and remembering to get it turned in, and all of that kind of stuff. And so I just did a very simple two question quiz with them that was about those two things, right? Like, are you having any struggles with this, because you’re having a hard time planning your kind of like work, right? And are you having any struggles with like, I just forget about the assignment and then it’s due, and I get the notification from the learning management platform that, like you should have already turned this in, and it’s not done. Not surprisingly, the majority of students said that they struggled with planning and organizing. A smaller percentage said that they struggled with the thing of forgetting that it was due and then, you know, and then not getting in. Though, I think there probably are more who remember it pretty close to the deadline, right? Like they don’t remember it after the deadline, but they might remember it pretty close to the deadline. And so really struggling with being able to kind of look at their week or at weeks into the future, and kind of think about, here are my different time commitments, and where am I going to fit in the work that I need to do for for these assignments and so and then in class, we talked about it, I was like, Wow, a lot of people are struggling with planning and organizing. I share with my students I never did anything until the last minute in college, because I think as professors, we can appear to them like, Wow. She like, really has she really has it together, and she’s so organized. I never did anything in advance. I’m shocked when I get a student who submits something before the deadline, yeah, because that’s not how most college students are functioning. And I also shared, you know, for college students, your executive functioning skills, they’re they’re still being developed. You’re still in that process. You gotta, you know, and and one of them said, Thank you so much, because I think other professors don’t understand that. And I said, I think professors know that. I think just sometimes they get like they just maybe forget that it can be hard to remember. Oh, this is what it was like when I was 20 and I was in college, and I was navigating all of these different things and trying to get my work done. So that was a really good conversation that has helped me. At that point in the class, it didn’t feel like I could like, sort of start from scratch, right? It was like the class was designed in a particular way. I thought it would end up being more disruptive if I was like, stop.
Lillian Nave 54:36
We’ll start out doing this anymore.
Meghan Donnelly 54:40
But I took a lot of notes about how I will do things differently for the future, and I had shared with you teaching a methods class in the in the fall. And one of the big takeaways that I had from this class that honestly, this class was awesome. It was one of my favorite. Classes that I’ve ever taught, but the way that the assignments were designed was not great, and so one of the big takeaways from that has been I am going to have fewer assignments, and so they are going to be higher stakes assignments. And I realized there is kind of just a trade off with this, especially with students who have a lot of stuff going on outside of their college work, but I’ve realized that one of the ways that I can kind of balance this or or like support them better, so that their chances of success in the assignments is much higher, is doing a lot more in class work, and this is a methods class, so we can do that right, like we can do a lot more in class to prepare them for doing these assignments, so that everyone has a good shot of doing well. And I also have already just decided this is the methods class. It will have 25 students in it. So I, I, I just I feasibly cannot say, oh, everyone can redo the assignment if they want to redo the assignment if they don’t do well. But I can say, you know, anybody who is at that place where, like, they haven’t yet reached competency, I can say, you can redo this assignment and have another shot at it if you don’t do well, because I think that’s the only way to do it equitably in terms of thinking about these kind of high stakes assignments.
Lillian Nave 56:26
So great. Well, good, yeah, so you’re already applying it to next year’s class for anthropology and sociology in the fall. And I guess my last question then is you have already shown me you’ve learned a lot from the course so, and I love all of the things like this really changed a lot about how you were teaching and how you’re going to teach. And it’s just fantastic. You’re such a good like, poster professor about, like, how UDL is so, like, invigorating. Like, it’s been such a good thing too about your teaching. It’s more fun, it’s more exciting and engaging, and you’re learning more about your students. So anyway, I just that’s what I love. So I’ll ask, what then do you think are your biggest takeaways from kind of going through this almost like a UDL cocoon and transformation now you’re a UDL butterfly professor.
Meghan Donnelly 57:27
I have been interested in UDL for years, and I was so excited that I had the opportunity to take this course. But I will say I think I was one of the people who thought, Yeah, I absolutely agree with all of that in principle, yes, but man, does it sound exhausting to have to redo everything I’ve already done? So, you know, especially for folks that have been teaching for years, and there is this kind of thing which I actually disagree with. There’s this sort of idea that, like, once you’ve created the course, then you can step back and, like, relax a little bit. This is not really my approach. In general. I tend to tweak and retweak and retreat. You know, I just think you you do need to keep working on your classes, but I can see someone who they’ve put so much time and energy into designing classes, and then you’re looking at the UDL guidelines, and you’re like, Oh man, I gotta, like, do all
Lillian Nave 58:29
of it a second. I gotta start over. There’s three whole columns and like, 29 things,
Meghan Donnelly 58:36
like, how am I gonna do this? And I think one of the things that the course does so well is emphasize you don’t have to do all of this right now. You don’t have to start from scratch. I think everybody, including you, involved in the course, are just so helpful in saying, just start where you can start, like, whatever you can do, whatever interventions you can make you, like, start with this one small thing, and it builds from there. And in my experience, it gets you excited about, like, Okay, this works. This is helpful. I have students that are benefiting from it. I have students who are more excited about learning because I’m trying these new approaches that makes you want to do more of it, right? And the other thing I just had a friend last week who was asking me, I was telling her about the course, and she said, Do you think it was worth it? And I said, yes, absolutely, because it provides so many concrete tools and approaches and examples that you can use. There just is so much where it’s like, well, I’m interested in doing this. Here’s this particular platform you can use, here’s this example of how you can make this happen. And I think that makes it easier. It makes it easier to not feel so overwhelmed. Calmed about what is a very big project like, it is a big project. Oh, yeah, to do that a whole course, absolutely, it’s huge. And, and it gave me the, you know, I started talking to my students about it and saying, like, Hey, I’m working on this project of redesigning my classes so that they’ll be more accessible, but it’s a big project, so it’s gonna, like, take a while, and you’ll see that I’m, like, tweaking things here or there. And honestly, students are so appreciative, even just of that willingness to try to make the learning environment more inclusive. You know, like, even just thinking about students, when I first asked students for my exams, is it okay with you if some students handwrite the exam and some students type the exam, and I would get responses from students, like, everyone should use the method that works best for them, like, they’re the poster children For UDL, right? Like, like they understand that, right, that it doesn’t matter what method you’re using if you’re able to express what you’re learning. And so students, I think, are really excited about it, and that makes me more motivated to do it. But, yeah, I love the course. It was great. Yeah, it was wonderful.
Lillian Nave 1:01:20
That’s so fantastic. I love being a facilitator for them. It’s just a fantastic way to meet amazing faculty like yourself, but also to journey through that and to see how I learn more as well. Just in every good course you’re learning from your students. And I just, I’m just going to shout out again to your your conversational quizzes is pretty much what they were, and what a great conversation to listen to later to really hear what your students are thinking. So thank you so much for being on the podcast. I really appreciate it, and I’m excited that you have all these new tools in your tool belt and can move forward with UDL. So thank you for joining me.
Meghan Donnelly 1:02:06
Thank you so much, Lillian, this is wonderful.
Lillian Nave 1:02:11
Thank you for listening to this episode of The think UDL podcast. New episodes are posted on social media, on LinkedIn, Facebook, X and blue sky. You can find transcripts and resources pertaining to each episode on our website, think u, d, l.org, the music in each episode is created by the Odyssey quartet. Odyssey is spelled with two D’s, by the way, comprised of Rex Shepard, David Pate, Bill Folwell and Jose Cochez. I’m your host, Lillian Nave and I want to thank Appalachian State University for helping to support this podcast. And if you call it Appalachian, I’ll throw an apple-at-cha. Thank you for joining I’m your host. Lillian Nave, thanks for listening to the think UDL podcast.