Welcome to Episode 75 of the Think UDL podcast: Academic Integrity through Assessment Design with Mark Glynn. Mark Glynn is the Head of the Teaching Enhancement Unit at Dublin City University. I have been wanting to talk to Mark for a long time about what he is doing with UDL in Ireland and beyond. There are so many things that he and his team are doing to help spread the word and train faculty in UDL, that it was difficult to focus on just one topic. Recently, he has been working on the intersection of UDL, academic integrity and assessment design which has really piqued my interest. And so today’s conversation centers on how to make assessments more “cheat-proof” and authentic, so as to work on the prevention of academic dishonesty rather than “catching” students cheating after the fact, so to speak. And the UDL principles are what guides this design! Mark has also graciously provided many resources that he mentions during our discussion and you can find all of those on our ThinkUDL.org website for this episode number 75. Thank you for joining us as we talk about authentic assessments, academic integrity and UDL!
Teaching Enhancing Unit (TEU) webpage on assessment design
UDL Progression checklist that we use to promote that “one small change” drive that we have in DCU
Great examples of how assessments have changed captured by Sally Brown
This blog post for Ouriginal
Lillian Nave 00:00
Welcome to think UDL, the universal design for learning podcast where we hear from the people who are designing and implementing strategies with learner variability in mind. I’m your host, Lillian nave. And I’m interested in not just what you’re teaching, learning, guiding and facilitating, but how you design and implement it and why it even matters. Welcome to Episode 75 of the think UDL podcast academic integrity through assessment design, with Mark Glenn. Mark Lynn is the head of the teaching enhancement unit at Dublin City University in Dublin, Ireland, and I have been wanting to talk to Mark for a long time about what he’s doing with UDL in Ireland and beyond. And there are so many things that he and his team are doing to help spread the word and train faculty in UDL that it was difficult to focus on just one topic. But recently, he has been working on the intersection of UDL, academic integrity and assessment design, which has really piqued my interest. And so today’s conversation centers on how to make assessments more cheap, proof and authentic, so as to work on the prevention of academic dishonesty rather than catching students cheating after the fact so to speak. And the UDL principles are what guides this design. Mark has also graciously provided many resources that he mentions during our discussion. And you can find all of those on our think udl.org website for this episode number 75. Thank you for joining us as we talk about authentic assessments, academic integrity and UDL. Thank you so much, Mark Glenn, for joining me today to talk about what you have been doing at Dublin City University. I’ve been following you for quite a while. So thank you so much for taking the time to talk with me today.
Mark Glynn 02:18
Thanks, Lillian. And thanks for the invitation.
Lillian Nave 02:22
I’m very excited about this. So I’m going to go right into my first question, which is, as you know, what makes you a different kind of learner?
Mark Glynn 02:31
I’ve heard this question so many times at this stage. And all the time as I’m using in the kitchen, how am I going to answer that question? For me, very similar to a lot of your previous speakers, I don’t see myself as two different there’s little bits and pieces that would make me slightly unique, but not completely different. In so far as my, the people around me, influenced me an awful lot. And I would definitely, if I was looking at a pedagogy type, I’d fit into the social constructivist, you know, but I learned by doing I learned by stuff resonating with me seeing examples. I’m not really big into reading reams upon reams of text. So I much prefer something that’s explained in ordinary language. I actually sometimes think it’s a skill that as academics, we miss out on we sometimes try to overcomplicate something. And listen, if you can explain it to your granny, you can explain it to anyone. That’s the way I’d look at it. And for me, that’s the way I like learning and also the way I like teaching.
Lillian Nave 03:43
Right. So making it understandable to your audience. Yeah.
Mark Glynn 03:47
And for me, that was a big difference between college and my school experience. I wasn’t a big fan of my school experience. But when I went into into college, they they made it understandable. They made it releasable. And they essentially treated me like an adult as those awkward teenagers who taught he was an adult long before I actually was,
Lillian Nave 04:10
you know, yeah, a wise soul. Right? Or cheeky
Mark Glynn 04:15
soul depends on who you
Lillian Nave 04:19
one or the other, right? Maybe both at once. Yeah, that knowing your audience is so important. I’m, I’m learning that my audience constantly changes to and as you say, you know, wanting to speak to your students as adults. It kind of kept them where they are, where it’s authentic, where it makes sense to them. Is I’ve noticed it’s changing a little bit too with my students. I’m getting first year students who have spent the last year and a half during a pandemic during their high school years. So then they come to college and they are not like the students I’ve had for You know, 18 years before the last two. And so it’s it’s trying to figure out what my authentic way of teaching is for a students who are, are slightly different, and how am I going to match their, what they’re coming into college with. So I think that is really important. And we’re gonna be talking about these sort of authentic things coming up to that’s what I’m really excited about.
Mark Glynn 05:24
In recent years, my students have actually been different because they’re not students or staff, because the main part only role is to support academic staff. So that in itself brings a different challenge when it comes to teaching.
Lillian Nave 05:39
Yes, absolutely. We have to know our audience and, and be able to speak to them, not below them, not above them, but but kind of explain right to where they’re going to grasp and kind of start running with the material. And it’s, it’s a delicate balance. I’ve, I’ve definitely struggled with that sometimes. So, okay, so in in my teaching career, you know, I’ve worked with a lot of different groups of learners. Sometimes students also like you, sometimes staff, I’ve had first generation students, international students, we also have, I’ve worked with the T fellows, which are teaching excellence, fellows that come from all over the world, to learn at Appalachian, about how to be better K 12. Teachers. I’ve taught at different institutions. And I’ve found that some students don’t understand some of the things that we maybe take for granted in academia, including academic integrity, and what academic integrity means. So how would you define academic integrity in your context?
Mark Glynn 06:43
For me, academic integrity definitions are there for everyone to see, you can have it and Google a local, very sort of different places, horrible for me, I want students to see academic integrity as part of their studies, rather than rules to have to follow. You know that that’s the way you would look at academic integrity, it should be something that’s that’s integral to what we do, and something that we aspire to all the time. But that’s the way I look at academic integrity. And, like, as I say, you can you can look up the definition anywhere and talk about honesty and morality, morality and commitment to commitment to behaving appropriately. But for me, I just want to see it as part of what we do. It’s part of everything we do and strive to be so.
Lillian Nave 07:40
Yeah, yeah, I can, you’re making that connection with the personal integrity to like, it’s just part of how you learn, right? You don’t want to cheat, you want to be a learner that is that has integrity. And that can carry that into the rest of the world, not just in college studies, or, you know, in the training or something like that, that that’s part of who the student is. And we often I feel like in our institutions, that academic integrity is, like you said, a set of rules. And it’s about, okay, well, you have to cite things this way. And if it’s like this, you cite it this way. Or you have to make sure you follow these particular rules, and you note these particular things. And if you don’t, then you’re cheating, or you are, you are failing, you are, you know, not doing what you’re supposed to do. And it it comes down to a lot of rules, we often have to put on our syllabus.
Mark Glynn 08:39
And that’s where I see there’s different unpeople that only introduce or only view academic integrity at a college level, I think miss out on that, because for me, it’s something that should come in from the very, very young level, not just citation and referencing, but it’s the ethical approach of doing things, right. Because it’s properly proper to do them. Right. And it’s just like UDL really, to be honest, it’s something we all should just do.
Lillian Nave 09:07
Yes, yes. And and I think we, I wanted to have this conversation, like when we talked preliminarily, I was really excited that this would be our conversation, because we could talk you and I could talk about a whole bunch of things about Universal Design for Learning. And, and that could be a series of episodes, but for this particular, you know, chance, I wanted to talk about that academic integrity, because of, first of all, a lot of changes recently and, and what that means for academic integrity, but also because I don’t think we talk about it enough in relationship to Universal Design for Learning. And we haven’t really defined it well or maybe understand it well. And it might be part of that hidden curriculum that we don’t because we don’t explain it to our students well and what it means in our context then students are tripped up, it becomes an artificial barrier, for example, different cultures are going to view academic integrity differently. So some students may come to a university, and they think they have to do everything by themselves, they can’t talk to a friend, they can’t seek a tutor. That would somehow be cheating, you know, this is kind of an extreme case, but that they, they have to do everything as an individual. Otherwise, they’re not learning or they’re not smart enough, or they maybe they don’t belong there. And I don’t agree with that, but, but there are others who might come from a place where we do all this together, we have, you know, we work together, we talk to each other, when we’re writing the papers, we have a peer review, we critique each other, where there’s a lot of kind of group learning social constructivism, right, where people are really helping each other learn. And yes, you might need to note, hey, I spoke with these two classmates, when I was formulating this paper, you know, I want to recognize their work in my thought process. But that’s not cheating, right? That’s not against the rules. And we just we haven’t really specified I think a lot of times we haven’t specified what is academic integrity, and what it might mean, even in your particular class, or your institution, or your educational, you know, experience, you know what it means. And that’s, I really have liked what you’ve been doing around that lately. And that’s, I wanted to ask you, yeah,
Mark Glynn 11:37
so for me, we have such similar views on this. Like it is about unethical behavior. First and foremost, today’s about whether it’s collusion or collaboration, you know, the it’s that which one is it? Which one does it fall into? And as you rightly said, there are cultural differences that come into it, whether it’s, we have what some international students think it is, it’s actually disrespectful if I don’t give the views of the experts in the field. And they literally copy and pasted the views of the experts in the field. Whereas older cultures, and older people would say, well, actually, it’s more appropriate to interpret their views and reflect on their views and give your own interpretation of it. Or some people, like you said, could have the support to get an extra tutor extra tuition. Maybe it’s not putting them at a disadvantage, or sorry, an advantage over other people who don’t have the money to get the extra tuition, to get that extra support beyond the people who are well off enough to, to have that or be in a position to do it. Are they cheating? Well, no, they’re just doing what everybody does and start helping themselves helping the children get to that next level. But where’s that? Where’s that? borderline between cheating and actually using all the resources at your disposal?
Lillian Nave 13:00
Right, right. And that’s what I love getting out into the open and and making sure that, that instructors, people who are in charge of this, you know, environment, educational environment, are making that known to to their students. So we’ve got we just have a clear understanding, and everybody’s on the same page. So it’s true
Mark Glynn 13:22
to exactly that. It’s to stop people thinking, this is how to get me this is how to punish me. It’s actually how can we do this properly? And and when you have the conversation, and the transparency of the conversation around to academic integrity, it becomes the norm. And that’s what you want. It’s not that we’re trying to cut you out, or trying to help you learn.
Lillian Nave 13:44
Absolutely, yes. Alright. So these, one of the biggest parts then of academic integrity are assessments. And I’m interested in your work and how your work with UDL has influenced your views on assessments.
Mark Glynn 14:03
I’ll preface this that I said in the introduction I learned from others. I am literally standing on the shoulders of giants because I’m I’m a lot of the work here that we’ll be chatting about today. It’s definitely being a team based work so that the people that I work with have collated. I have a wonderful team within the teaching enhancement unit. So reporting on behalf of those I think is the best way forward. But I in the teaching enhancement, you know, we wanted to tackle assessment and academic integrity with a proactive approach. Rather than trying to catch them after they’ve cheated. We wanted to catch them before they did. And we wanted to do that true assessment design. So working on the basis that prevention is better than cure. We wanted to look see how can we design assessments How can we as an institution behave or how can individual lecturers behave, what approaches they can take to reduce the opportunity for, for plagiarism, to reduce the rationalization for plagiarism and reduce the pressure on plagiarism. Those three points are actually from a thesis theory called the Cressy fraud triangle. And it comes from from the world of banking, but I would apply it to the world of, of education, because fraud, and plagiarism to me are one of the same thing, fraud and cheating. So, what we’ve decided to do is to look at the literature and see what’s out there in terms of proactively eliminating plagiarism. And we found seven different principles when we collated all of these different papers that we pulled on assessment design, we found seven different principles, different ways that we can go about proactively reducing plagiarism. And we took those principles and then circulated it to my counterparts across Ireland, and indeed, well, anywhere grad that I had networks I was pushing it out to. And I said, we found seven, what are we missing. And we got a variety of different responses back from actually several 10s, if not even 20 responses back from different institutions. And we were able to combine all of them together, and condense them down to 12 principles of academic integrity assessment design principles, and I’ll circulate them to you for the podcast. But firstly, the principles again, these 12 can be broken down into three distinct areas. So what an institution can do, what a lecture can do. And then the student partnership or student ownership. And again, if I remind you of the Cressi, fraud triangle, each one of them pressure, opportunity and rationalization, we want to look at how those 12 principles can be used to tackle each one of those points of the triangle. And essentially, the smaller we can make that triangle, less chance people have of of committing plagiarism.
Lillian Nave 17:16
Wow, it is fantastic. Can you tell me more about those institutions in what the institution can do, what lecturers can do and what students ownership brings to it? And yes, I will have these in the resources on our webpage for our listeners, Okay,
Mark Glynn 17:32
super. So that what the institution can do is, well, obviously, you can set high standards, you can set the values, you have an academic integrity policy, and you make sure that it’s consistent across all the different programs that you don’t have one approach in the School of Law and a completely different approach. In the School of Chemistry. You want to make sure that the students has all that detailed information from day one, about breaches of academic integrity, and how they might avoid them, and all that sort of stuff. And essentially, like academic integrity is what we build our reputation on. So an institution cannot risk not having academic integrity embedded in in what we do. So we must take steps to make sure, for example, that we update and edit our assessments and our our program assessment strategies on a regular basis. Because if for example, and I genuinely think this is is unethical, of a lecture to make an assignment and keep that for the next 1020 years, just not edited at all, a lady called Sarah Eaton, who’s from your neck of the woods, gave a fascinating presentation. And she was the person actually who gave me that phrase on an ethical behavior from lectures and doing that. So there’s different processes that we can put in place as a university regular assessment reviews, putting the academic policies in place, putting the supports in place to enhance academic integrity and having that transparency about academic integrity and having that positive, proactive approach rather than a reactive punitive approach. And from a lecturers point of view, there’s so many things they could do. Ensure obviously I talked about regularly updating your your assessments will ensure they’re authentic, ensure the current under relevant, give students scaffolding, and again, this is all relevant to UDL. If you scaffold an assessment, maybe for I can think it’s a it’s an end of semester essay they must do. Well give them key points and milestones throughout the semester as to what they must reach. Rather than leaving all the pressure to the last minute. Give them a rubric to show them what They must do an audit where the marking criteria is, is spread out and what’s getting more marks versus less marks, what you expect of your students. And all these little things can be done. And they are just little things. And this is where I love the approach of Tom Tobin, the One Small Change, there’s loads of these little things that we can change in each one of our assessments. And it will evolve over time. And that’s all we ask people to do is to look at their assessments, look at these 12 principles and say, right, how many of those Am I able to, for want of a better expression tick the box against or how many of those I’m able to address this year, and what will I work on for doing next year. And that would be from a lecture point of view, and then looking at from a student ownership point of view. For me, I would see it as trying to reduce and this is where student ownership comes in is trying to reduce the rationalization element of the Crafty fraud triangle, the rationalization like show them tell them how to cite properly, tell them how to reference properly, gives them the option of choices within the assignment, again, UDL connotations in there with the choices, but give them the element of choice because they can then personalize the assessment. And they can say it wasn’t relevant to me. But also, it’s harder for them to use the likes of contract cheating services, because you’re asking them to do something personal, and maybe related to other course material are related to their own experience outside. So all of these things, all of these elements of the these 12 principles do have direct overlap with universal design, as well as improving academic integrity. So going back to the tick box analogy, you’re taking two boxes at once, by being good and ethical behavior.
Lillian Nave 21:52
Right, right. And those so many things that you have said, make my eyes light up with UDL principles. And one of the continuous I think ideas that are in each of these, the institution, the lectures and the students, is this transparency, and that the clarity that we need to give our students from the institutional level, and from our just our classes, like from the lectures or instructors, and even the students being able to understand what is is needed, what they have to do and things like a rubric, or I love providing annotated examples, like here’s a great essay, right? And look at what happened in the first paragraph, it laid out the points well, and here’s the citation that was used. So they have really good examples. And that’s not cheating, either. Right? Some, some instructors, I have heard will say But wait, you know, we want them to figure it out. But providing an example is, is helping them to figure it out. And they still have to do all the work. We’re just, I think taking away a barrier of how am I supposed to start this? I’ve never done this before. I don’t even know what it looks like how would you know how would a presentation to accompany look like? I would have no clue because I’ve never done it before?
Mark Glynn 23:21
And don’t exactly you’re taking away the fear that students may have? Yeah, the rationalization that they’d have where, oh, well, I’ve never done it before. So of course, I’m going to copy off Lillian, because she’s going to load your time. Or the pressure that you’re under, when I have this assignment, I know nothing about it, I need help from somebody else. So I’m going to ask Lillian to help me with this. And, and again, like we need to get into into my head and into your head as students and just sort of think, well, it’s done. Okay, how much is it collaboration versus collusion? The point that you brought up earlier on. So for me, if we are transparent if we give them sample answers, and as you say annotations to previous answers, you’ve shown them what you’re expecting from them, you could also redesign the assessment, where there’s an element of peer review in this beforehand. So before it even gets to you as a lecture, some of the peers have looked at it, and you could look and say, Oh, I see what Johnny’s done with his I’m gonna actually do that with mine. I really liked the way he structured that. Both your grading criteria will change accordingly then as well, because you’re you know that they’ve seen these examples beforehand. So there’s a whole load of different things where we can help people designed or assessments better to accommodate more learners and better accommodate more learners.
Lillian Nave 24:50
Yeah, I had a really fantastic result with an assignment I worked on a couple years ago, in which there was a peer review within that session. assignment was an annotated bibliography, which is using citations and very much one of those kind of academic integrity assignments. And I found that students were not, weren’t doing well, well, the first time I signed it. And so I realized that I needed to clarify what I was asking because I was not getting, you know, the result that we needed to get right that they were supposed to be finding peer reviewed scholarly journals, and, and then analyzing if it would be a good source or not. And I thought, Okay, well, I need to revamp this assignment and included a day where they brought in their assignment. And then we worked as a workshop. And they passed around their annotated bibliography to three other people. And we had like a rubric in front of us about what is what does it mean to be timed, timely? Is it authoritative, you know, what about the source? And they were able to, you know, to see by looking at one other students, they’ll say, Oh, I didn’t even think about this. And if they see another student, right, oh, I didn’t do that in mind. And I said, That’s okay. If by the end of the of the class, you get yours back and you say what you would change and what you would do right the next time, then you’ve learned something, right? You’re much better.
Mark Glynn 26:23
Let’s flip that on its head to where we’re thinking of the student who has prepared a good annotated bibliography. He or she then can sort of puff out their chest, and they’re thinking, Yeah, I did, well, I’m seeing this. And so it gives them confidence as well in their own learning. So it’s a win win situation, as far as I’m concerned, and it’s, it’s not rocket science, because if you cast your your mind back to primary, or what we what you would call Elementary School, Teacher, we did that we passed her homework sheets around the classroom as we were kids, so all we’re doing is just the high school that that that college to university version of that, we’re just taking it upper level.
Lillian Nave 27:03
Yeah, and we’re learning from our peers, and we’re learning, you know, learning more ourselves too, as we do it. You know, anytime you have to teach a lesson, you learn so much more than if you’re just, you know, getting that lesson from somebody else. So anytime you got to say, I looks like this part, I would, you know, change or it needs some some help here, I couldn’t find this, then it’s a, it’s a different level of learning I found and students really have agents,
Mark Glynn 27:33
yeah. And that’s just agencies to keywords, you’re giving them agency into their work. So that is going back to the Cressi fraud triangle reducing the rationalization because they’re, they’re proud of our work, they know how to do what they need to do. So they’re going to put the time and effort into it. And if they see you, and going back to your example of within the class, having that workshop to flesh it, I mean, you’re spending time and effort on their assignment help promote, so they will feel I’m gonna put time and effort into this assignment. Whereas if you’re the lecture that just gives out the same assignment time and time again, and gives out scantly feedback or no feedback at all. Students won’t care. They’re just sort of, I’ll move on to the next one.
Lillian Nave 28:15
Yeah, right. And, and thinking about assignments, if if you’ve got a an instructor lecturer who’s given the same assignment to the same class of 50, or 100, for the last five years, you better believe there’s examples online, somewhere, right, 100%.
Mark Glynn 28:30
And now, now, more than ever, it’s easier to find stuff online. And for me, one of the challenges that we have and don’t get me wrong, I’m possibly one of the biggest kids should know even though we barely know one another, I love the internet and all the different technologies that are available. And when I go on Google and I search for something, all of a sudden, the next ads that I’m looking at, and the next website is relevant to my previous search. Brilliant, of course, yes. However, put yourself in the students shoes, they’re getting ads, based on their profile based on their searches. They’re getting ads from contract cheating sites. They’re getting ads from all these services that we used to be able to detect years ago because they’d have to come in and stick a notice on the student board. And we’d rip it down straightaway. As soon as we’d see us. Well, now, that’s gone underground. And we’re not seeing that at all. Which comes back to my point about wanting to have the academic integrity conversation out in the open. And let us have it and do everything we can to proactively reduce the amount of plagiarism that goes on.
Lillian Nave 29:33
Yeah, and you know, I think about that to carrying forth into their lives, like wanting to be people with integrity, and then you know, you go into your whatever your job or your life, whatever it takes you that you have that integrity as well. And I think about the the flip side, like what if students are saying, well, I’m never going to use this right? I don’t it doesn’t matter if I cheat because I’m you know, I’m never gonna do this and that’s where that authentic part comes in, right? That needs to be authentic, because I would be more likely to kind of put low effort into something, if I think well, I just need to hand this in to pass this class. And I’m never going to do this, you know, I’ll never need to know this term exactly,
Mark Glynn 30:15
I show where there’s authenticity in their worth or personalization in there. Because remember, and if you take part time, student, for example, there, they have an awful lot of pressure outside, they have their day job going on, or they have family they need to look after, and then they’d come to study. So if you give them a useless assignment, that means nothing to them in their day job, or in their day life, there’ll be like, Oh, just let me get it and get it over and done. Whereas if you give them something that’s relevant to their day job, they’ll actually do a really good job of it, and then want to improve further their job as well as for college. And you’ve built that intrinsic motivation, you’ve tapped into that motivation for students to do better. And whereas the passing a true delight of the text matching services, that’s the extrinsic motivation, you know, you wanted to get better results, go for intrinsic every time, make it authentic, make it something they can have a passion about. And you’ll get brilliant assignment products for want of a better term, which again, you could learn from as an educator.
Lillian Nave 31:27
Yeah, and you’ve just described to that whole first column, the green column about engagement and motivation. And the Universal Design for Learning principles is, is creating that, you know, bringing in interest and motivating students sustaining them throughout the process of that assignment, you know, how is it worthwhile? How is it valuable, and helping them to see that value to complete it? With Integrity? It’s all I see it very linked, as you do as a link to our Universal Design for Learning principles.
Mark Glynn 31:59
I can I look at the principles and I can nearly draw direct lines between them and under 12 principles that we have, by no means are we saying that the 12 principles are, are to the standards of the cost guidelines. But we can definitely see the relevance. And it’s it’s taking the two boxes at once. It’s helping us design a better learning experience for for students. Again, Transparency is key. Personal choice is key. giving them information upfront is key, all of this UDL and academic integrity, and we even view it on our language is is so important to the language that we use when we’re saying improving academic integrity, not catching plagiarism. You know, it’s not not catching cheating. Yeah. And it’s how to make it, how to make it better how to make the learning experience better. And that’s what UDL does, as well. So,
Lillian Nave 32:55
yeah, yes, absolutely. And that word transparency we’ve been using a lot, I think, is key, if it makes me think of, I’ll add this to our resources, the tilt, transparency, and learning and teaching by Marian Winckelmann, I need to get her on the podcast, too. But that that goes along with the idea of the purpose. There’s three parts that purpose, task and criteria. So letting your students know what the purpose of this assignment is, like you’re doing this so that you can be a better nurse one day, or you can solve a problem in your economics job, right? And then the task lays out exactly what they what a student needs to do to be successful. And the criteria are the things that we’ve just talked about a rubric, or an annotated example of a student’s work, you know, or something, something like that. And that that transparency is, I think, absolutely essential, and something somehow we haven’t been doing forever. Like, why didn’t we do
Mark Glynn 33:56
this? Yeah, it just went when you think of it, it actually makes common sense. You know, and even when you think about making documents, but making them accessible, since I’ve learned how to do that, and it only took somebody to show me really a difference that will make but since I’ve learned to do that, I’ve actually became better at my job. I’ve made better documents, I’ve given better presentations, because I’ve made stuff. I’ve used the accessible tools within within the documents. And what I want to do is integrate that into the assessments. So when the students are actually compiling the Word document or compiling the PowerPoint presentation, or whatever it is they’re doing, as part of it, that we embed UDL into the assessment as well, that we showed them that they must have all texts and why they should have we showed them that they should use Heading One, Heading Two, Heading Three. And again, not just for screen readers, but it’ll generate your table of contents very easily, you know, simple stuff. It just it makes sense. That’s what gets me, you know?
Lillian Nave 35:03
Absolutely it does all of this makes perfect sense to me. Now, I don’t know why I didn’t know it, you know, before I had been introduced to Universal Design for Learning. But now it makes perfect sense.
Mark Glynn 35:16
And it’s the same with the academic integrity principles, there is nothing majorly scientific or heavy about the 12 principles, you’d nearly say it’s the 12 principles that are bloody obvious. You know, when you’re when you’re looking down truest. It works, you know. And what we want to be able to do is to not put where we talk, from a UDL point, not put barriers in front of students, we actually don’t want to put barriers in front of lectures when it comes to assessment design. Yeah, which is why we made it as simple as we did, and drilled it down to, to the 12, the 12 principles. One of the other things that you mentioned transparency, but one of the other things is connection, is what I think is really useful. And it’s the connection between assessments. So why do I point that out, it allows people to connect between modules, because far too often, the chemistry teacher teaches chemistry and doesn’t care about biology. And the biology teacher teaches biology and doesn’t care about what’s going on in the physics class, and so on. But the student has the overall learning experience of the science course. So we need to connect those modules. And connecting those modules helps people not just people with learning disabilities, but helps all students connect, start learning. And even if we do it a simple by connecting our assessments, that the assessment from module one feeds into the assessment for module two, maybe, for example, you mentioned doing an annotated bibliography. So that could be the first assessment. And then the second assessment is taking the topic that you did the annotated bibliography on, write a paper on or do a presentation on. And then your your third assessment, maybe in a different module is about something else that maybe you have to now write a proposal or a solution to a challenge that has came up from the area that you’re researching. You could you mentioned rubrics earlier on, you could have rubrics that within the rubric for the second assignment, it says, What have you done with the feedback from the first assignment to make this second assignment better? So simple little things like that structure, a scaffold, and there’s your UDL and your assessment design, put into one
Lillian Nave 37:32
Exactly. Right, right. And, and we often can see those connections as lectures or instructors, we know that the assignment for Module One leads into the assignment for Module Two, but we might not have made that really clear to our learners,
Mark Glynn 37:49
same percentage of time we didn’t make it, we assume a connection is made on a course the students see it, but in actual fact, they don’t.
Lillian Nave 37:58
Yeah, they’re so flooded with this new information. And there they are. Really, amateur learners, you know, new in that field, oftentimes, and don’t see that, oh, this is building towards this regular product that we always see in this field, the big because they’re just not used to the regularity, or they don’t know what’s coming. And I often think about a.to.id type of drawing, that we look at it and it looks like a sea of dots that don’t make sense. But until you follow and connect those dots with all of the numbers, you realize, oh, it’s a cat or it’s a giraffe, right? Or something like that. It makes sense at the end, but our students are just going from one to the other, not completely confused, not knowing what’s going on. So if we can situate it like, here’s our first assessment, it’s the first step in a large research process that you will be doing all semester, it fits in here, you need to do it. So you can figure out who is a prominent voice in this area, you’re going to need to find, you know, a philosopher that is going to be able to speak on the topic that you’re going to get to you know, and they don’t realize that you have it all planned. But we have to again, make that transparent and make that connection transparent to our students. And I am guilty of that too.
Mark Glynn 39:18
I’m me too. Absolutely. I’m not pretending to be perfect by any stretch of the imagination. But I used a funny you used to dot dot connected dots have never, never made excuse me that for a poem but never made that connection myself. I’ve, I’ve used the the example of where when, you know, look at those optical illusions. You know, when you look at the image, you see a lady and when I look at the image, I see a doc and now we see completely different things. But then once you point out where the lady is, then I don’t see the duck anymore. All I see is the lady so that’s where what I was trying to explain the same point two lectures I said, You think everything is perfect and you think everything is clear. What’s your students don’t see that you’re stupid. See something else, and trying to put yourself into the shoes of the student. And what I think is a really good way to do that. And going back to scaffolding and being transparent with it, is get a student from last year to talk about the assessment from last year. So talk about, that’s a great idea. And from this year, they’re hearing it in the students voice, though, like because when I went to one the chemist by training way back when I couldn’t even tell you what color white lab coat is. Now, let me tell you both. So but when I was doing my, my masters and did various different certificates, teaching a learning reflection is key to what you do. When I was a chemist reflection was looking in the mirror, you know that that’s what I taught a section was. So for me, if I when I got or if a chemistry former chemistry graduate, came to me and said, Oh, well, I knew that was our I thought it was looking a mirror. But now this is what it is now, well, I could I can resonate with that person. That would make my learning that little bit easier. So if I got a student from the previous year to talk me through their assessment, and it could be something as simple as recorded video that a lecturer puts up, yeah. And displays to the students in the same way, you would get the lecture to put up sample answers. Get them to put up a couple of just students chatting about the assessment, how they get on last year, what they talked about at first, and what they think about it now, again, trying to make it more authentic for them, get get alumni members get graduates to say, God, I remember Macklin waffling here, but now I realized it was the best subjects because I’m out in the world. And that’s what I do all the time. You know?
Lillian Nave 41:45
Yeah. And also our students are going to know their process much better than we would I had students, for example, I had students come back to talk to my friend, first year students about the class and the things that they said for advice was not going to be the things I would say, Well, you know, yeah, things like, You got to make sure you get to the, you know, get the library portion done early, because you don’t have enough time because it comes at this point in the semester, when you’ve got all these other things that of course, I didn’t know were happening. But the students were were the experts, at least and in that, and I was always surprised, like, oh, I never would have thought of that. During the semester. Give me
Mark Glynn 42:30
a give me a funny example, when I was teaching chemistry I was, I got the students to find a YouTube video. And I said, Find any YouTube video that is related to some content on our course, on the syllabus. And the deal was that every student had to post the video on the discussion forum, within our VLE. And you couldn’t put up a video that somebody else has put up already. Right. So they had the flexibility of finding this video at their leisure. So there was a UDL element there, they could personalize it a video that meant something to them that want to relate against easy experiment and the easy assessment. The students were looking for a catch, you know, it was that easy. Yeah. But what happened was, I said, and give me a couple of sentences about why you think this video is good. And I got there is over 80 students, so I got over 80 different videos come back to me, of which 50 or more actually really good videos. And that’s probably being a bit optimistic about but I asked them the students to rate everybody else’s video, I said, just just give me your top five, give it a score out of five. And the videos that I taught were absolute childish tripe, the students loved, I did a video on I still remember, this is 20 years ago, and the students found a video about how to name organic chemistry compounds. And so I thought, right, this is very relevant to our syllabus. But it was like a Sesame Street song or the you know, the kids TV problem. But yeah, it’s like Sesame Street song on how to do the names for us. And I’m not gonna sing it now. So don’t worry about it. But I, if I found that trying to research for my class, I would have ignored it. And so that’s a way to challenge yeah, that got the top rating in the class. Yeah, because they taught it to help me remember how to do it. So that I learned a very valuable lesson there that is always try bring the students and and in this case, it was a students as partners now what’s known as I didn’t know what it was called at the time, but students with partners and assessment rather than doing stuff to the students do stuff with the students. And that opened my eyes, you know?
Lillian Nave 44:48
Yeah. And that’s it. That is a very authentic type of collaboration to Yeah, and brings ownership to the
Mark Glynn 44:57
unfolding because the students who replied because obviously they had all of YouTube to search from whereas the students that went second had all the YouTube minus to one video. And it goes student that went at it had to sift through all the others. What happens to things really interesting happened? One, the students that did it the first couple, when they saw other videos posted by the colonics. They said, Oh, can I do it again? So like, How many students do you know what else to do? And assessment a second time? You know, I’m just wow. The second thing was on the students said to me years later, he said, I was one of the last students to submit the video. But what happened was, every time I went onto YouTube, because I’d watch 70 Odd other chemistry videos, they kept on getting recommendations for chemistry videos. So you know, yes, I was teaching them chemistry subversively true the power of Google, you know?
Lillian Nave 45:50
Yes, right. Oh, what a? What a great addition. I love it. So well, we’re in a strange time now. And I’m actually going to ask the next two questions kind of combined into one about how you think COVID-19 And the pandemic has changed the assessment landscape and kind of in additionally, how do UDL and academic integrity figure into this change in the assessment lens,
Mark Glynn 46:15
so So for me, and not for one second to go on to paint COVID in a positive light, it’s been a horrible pandemic for everybody worldwide. However, there’s a silver lining to it. And I think the silver lining, in this case, his assessment, and how it’s changed assessment, when we had the height of the pandemic, and we had a lockdown. And for far too many months than I care to remember. But when we had the lockdown, all of our assessments had to change all of our traditional exam hall assessments, which are two hours and we cram however many 100 students into this sweaty exam hall to try do these exams. That to me is on authentic assessments, just that’s called a call it what it is. But that had to stop, we had to come up with a different way of assessing, we had to think about the traditional assessments that we’ve done for aeons of putting people into these examples, under stressful situations, to okay, how am I going to assess it? How am I going to change it? And as much as there were people that had concerns about academic integrity, and they tried to introduce the software that would proctor them remotely, which I think yes, nightmare and totally unaccessible won’t even go down that road. But as much as a good portion of people try to do that and try to essentially mimic what they were doing in an exam hall in an online environment. Loads of people took the better approach of actually redesigning their assessment. And what I’d like to think and I don’t have to figures, the exact figures off the top my head. But I’d like to think that the large majority of those that made the changes through assessment have kept their changes. Yes. And that to me, is progress. That to me is improving it and yes, you will have your detractors, and you will have the people saying, Oh, well, how do you know it’s them? That’s doing this. Like, to me, you’re always going to get that no matter what assessments you do. And and if you’re taking that approach, when he wouldn’t have a continuous assessments at all, everything would be in an exam hall scenario. what frustrated me was where the professional bodies are coming to us, telling us how to assess I know you must do it in an exam hall, because if not, we want to credit your program. They were as we should have been driven by the pedagogy, and what’s best for the student learning experience. And, and without a shadow of a doubt. What’s best is authentic assessment. And for the large portion of modules. Authentic assessment is not a two hour written exam. I can’t remember the last time I wrote for two hours, ever. Yeah, no.
Lillian Nave 49:05
You mean it’s not an everyday occurrence in your adult life,
Mark Glynn 49:07
but like my daughter would come back now. She’s long since graduated. Now when she came back, she in her hands were sore from writing.
Lillian Nave 49:15
Yeah. Oh, I remember that. Yeah, links used
Mark Glynn 49:17
to type when she’d be typing probably quicker and better than she’d be be writing that literally. And it was more more difficult for a lecturer to interpret her writing. Whereas if it’s typed, easy to do, spellcheck can be done. Yeah, formatting can be done. It’s just a nicer experience for the student and the staff member. So yeah, again, it’s a no brainer. I hope those those assessment practices continued. That’s, that’s that would be my silver lining to the cloud studies. COVID-19.
Lillian Nave 49:49
Yeah, I cannot remember where I heard this, but I heard the term COVID dividend, and there are certain COVID dividends that came like spending a little more time with my family, right? Because we were all at home maybe too much time. But that was a COVID dividend. And I think in academia, this assessment, rebirth really, of authentic assessments and multiplicity of different kinds of assessments, um, has been a dividend that has come from the the COVID 19 pandemic.
Mark Glynn 50:22
And I think what happens with assessment is we get comfortable, and we assess the way we were assessed, and there are electrodes were assessed the way they were assessed, and they think it is the right thing to do. And we don’t think of the long term learning attends to be surface learning that happens then, because students cram and students just learn off what needs to be done. Where are they going to learn off? Where they’re going to learn after model answers, where are they getting the model answers to get them from somewhere else? So that’s academic integrity problems, then that we have. It’s not they’re tall. So what they’re doing, they’re regurgitating somebody else’s content, even though they’re doing it in an exam hall scenario, when we know it’s them. It’s just you’re testing whether they have good memory or not. You’re not testing the content.
Lillian Nave 51:08
Right? And will they remember it? Five years later? Actually, probably not.
Mark Glynn 51:12
There’s a gentleman called Phil race over in the UK. And I don’t use that term lightly. He is an absolute gentleman. And he has many an article about the authentic assessment and the how the exam hall is the wrong approach. And we need to be better, we need to do better.
Lillian Nave 51:30
And I’ll have to include that in our resources as well.
Mark Glynn 51:33
Yeah. All little all the stuff fill ratios is brilliant. Okay,
Lillian Nave 51:38
great. So, um, we, I think you have laid out a very good case about why we should be creating authentic assessments and why it is an ethical imperative to, and how universal design for learning figures into it. So if someone is listening now, and is inspired to change around their assessments, what advice do you have for someone who is interested in creating an authentic UDL guided assessment in their higher ed courses, or wherever they’re needing to assess learners? And they just don’t know where to start?
Mark Glynn 52:19
Where to answer that question. For me, oh, one small change. Again, going back to the Tom Tobin thing, don’t be afraid to change. Like an actual fact, by standing still, you’re moving backwards with that expression. I can’t remember who said that. But for me, you need to change and you need to evolve. So just to one one small bit, maybe give the rubric to students. Maybe and here’s a, here’s a what could be a game changer. give the students the similarity report, a text matching service report, give it to them before the deadline, allow them to see us before the deadline, because that want to know them learn from us and not be punished by us. You know, I don’t know, allow them to submit drafts, or allow them. Again, a lovely one, from Sarah Eaton. Again, the lady I mentioned earlier on, she said, be kind. And and she said there was a policy expression she had, it was a get out of jail card. Right. And I loved that expression to get into jail cards for the assessments where if a student is under pressure and isn’t going to meet the deadline, allow them one get out of jail card allowed them to be late, say no questions asked. Just 11th I thought I was a lovely example. And each one of those changes. So give them the rubric in advance, give them the text matching in advance, give them a get out of jail card. That’s all in the command of each individual lecture to do 100% Absolutely no doubt about it my mount. So they are three quick examples of one small change.
Lillian Nave 54:05
Yeah, you’re, you’re making me think I need to apply that Cressy fraud triangle to my assessments and find out if I am pressuring students into rationalization and kind of look at what my parameters are. If I only give them two days to do this really long assignment. I I’m almost pushing them into doing a poor job, right? If it takes a lot longer. And in thinking about how I can rethink from my students perspective, what that might look like to them. And I hadn’t thought about the ways that we sometimes as instructors are pressuring our students to cheat because we’re not either providing the time or the scaffolding or, or the clarity, I would say for what Students, we
Mark Glynn 55:00
need to look at it for far too often we look at our module, right and have the expression that’s my module, but not exist yet program. And the student has several different modules. So where you may think in your module of three assessments, that’s exactly what’s needed. Actually, no, I put a fourth in because they learned so much better when I give them this extra assessment. Well, if every lecture did that, and all of a sudden the student has 24 assessments or 25 assessments to do in a 12 week time period. Yeah. And more than likely, it’s the same deadlines. We have this thing called week seven syndrome, where week seven within the semester, all the assessments start. And you’re thinking if lecturers start to talk to one another, and say, Well, God forbid, where if they start talking to one another, and plan their assessments? And going back to what I said earlier on connecting thorough assessments, that that would really, really help.
Lillian Nave 55:57
Yeah, absolutely. The This has given us a lot to think about. And I know our resource page is going to be full of the resources you’ve told me about. And those are the 12 principles on assessment design. That’s going to be really helpful. I’m really excited to put all this together
Mark Glynn 56:17
delighted to share it with you. And as he said, it’s really it was a team effort as my colleagues, I won’t name every single one of them said, but they I literally have stood on the shoulders of giants in this respect. So be delighted to share all of the content that we have.
Lillian Nave 56:33
Well look at you modeling academic integrity, right. Yeah. Just fantastic.
Mark Glynn 56:38
I just know, I’ve been slated after they listen to it, if I don’t.
Lillian Nave 56:42
That’s right. That’s right. Oh, that’s perfect. Well, thank you so much. I really appreciate the time you’ve given me and all of this really great information that helps us to be better instructors and lectures. And I just can’t thank you enough for the work you’ve been doing for a long time, and I’m really glad I got a chance to talk to you.
Mark Glynn 57:01
It was absolutely my pleasure. Thank you very much pleasure and privilege.
Lillian Nave 57:16
You can follow the think UDL podcast on Facebook, Twitter and Instagram to find out when new episodes will be released, and also see transcripts and additional materials at the think udl.org website. The think UDL podcast is made possible by college star. The star stands for supporting transition, access and retention in post secondary settings, and the website provides free resources and instructional aides based on UDL principles. If you’d like to know more, go to the college star.org website. Additional support for the podcast is made possible by Appalachian State University where if you call it Appalachian, I’ll throw in Appalachia. The music on the podcast was performed by the Odyssey quartet comprised of Rex Shepherd, David Pate, Bill Folwell and Jose Cochez, our sound engineer is Tanner Jones and I am your host, Lillian Nave. Thank you for joining us on The think UDL podcast.