Feedback

We have already touched upon the subject of feedback, and its peculiar and often subjective qualities and content.

Today we resume the topic of feedback and challenge you to an exercise in feedback.

We know that good feedback is always contextual, and is often tricky. Some of the trickiness is in the object of the feedback. Usually, it’s easy to give your opinion and it’s hard to refraining from sounding judgmental. “Your essay was great!” or “The paper is poorly written” are examples of such. These don’t provide the student with enough context and a direction for improvement, or in the former case, don’t point out what exactly the student did right so she can carry it over to her other assignments.

The Judge, the Transparent Reader, and the Transparent Reader Plus Advice Giver

Barbara Walvoord and Virginia Anderson in Effective Grading bring up a concept of the “transparent reader”. I haven’t been able to find any other literature where this term is used, but it appears fairly self-explanatory and, well, transparent!

Walvoord and Anderson propose three categories of feedback givers – the Judge (“Your essay was great!”), the Transparent Reader (“I was confused about the position you are taking on this issue”) and the Transparent Reader plus Advice Giver (“Your arguments in the second paragraph contradict each other. You may want to reconsider the selection of arguments or present which ones you consider valid and support your position”). It’s noted that even in the advice giver role, the instructor offers at least a couple of suggestions so that the student still needs to choose from the examples or come up with a solution of their own. The goal of the advice giver is not to “fix the paper” but rather “to coach the writer”.

Walvoord and Anderson encourage becoming the Transparent Reader, and giving feedback from that position.

Coach the Writer

Keeping in mind the purpose of the feedback, which is to help the student produce a better paper, the instructor will tailor her message to communicate priorities and suggest ways to improve.

Feedback needs to be finely balanced in that it should both be as concrete and tangible as possible – “In the second paragraph, you make a claim but don’t provide any arguments or evidence to support it” – and reflective – “Your choice of words takes away from the message so that I as reader was confused about your position on this issue. How can you articulate your message better given your intended audience?”

Adjectives

A lot of times we are encouraged to use adjectives to better describe our experiences, feelings etc. There is an opinion that when used in feedback, adjectives tend to gravitate to the judge feedback, and are easy to move focus away from the assignment and to the student. Using verbs and descriptive statements about what was or wasn’t done may be more productive and make feedback more effective.

This is the exercise part: Next time you give feedback to students, make it a must not to use any adjectives. Let us know how that goes for you and for your students! We would love to hear!

Timing

Of course, the timing of the feedback is also essential. If it’s given to students at the end of the class, chances are there will be no follow up for them – just like there is no resubmitting the final paper at that point! Building feedback into the writing process and commenting on a draft are going to give students both opportunity and time to work on their submissions. We will be offering a webinar with our ideas about it and invite your input and feedback, so stay tuned for more information!

Peer Feedback

In addition – or in lieu of instructor feedback in certain cases – peer feedback can serve a similar constructive role. The instructor role will be amended in this case, as it is now necessary to either provide scaffolding for peers reviewing each other’s papers or to coach them and provide examples – and also challenge them to practice being the “transparent reader” and giving feedback relying mostly on verbs and not adjectives!

Reference:

Walvoord, B. & Anderson, V. (2010). Effective grading : a tool for learning and assessment in college. San Francisco, CA: Jossey-Bass.

Advertisements

Randomize Group Enrollments

Instructors often like to have control over student group assignment. Sometimes, it matters not who is where, but a rather quick assignment is needed. While there is absolutely no way to change the type of the group set already created (at present), it’s possible to have the system randomly assign all the students (more or less evenly) across all the groups.

If you would like to know how to do so, here are two simple steps:

1. On the groups page (make sure your Edit Mode is switched on), mouse over the group set line and select “Edit Group Set Membership” from the drop-down menu:

Groups Page

2. Use “Randomize Enrollments” to fill in all the groups at once. You will have a chance to go through the users and remove extra roles (assistants, teaching assistants, leaders, instructors etc.)Edit_Group_Set_Enrollments_–_New_Student_Orientation

Gamification – An Implementation of Immersive Role Playing

Former colleagues of mine, Robert Prince and Owen Guthrie of the University of Alaska Fairbanks, developed a journalism course wherein the professor constructed the lesson plan to mimic the internal organization of a struggling newspaper. The students in the course were new hires looking to climb the professional ladder he’d constructed for them, from intern all the way up to Editor-in-Chief. Students covered different subjects and wrote articles in a variety of forms in order to fulfill the obligations of their current position at the paper while, at the same time, meeting certain requirements for promotion. At the end of the course, student success was evident in how high a position at the made-up newspaper he or she had achieved.

As was covered in a previous post, one of the principal ambitions of gamification is to design professional and educational processes with heightened consciousness of what motivates engagement. The models for engagement are drawn, with good reason, from games. Games are entirely focused on engagement, and the craft of game design and all the analytics coming out of the multitude of successes and failures in this huge international industry have resulted in a science of engagement. Just how does EVE Online inspire so many people to pour hours into a game that largely plays out on spreadsheets and income ledgers is a mystery that Accounting Profs need to solve.

Of all the angles of engagement coming out of gaming right now, role playing has probably the largest profile when it comes to education, and teachers are experimenting with many methods and to different degrees how to implement role playing into their lesson plans. Mostly, role playing in learning manifests as discrete exercises lasting the length of a single assignment or activity. But some are experimenting with more immersive role playing infrastructures, searching for a course structure that will be as compelling as it is educational.

It was not my project, but I worked with these (great!) guys quite a lot, so I was around for a lot of discussions surrounding its design and execution. An interesting challenge arose:

Promotions had to be made in private or else their representation of a student’s success (or lack thereof) threatened to break FERPA regulations. Students could share their current or past positions with their classmates, of course, and I believe they were even encouraged to do so. However, the inability to require that they do so made collaborations between students of separate roles difficult to design.

The course was a success, I believe, but even more than that it was a lesson in how innovation abuts unpredictably with constraints. One of the principal advantages to creating an organizational model within a course in which students earned roles that defined their contribution to the organization is the opportunity to create dynamic collaborative activities – Editors assigning stories to Staff Writers and Copy Editors heading up peer review, for instance. By many trains of thought, creating organizational simulators within courses that are preparing students for professional placement within the type of organization being simulated is a logical extension of the gamification movement.  However, the inability to require that students be public with their roles within the organization, which, in the case of the course detailed above, reflected their relative success within the organization and thus grades within the course, did not allow for that structure.

A New Paradigm: An Integration of Online Social Work and International Travel

We will be shifting gears today a bit, and will feature an article authored by Leslie Yaffa and Amy Storch on the Social Work in the Caribbean course and their experiences in Jamaica with a cohort of online students who came together for the first time.

http://www.nyu.edu/classes/keefer/waoe/yaffal.pdf

If any of you would like to share your recent publications, shoot us an email! We are always happy to hear from you!

Gamification: The Bartle Test

In academia, “Gamification” is the (lately hyped) practice of applying game design to instructional design. The basic idea behind it is that well designed games motivate players to spend inordinate amounts of time learning the system of the game and boatloads of extra lives and respawns trying to beat/win/“game” the game. If one were to effectively align the artificial system of a game with the real-world process of learning a specific subject, perhaps students would be equally as motivated to learn as they are to study the 100,000 pages of content on the World of Warcraft wiki.

Class is already a game, of course. Generally speaking, most teachers know to scaffold the large “boss” challenges (tests, final papers, etc.) with smaller, “minion” challenges (quizzes, discussion, etc.) so that students have familiarized themselves with the tools they’ve been provided and the environment in which they’re working before they’re flung headlong into ultimate assessment. For the most part, educators understand implicitly the motivational nature of scoreboards (grades) and co-op missions (group learning). The promise of gamification is not that it represents a paradigm shift in education, but that it is a valuable lens for evaluating course design in light of student engagement.

The Bartle Test is one potentially useful product of that lens. Some educators are already using it to inform their course design, primarily in planning how students will interact with each other.

The premise of the Bartle Test is that each player can be mapped onto a 2-dimensional, four-quadrant plane of gamer qualities. No one is entirely any one quality; rather, the qualities are given relative weights to indicate the strength of that quality in the person tested. The result could be exhibited visually like so:

bartle test copy

The individual qualities are:

The Achiever

The Achiever in us feels fulfillment “beating” the game and, if extra content is available, “completing” it so that there is no gem left unclaimed nor item unfound.

in education…

Achiever students are fulfilled by completing assignments to the extent that they turn in extra credit even when extra credit will not improve their grades. Achievers may dislike purposefully ambiguous instructor expectations because achievers value the destination above the experience and would rather complete the assignment than wonder about it.

The Explorer

The Explorer in us is fulfilled by wandering for the sake of wandering. Beating the game on its own terms is a secondary priority to learning about it. Far more pleasurable than beating the game may be adding to its lore, learning more about it than the game’s creators, or even hacking it.

in education…

Explorer students are fulfilled by mapping unmapped regions of a subject. For an explorer, the experience is more important than the destination. Explorers may dislike assignments with guidelines that are too restrictive to allow for unique experiences or expressions.

The Socializer

The Socializer in us is fulfilled by social engagement, and plays games for the sake of being with other gamers.

in educaiton…

Socializer students are fulfilled by cohort- and relationship-building. For socializers, the infrastructure of a course may function primarily as a setting for social interaction. Socializers may dislike assignments that do not have social components, and they may also dislike assignments with social components that pit them against other students.

The Killer

The Killer in us is fulfilled by beating other gamers.

in education…

Killers are fulfilled by zero-sum achievement, wherein their “win” means that someone else “loses.” They are to be distinguished from Achievers in that abstract measurements of success, such as grades, may not count for as much as success achieved through direct competition. Killers may dislike assignments that don’t exercise their compulsion toward conflict and domination. Killers may respond to a shared “leaderboard” with increased desire to have the highest score; by contrast, Achievers may respond to a “personal best scoreboard” in which they are only in competition with themselves.

I took the Bartle Test, and my results are kind of what I’d expect them to be. I think a teacher, provided with the results of the test, might understand a little better what did and didn’t motivate me. Even without students’ results, however, the Bartle Test, and gamification in general, serves as a reminder that course design isn’t just about different styles of retention, but also different styles of motivation.

A note for those who want to take the Bartle Test: It uses some gamer lingo and is targeted specifically at people who play MMORPGs, but I found that I could replace its reference to video games with games that I’ve actually played, such as chess and softball, so that it didn’t feel like I was making up my answers.