Solving the Assessment Puzzle

Photo of Rubik's Cube on a table, with a man sitting behind it and looking at it.

Education image created by Freepik.

By Sarah Hansen, OCW Educator Project Manager

Assessing students’ learning is one of the most important things we do as educators. It’s also one of the most complicated. There’s a lot to consider:

  • When will assessment happen? (Along the way? At the end of the course?)
  • How will we collect useful information about student learning? (Through writing samples? Surveys? Online reading questions? Student self-assessments? Performance assessments? Something else?)
  • How will we assess work that doesn’t have right and wrong answers, like creative writing or digital media projects?
  • How will we assess work students complete in teams? (It’s hard enough to assess students individually! But we know collaboration is an essential skill—so how do we measure it in a way that’s fair to individuals?)
  • How will we effectively communicate feedback to students (Via rubrics? Written comments? Oral exams that function as educative conversations?)
  • How will we use assessment to improve our own teaching? (When should curricular iteration occur?)

For every group of students, there’s a different combination of productive approaches to assessment that instructors need to configure. It’s a shape-shifting puzzle that can be exciting, enervating, and downright addictive. If you’re an educator and you’re intrigued by “the assessment challenge,” you’re not alone. MIT instructors are thinking hard about measuring student learning, providing feedback, and improving their teaching based on what they learn through assessments. In the following short videos, six MIT instructors candidly share the assessment strategies they’ve been trying in their own classrooms:

  • In 16.842 Fundamentals of Systems Engineering, students in a Small Private Online Course (SPOC) worked in teams to participate in an international imitation satellite design competition. Aero/Astro Professor Olivier de Weck shares how he assessed work students completed as teams, how he conducted online written and oral exams, and how made use of students’ personal reflective memos to understand what they learned in the course.
  • Elizabeth Choe gets into the nitty gritty of how she approached assessment and feedback in the creative context of 20.219 Becoming the Next Bill Nye, a course in which students conceptualized and produced educational videos (no multiple choice tests here!).
  • Takako Aikawa discusses how she used a daily grading system and interview tests to provide students with feedback about their language learning in 21G.503 Japanese III.  (You can view this video in Japanese, too.)
  • In CMS.611 Creating Video Games, students worked in teams to develop games for a real client: The Red Cross/Red Crescent Climate Center. Sara Verrilli shares how instructors assessed these projects, emphasizing that students’ processes and project management skills were more important than the final products.
  • Professor Joe Schindall opens up about grading in ESD.051J Engineering Innovation and Design, noting that students’ “passion of engagement” and their willingness to try new things were factors the instructors considered when assessing student learning in this Engineering Systems Division course.
  • Professor Catherine Drennan shares how she uses clicker competitions to engage students and formatively assess learning in 5.111SC Principles of Chemical Science. (Spoiler Alert: Things get heated.)

Want more MIT instructor insights about assessment? Head over to our OCW Educator portal and click “Assessment.” Then filter your results by topic, such as feedback, formative assessment, performance assessment, student self-assessment, and more.

If you find a strategy on our site that helps you solve (or inspires you to think differently about) your assessment puzzle, we want to hear from you! We’ll share some of the trickiest puzzles with the most creative solutions on our Facebook page. Go!

Insights on Teaching Japanese, in Japanese (and English)

Traditional Japanese masks at Senso-ji (浅草寺) in Tokyo, Japan. (Image courtesy of rita11836 on Flickr. License CC BY-NC-SA.)

By Joe Pickett, OCW Publication Director

OCW has just published 21G.503 Japanese III, the third in a four-course sequence on Japanese taught at MIT. With relatively few Japanese speakers on the MIT campus, the instructors must make the most of what happens in the classroom and motivate students to work hard outside it.

The course site’s Instructor Insights feature brief video interviews with one of the instructors, Takako Aikawa. She addresses key topics in language instruction, such as grammar and drill sessions, developing students’ language skills, assessing students, and teaching language through culture.

Each interview is presented twice: once in English…

…and again in Japanese.

OCW has published similar interviews in two languages for 21G.101 Chinese I (Regular) and 21G.108 Chinese II (Streamlined).

Instructors of any language will surely benefit from the reflections and advice offered in all of these interviews. Maybe students even more!

Some Timely Courses for Our Trying Times

Two men standing in a muddy debris-strewn street.

Residents begin to assess the damage after Hurricane Maria hit the island of Dominica in September 2017. (Public domain image by Roosevelt Skerrit on Flickr).

By Joe Pickett, OCW Publication Director

We noticed that these courses, published in the past month, seem particularly relevant in light of recent events.

11.027 Global City Scope—Disaster Planning and Post-Disaster Rebuilding and Recovery, taught by Cherie Miot Abbanat.

What’s your town’s disaster mitigation plan? Does it even have one? Does it seem like a viable way of handling an emergency, or is it just a report that sits on a shelf?

Analyzing and evaluating one of these plans is one of your assignments when you take this course.

And what course could be more timely? Recent months have seen so many mind-blowing disasters, one after another—hurricanes of phenomenal destructive power, monster wild fires, crushing mud slides, earthquakes, a bomb cyclone.

The course has four modules: Disaster Mitigation, Preparedness and Planning, Disaster Response, Disaster Recovery and Rebuilding.

21A.429J Environmental Conflict, taught by Professor Christine Walley.

Is there a recent  environmental issue that has not generated conflict? Fracking? The regulation of chemicals and toxins? Offshore oil drilling? The status of natural parks and preserves?

This course provides the theoretical frameworks for thinking about such conflicts, and focuses

…on a number of often contentious issues, including: ideas of “nature” and the politics and practices of nature conservation; the links between toxic pollution and health effects; the complexity of human / non-human relations as seen through the lens of multispecies frameworks; and debates over crucial contemporary issues ranging from climate change to natural gas exploration.

17.269 Race, Ethnicity, and American Politics, taught by Professor Ariel White.

The course description sums it up nicely:

What is “race”? How could we possibly measure it, and does it really matter? What does it mean to say that a policy is discriminatory, and how have social scientists and courts tried to measure racial discrimination? What do Americans think about race in the 21st century, and how do these opinions shape their voting and protest behavior?

After taking this course, students will be able to discuss different ways of imagining race and ethnicity, and their historical underpinnings. They will be able to describe and critique the ways in which racial attitudes are theorized and measured, and think about how these different attitudes are expected to shape political behavior…

Teaching a course on such a sensitive subject requires more than a little thoughtfulness. The course site includes some fascinating Instructor Insights, including “Facilitating Talk about Race and Ethnicity” and “Fostering Intuition about Social Science.”

21H.983 Gender, taught by Professors Lerna Ekmekcioglu and Elizabeth A. Wood.

As much as gender is discussed in the popular media, this course explores some challenging questions that you don’t see posed very often, at least not directly:

How does gender work? How is the body itself sexed and gendered in different times and places? How do gender, race and class work in historical context? Does gender influence state formation and the work of the state? What role does gender play in imperialism and in the welfare state? What is the relationship between gender and war? How does the state regulate the body in the modern world? What are some new directions in the study of gender?

17.480 Understanding Military Operations, taught by Professor Owen Cote.

Right now some 300,000 US military personnel, often using highly sophisticated technology, are deployed in over 150 countries around the world. This course offers the chance to assess the thinking behind deployments like these and how they might change in the future.

The course covers a full range of topics, from military doctrine to tactical mobility. As the course description states:

This seminar will break apart selected past, current, and future sea, air, space, and land battlefields into their constituent parts and look at the interaction in each of those warfare areas between existing military doctrine and weapons, sensors, communications, and information processing technologies. It will specifically seek to explore how technological development…is influenced in each warfare area by military doctrine.

Look What Happened Over the Holidays

Screenshot of OCW homepage, with "For Educators" dropdown menu exposed.

Did you notice? There’s a new tab at the top of the OCW homepage: For Educators!

Under this tab, we’ve collected all OCW and OCW-related resources that are of special interest to educators—and really anyone interested in education.

So please take a look and explore this side of OCW. You’re bound to make new discoveries!

The Year is Ending, but these Teaching Insights are Fresh

By Sarah Hansen, OCW Educator Project Manager

MIT instructors share their teaching approaches in a special section of their OCW courses, called “Instructor Insights.” In these sections, you’ll find instructors discussing topics of interest to education professionals, such as course design, active learning, and engaging learners.

The year may be coming to a close, but we’ve recently published 11 courses with new Instructor Insights—and they are super fresh! Below are a few highlights:

Find something you like? Share directly to Facebook using our “Share Quote” feature.

And if you like these, there’s many others in our collection of all OCW courses with Instructor Insights.

Serving you and millions of learners ✏

“I am a computer enthusiast and I always
wanted to attend an institute of higher
level, but then I found a job with shift
work very hard, which does not allow me
to attend.

In addition, the high costs that I could
not support, having lost my dad at 13 and
having to help with household expenses,
were prohibitive.

So I decided to remain a self learner and
I went in against a big problem: which
sources are reliable and which are not?

Internet is immense and the risk of
getting the wrong information is high …

As I became aware of MIT OPENCOURSEWARE
I was delighted to find an authoritative
source where I could find a lot of
information for free. It’s really a
fantastic idea and I thank all the
teachers who volunteered to share
their knowledge.”

-Alessandro, Independent Learner, Italy

 

Dear Friend of OCW,

It’s a privilege for us at OCW to serve enthusiastic learners like Alessandro and millions of others who look for free and reliable educational resources to support their educational pursuits.

Please consider OCW in your end-of-year giving so that we can continue to openly share MIT materials with a world of people seeking to refresh their knowledge, learn something new, or gain the understanding they need to fulfill their life goals.

Your donation, large or small, will help us to keep publishing and distributing the educational materials that make a difference to learners everywhere.

Why not be part of OCW’s effort to advance learning? If you can, please support OCW today.

Sincerely,

Joe

Joseph Pickett
Publication Director
MIT OpenCourseWare

 

ocwbythenumbers2

Nine-Day Workshops with Lessons to Last You a Lifetime

Collage of four photos showing students working on and showing their electronics systems.

Students in an electronics workshop that features Arduino microcontrollers collaborate to design a prototype. (Image courtesy of Andrew Ringler.)

By Joe Pickett, OCW Publication Director

You’ve got the month of January off from regular classes, and you want to do more than sleep in. It’s a good time to experiment, to do something unusual, maybe create a software project. But what if you don’t really have the background for this sort of thing?

Sign up for a workshop that requires no experience at all!

Two such workshops taught by the same instructor have just appeared on OCW site: Collaborative Design and Creative Expression with Arduino Microcontrollers and Learn to Build Your Own Videogame with the Unity Game Engine and Microsoft Kinect.

In the Arduino workshop, students in small teams create different projects using Arduino microcontrollers, including a hand-motion controlled “car,” dazzling light displays, and a punching glove that measures the intensity of its blows.

In the Videogame workshop, student teams create videogames in which the player moves and controls an object in space by body motions: animals try to escape from a zoo, cubes assemble to build and decorate houses, objects traverse landscapes full of obstacles.

The OCW workshop sites have videos of class activities and student-narrated projects, so you can see what the students did and how they thought about what they made.

Fostering Learner Self-Confidence

What these workshops have in common is an unconventional teaching methodology championed by Kyle Keane and Andrew Ringler, two of the instructors, and shared in their Instructor Insights for the Arduino and the Videogame workshop. Each set of Insights is tailored to the demands of that particular workshop.

The main goal for the workshops is to help students build confidence so they can gain independence from their instructors and learn on their own. To do this, they employ a variety of techniques to shake students out of their accustomed ways of thinking about learning and creativity, so they can move forward and be productive.

Building Productive Teams

To work with a team you first have to get on one, and to do that, it helps to know which people seem best suited as teammates. The workshop employs some techniques used in improv comedy to get people familiar with one another fast. Keane explains:

I use improv warm up exercises (games performers play to get ready for a show) to help participants explore how verbal and nonverbal communication impact their collaborative relationships in the workshop.

He encourages students to explore different possible teams and to not be afraid to move out of one and join another. With little or no experience, students are bound to make impractical suggestions and show a certain degree of ignorance.

Modeling Vulnerability

To help defuse student’s fear of embarrassment, Keane shares his own, and in doing so he models vulnerability, which

…is not a very common post-secondary teaching strategy, but…it’s an important thing to do when building team dynamics, because, let’s face it, opening yourself up to critique is terrifying…So, as instructors…we stand in the front of the classroom and talk about how it feels to be vulnerable. We’re weirdly explicit about it, but we find it extremely effective.

Going hand in hand with this technique is showing (rather than telling) students that it’s OK to ask for help: “It’s better to coach them and to model how to bring in others to solve problems.”

Being Creative, Not Original

In a nine-day workshop, there is hardly time to reinvent the wheel, yet in conceiving a creative project, students often think that’s what they have to do. To defuse this dynamic, Keane reframes the creative process away from being completely original to building on existing ideas and taking them in new directions. So the workshop

…involves students mimicking, step-by-step, projects that have already been built and then deviating from them—to give students permission to build on existing work.

Failing on Purpose

At the same time, to get students comfortable with risk-taking Keane gives them “assignments that ask students to do the impossible (like build a video game in six hours as a team, for example).”

These present opportunities for learning how to work with people having very different skills:

Participants don’t truly understand they need to collaborate with people who bring different skill sets to the work until they fail at a project…[Failure] helps drive home the importance of working in groups of people with diverse interests and abilities.

In Keane’s view, if a project is “designed to fail,” it holds the potential for longer-term success:

If it’s a designed-to-fail project…you pick something that’s kind of kooky that you want to learn, because no one’s going to know that you overstretched your skill set and tried something that was outside of your range. In this workshop, we explicitly allow (and encourage) participants to take these risks.

Moreover, doomed projects

…free participants to do things they might consider ridiculous, crazy, or imaginative. If you know the project is not intended to be successful, why not stretch your perceived boundaries?

Indeed, why not?