Solving the Assessment Puzzle

Photo of Rubik's Cube on a table, with a man sitting behind it and looking at it.

Education image created by Freepik.

By Sarah Hansen, OCW Educator Project Manager

Assessing students’ learning is one of the most important things we do as educators. It’s also one of the most complicated. There’s a lot to consider:

  • When will assessment happen? (Along the way? At the end of the course?)
  • How will we collect useful information about student learning? (Through writing samples? Surveys? Online reading questions? Student self-assessments? Performance assessments? Something else?)
  • How will we assess work that doesn’t have right and wrong answers, like creative writing or digital media projects?
  • How will we assess work students complete in teams? (It’s hard enough to assess students individually! But we know collaboration is an essential skill—so how do we measure it in a way that’s fair to individuals?)
  • How will we effectively communicate feedback to students (Via rubrics? Written comments? Oral exams that function as educative conversations?)
  • How will we use assessment to improve our own teaching? (When should curricular iteration occur?)

For every group of students, there’s a different combination of productive approaches to assessment that instructors need to configure. It’s a shape-shifting puzzle that can be exciting, enervating, and downright addictive. If you’re an educator and you’re intrigued by “the assessment challenge,” you’re not alone. MIT instructors are thinking hard about measuring student learning, providing feedback, and improving their teaching based on what they learn through assessments. In the following short videos, six MIT instructors candidly share the assessment strategies they’ve been trying in their own classrooms:

  • In 16.842 Fundamentals of Systems Engineering, students in a Small Private Online Course (SPOC) worked in teams to participate in an international imitation satellite design competition. Aero/Astro Professor Olivier de Weck shares how he assessed work students completed as teams, how he conducted online written and oral exams, and how made use of students’ personal reflective memos to understand what they learned in the course.
  • Elizabeth Choe gets into the nitty gritty of how she approached assessment and feedback in the creative context of 20.219 Becoming the Next Bill Nye, a course in which students conceptualized and produced educational videos (no multiple choice tests here!).
  • Takako Aikawa discusses how she used a daily grading system and interview tests to provide students with feedback about their language learning in 21G.503 Japanese III.  (You can view this video in Japanese, too.)
  • In CMS.611 Creating Video Games, students worked in teams to develop games for a real client: The Red Cross/Red Crescent Climate Center. Sara Verrilli shares how instructors assessed these projects, emphasizing that students’ processes and project management skills were more important than the final products.
  • Professor Joe Schindall opens up about grading in ESD.051J Engineering Innovation and Design, noting that students’ “passion of engagement” and their willingness to try new things were factors the instructors considered when assessing student learning in this Engineering Systems Division course.
  • Professor Catherine Drennan shares how she uses clicker competitions to engage students and formatively assess learning in 5.111SC Principles of Chemical Science. (Spoiler Alert: Things get heated.)

Want more MIT instructor insights about assessment? Head over to our OCW Educator portal and click “Assessment.” Then filter your results by topic, such as feedback, formative assessment, performance assessment, student self-assessment, and more.

If you find a strategy on our site that helps you solve (or inspires you to think differently about) your assessment puzzle, we want to hear from you! We’ll share some of the trickiest puzzles with the most creative solutions on our Facebook page. Go!

Insights on Teaching Japanese, in Japanese (and English)

Traditional Japanese masks at Senso-ji (浅草寺) in Tokyo, Japan. (Image courtesy of rita11836 on Flickr. License CC BY-NC-SA.)

By Joe Pickett, OCW Publication Director

OCW has just published 21G.503 Japanese III, the third in a four-course sequence on Japanese taught at MIT. With relatively few Japanese speakers on the MIT campus, the instructors must make the most of what happens in the classroom and motivate students to work hard outside it.

The course site’s Instructor Insights feature brief video interviews with one of the instructors, Takako Aikawa. She addresses key topics in language instruction, such as grammar and drill sessions, developing students’ language skills, assessing students, and teaching language through culture.

Each interview is presented twice: once in English…

…and again in Japanese.

OCW has published similar interviews in two languages for 21G.101 Chinese I (Regular) and 21G.108 Chinese II (Streamlined).

Instructors of any language will surely benefit from the reflections and advice offered in all of these interviews. Maybe students even more!

Look What Happened Over the Holidays

Screenshot of OCW homepage, with "For Educators" dropdown menu exposed.

Did you notice? There’s a new tab at the top of the OCW homepage: For Educators!

Under this tab, we’ve collected all OCW and OCW-related resources that are of special interest to educators—and really anyone interested in education.

So please take a look and explore this side of OCW. You’re bound to make new discoveries!

The Year is Ending, but these Teaching Insights are Fresh

By Sarah Hansen, OCW Educator Project Manager

MIT instructors share their teaching approaches in a special section of their OCW courses, called “Instructor Insights.” In these sections, you’ll find instructors discussing topics of interest to education professionals, such as course design, active learning, and engaging learners.

The year may be coming to a close, but we’ve recently published 11 courses with new Instructor Insights—and they are super fresh! Below are a few highlights:

Find something you like? Share directly to Facebook using our “Share Quote” feature.

And if you like these, there’s many others in our collection of all OCW courses with Instructor Insights.

Nine-Day Workshops with Lessons to Last You a Lifetime

Collage of four photos showing students working on and showing their electronics systems.

Students in an electronics workshop that features Arduino microcontrollers collaborate to design a prototype. (Image courtesy of Andrew Ringler.)

By Joe Pickett, OCW Publication Director

You’ve got the month of January off from regular classes, and you want to do more than sleep in. It’s a good time to experiment, to do something unusual, maybe create a software project. But what if you don’t really have the background for this sort of thing?

Sign up for a workshop that requires no experience at all!

Two such workshops taught by the same instructor have just appeared on OCW site: Collaborative Design and Creative Expression with Arduino Microcontrollers and Learn to Build Your Own Videogame with the Unity Game Engine and Microsoft Kinect.

In the Arduino workshop, students in small teams create different projects using Arduino microcontrollers, including a hand-motion controlled “car,” dazzling light displays, and a punching glove that measures the intensity of its blows.

In the Videogame workshop, student teams create videogames in which the player moves and controls an object in space by body motions: animals try to escape from a zoo, cubes assemble to build and decorate houses, objects traverse landscapes full of obstacles.

The OCW workshop sites have videos of class activities and student-narrated projects, so you can see what the students did and how they thought about what they made.

Fostering Learner Self-Confidence

What these workshops have in common is an unconventional teaching methodology championed by Kyle Keane and Andrew Ringler, two of the instructors, and shared in their Instructor Insights for the Arduino and the Videogame workshop. Each set of Insights is tailored to the demands of that particular workshop.

The main goal for the workshops is to help students build confidence so they can gain independence from their instructors and learn on their own. To do this, they employ a variety of techniques to shake students out of their accustomed ways of thinking about learning and creativity, so they can move forward and be productive.

Building Productive Teams

To work with a team you first have to get on one, and to do that, it helps to know which people seem best suited as teammates. The workshop employs some techniques used in improv comedy to get people familiar with one another fast. Keane explains:

I use improv warm up exercises (games performers play to get ready for a show) to help participants explore how verbal and nonverbal communication impact their collaborative relationships in the workshop.

He encourages students to explore different possible teams and to not be afraid to move out of one and join another. With little or no experience, students are bound to make impractical suggestions and show a certain degree of ignorance.

Modeling Vulnerability

To help defuse student’s fear of embarrassment, Keane shares his own, and in doing so he models vulnerability, which

…is not a very common post-secondary teaching strategy, but…it’s an important thing to do when building team dynamics, because, let’s face it, opening yourself up to critique is terrifying…So, as instructors…we stand in the front of the classroom and talk about how it feels to be vulnerable. We’re weirdly explicit about it, but we find it extremely effective.

Going hand in hand with this technique is showing (rather than telling) students that it’s OK to ask for help: “It’s better to coach them and to model how to bring in others to solve problems.”

Being Creative, Not Original

In a nine-day workshop, there is hardly time to reinvent the wheel, yet in conceiving a creative project, students often think that’s what they have to do. To defuse this dynamic, Keane reframes the creative process away from being completely original to building on existing ideas and taking them in new directions. So the workshop

…involves students mimicking, step-by-step, projects that have already been built and then deviating from them—to give students permission to build on existing work.

Failing on Purpose

At the same time, to get students comfortable with risk-taking Keane gives them “assignments that ask students to do the impossible (like build a video game in six hours as a team, for example).”

These present opportunities for learning how to work with people having very different skills:

Participants don’t truly understand they need to collaborate with people who bring different skill sets to the work until they fail at a project…[Failure] helps drive home the importance of working in groups of people with diverse interests and abilities.

In Keane’s view, if a project is “designed to fail,” it holds the potential for longer-term success:

If it’s a designed-to-fail project…you pick something that’s kind of kooky that you want to learn, because no one’s going to know that you overstretched your skill set and tried something that was outside of your range. In this workshop, we explicitly allow (and encourage) participants to take these risks.

Moreover, doomed projects

…free participants to do things they might consider ridiculous, crazy, or imaginative. If you know the project is not intended to be successful, why not stretch your perceived boundaries?

Indeed, why not?

Inspire your network with our new “Share Quote” feature

When you read something that’s inspiring, do you want to spread the word?  We hope so!

OCW’s growing collection of Instructor Insights pages is chock full of inspiring ideas for educators, where MIT faculty talk about how they teach. Our brand-new Facebook “Share Quote” tool makes it easy to share your favorite nuggets from these pages.

Simply highlight any text on an Instructor Insights page. When the “Share Quote” bubble pops up, click on it, and a Facebook post window pre-filled with your selected quote will appear. Add optional commentary, click the “Post to Facebook” button, and you’re done!

Who Knows That Healthcare Can Be So Complicated? This Sloan Instructor Does!

A female health worker logs data on a handheld device as two others look on.

Sangath health workers in Goa log patient data. (Courtesy of Frederick Noronha and Sangath on Flickr. License: CC BY-NC.)

By Joe Pickett, OCW Publication Director

Healthcare can be mind-boggling in its complexity, but diagnosing and treating patients has become routine business in counties like the US. You feel bad, call up, and go to see someone, and they give you advice and a therapeutic regimen to follow. Unless you’re part of an unlucky minority lacking health insurance, it’s pretty simple from the viewpoint of a patient.

But it’s hardly simple in much of the developing world, where communications are spotty, transport is unreliable, facilities are sparse, appropriate interventions are in short supply, and, perhaps most crucially, trained healthcare providers are relatively few and face overwhelming demand on a daily basis.

Non-governmental organizations—commonly called nonprofits in the US—have stepped in heroically to try to improve this situation and give ordinary people living in impoverished conditions a chance at a healthy life. But these heroes can themselves be overwhelmed. Far too often, they have little opportunity or resources to find, let alone implement, innovations in how healthcare is delivered.

So, you smart MIT Sloan graduate students, what advice can you come up with to help a mission-driven NGO innovate in delivering quality healthcare to those who most need it? To add to the challenge, the focus is on mental health and developmental disabilities—areas where problems are rarely solved with a single treatment.

By the way, you don’t have three years to study and develop your ideas. You have three days.

Ready, set, go!

Working Remotely with an NGO

Such is the academic challenge of 15.ES718 Global Health Innovation: Delivering Targeted Advice to an Organization in the Field.

The course—an intensive workshop, really—is taught by Dr. Anjali Sastry, and the site just published on OCW reflects the 2015 iteration of her teaching. In this instance, the class connected with Sangath, an inspiring NGO that provides mental health services to poor people in India.

The workshop represents a logistical tour de force. Just identifying a partner organization to collaborate with the class, writing a proposal that must convince the NGO’s board, defining researchable topics, and lining up people willing to make themselves available at short notice for the class, is a huge task. But Dr. Sastry has been at this sort of thing for quite a while, and she can be very convincing.

Priming Students to Produce Meaningful Ideas

An introduction to global health issues and to the selected organization sets the stage. After learning about the staff team they will work with, the students explore an array of readings, form groups to tackle specific areas proposed by the partner, and pinpoint key topics. By the day’s end, they have formulated meaningful questions to put to NGO staff in their initial conversations.

To expand their thinking, students select from a roster of expert researchers, entrepreneurs, and clinicians whose work may be relevant to their projects, visiting some in their labs and offices across the MIT campus and elsewhere and calling others to glean their ideas and learn of their innovations.

Drawing on this high-speed networking and their own creativity, the students develop a presentation for the NGO panelists, who weigh this advice and provide feedback based on their very practical experience. Then the students refine their presentations once more and share them with the class. Throughout it all, Dr. Sastry is there, serving as a sounding board and advisor, helping the students shape their work into something valuable.

Representing Process with Resources Galore

This intense process is represented on the OCW course site in multiple ways: class activities, readings and materials (including extensive resources on the Indian healthcare system and healthcare delivery, the status and needs of the Indian population, and Sangath itself), lecture notes, and more. Perhaps most important are the Instructor Insights of Dr. Sastry, which cover learning goals, selecting a partner organization, logistical challenges, motivating students, and providing students with useful feedback so the ideas they develop are actually useful. In keeping with another thread of her work on learning from small failures, Dr. Sastry also shares her reflections on what worked—and what she’d do differently.

If it seems amazing that an instructor can undertake a course like this, further amazement awaits on OCW’s other Anjali Sastry course sites: 15.232 Business Model Innovation: Global Health in Frontier Markets and 15.S07 Global Health Lab.