Making the Migration from Post-tests to Continuous Assessments and Performance-based Assessments
Many learning designs follow a standard model, but this is not the only way to get learners to demonstrate mastery. In her video on eLearning Assessment Trends and Strategies, Sabramowicz asks if traditional assessment methods make learners regurgitate the material they’ve learned.
The ability to provide great content for a learner in a concise and visually pleasing format is one thing. Effectively assessing learners’ abilities to translate learning into performance is another.
Taking a cue from Duolingo, Sabramowicz advocates for using continuous assessment to teach learners.
How Does Continuous Assessment Work With Duolingo?
Duolingo follows a slightly different path than traditional eLearning methods.
Learners encounter a question. Learning progresses from one question to another until the student runs into a question they cannot answer. Only then are they provided with the instruction they need to answer that question.
Using Continuous Assessments in SBeL
Often in workplace environments, learners already have a wealth of information a click away. They are on an information overload. If this is the case, the goal for instruction shouldn’t be to present the content in novel ways or to extract micro modules out of large content. The goal should be to find ways to assess whether learners can migrate from knowledge acquisition to application.
What if the way we present content to learners hinders instead of enhances their ability to develop more effective problem-solving skills? Instead of presenting the material to learners, what if we first present the test or the problem they need to resolve?
Learners who perform as expected can move on to applying their knowledge, while learners who underperform are provided feedback and instruction before they are assessed again.
Scenario-based eLearning design lends itself well to this format as it enables learners to learn, receive immediate feedback, and apply their learning to the following questions. With this method, Instructional designers can present the learner with a problem and direct them towards gaining instruction if they require it. Engaging learners this way may help them find better and more efficient ways of learning than if they had learned using traditional learning methods.
Making Design Decisions
There’s the danger of designing assessments that focus on a single outcome, such as the ability to recall facts. If this is the sole goal of instruction, this is fine, but if not, it is a waste of resources and the learner’s time.
Take, for instance, a course designed to help a cars saleswoman sell more cars. At the end of the course, the assessment tests the learner’s ability to recall information such as car trim levels and specs. Let’s say the pass mark is 80%, and this saleswoman scores 95%. It seems remarkable, right? Up until you discover down the line that she is not able to sell cars.
So, the learner passed the quiz, but there’s no change in performance. Baffling right? You look at the beautifully designed course with the excellent test questions and wonder where the disconnect is. At this point, you realize that the learner’s ability to recall all the details of the latest car isn’t the skill you want learners to transfer to the job. The saleswoman can look up details from a brochure.
If you reimagine the assessment through the lens of problem-first, the assessment strategy will grow beyond testing for factual knowledge to testing for knowledge transfer.
Using a continuous assessment strategy, instructional designers can build assessments that test communication skills, provide a rationale for car recommendations, and address objections or queries from a customer to make the sale.
When designing for performance, we want to ask questions like:
- What kind of assessments should we plan to ensure near or far skill transfer?
- What skills do the learners need?
- Are they hard skills or soft skills?
- What kind of everyday decisions do they need to be making on the job?
Asking these questions sets designers on the right path to designing well-thought-out assessment questions that elicit the behavior we want to see the learner perform on the job. To do this right, we must first identify the problems learners need to solve on their jobs.
How Effective is a Continous Assessment Senario-based eLearning Design?
By design, Scenario-based eLearning (SBeL) continually assesses learners by requiring them to apply their knowledge and skills when presented with a problem. SBeL channels learners into different experiences based on their choices and decision points, thus effectively acting as a form of continuous assessment.
Looking back to Sabramowicz’s mini case study of Duolingo’s assessment methods. Some might argue that asking learners question after question might prove frustrating when they want to learn about a topic, and understandably so. There’s a thin line between annoying learners and engaging them with the learning process. A continuous assessment strategy might cause disengagement for learners who have no background in the topic or experience to draw from. However, this is rarely the case with adult learners in the workplace, as they usually have some frame of reference to draw upon when learning. Besides, SBeL works best for learners with moderate to an advanced expertise.
One question to ask is if decision points provide enough assessment opportunities for learners to demonstrate mastery. You might need to incorporate more ways to test learners, and you want to ensure that whatever you design simulates real workplace situations.
The Case for a Continous Assessment Strategy in Scenario-based eLearning
The continuous assessment strategy works well with Duolingo because almost everyone has, at the very least, moderate experience with learning a language. So, when faced with questions, learners can draw on their experience with language structure and word origins to make connections in learning.
As an instructional strategy, continuous assessment can be challenging when introducing new content. Also, novice learners might struggle to learn from a course delivered from start to finish using continuous assessment as an instructional strategy. In addition, a continuous assessment strategy does not eliminate the need for benchmark tests. It simply enables a low-risk environment for learners to fail forward. So, when using this strategy, you will want to make sure you are mindful of your learner’s characteristics and the instructional goals of the course.
Benefits of Using a Continuous Assessment Strategy in Instructional Design
It Lends Itself to How Adults Naturally Learn
One way that adults learn is from a ‘need of moment’ learning. They search for a way to resolve problems when they arise. Continuous assessment as a strategy reduces the need for any extraneous processing during learning. Learners can quickly move past what they know and focus on what they don’t know, as instead of wading through mandatory content, they get to learn from the problems they have to solve.
Continuous assessment is a good fit for scenario-based learning since it may reduce unnecessary cognitive processing. Learners can answer questions as they progress through the scenario that allows them to skip content they know or otherwise proceed to gain mastery of the content they need to know to perform well on their jobs. With every choice or decision the learner makes, a well-designed scenario should adapt intuitively to the learner’s needs. In some ways, this is a form of continuous assessment.
Consider an eLearning module that teaches employees how to sell the suitable camera model to a new customer. Rather than providing details of the cameras’ specifications in the module, the module could begin by asking questions about what kind of camera an outdoor enthusiast or professional would find helpful. In deliberating over these questions, the learner can draw upon his past experiences. The learner can then match the customer’s needs to the right camera and learn the specifications.
It Increases Proficiency
Through a continuous assessment process, the learner will become proficient at selling the right camera with the right specs based on the customer’s needs. Such an approach would achieve the desired outcomes better than a module introducing the specs and testing learners on the camera specs at the end of the course.
Learning through continuous assessments can provide learners with a more satisfying learning experience than information-practice-assessment-type modules, but these types of modules are more challenging to design. Incorporating continuous assessment into eLearning scenarios can impact the design in several ways. Adding more branching scenarios to meet learners’ needs is an obvious one. If done right, the learning experience will be far superior to anything traditional models of instruction can offer.
Conclusion
As testing remains the most objective method to assess the instructional effectiveness of your course, you should design tests that allow learners to demonstrate the skills they have learned. Make your branching scenarios identify the learners’ knowledge gaps so that you can recommend specific instructional content to fill those gaps. This will ensure that every learner finishes the course with a solid understanding of the workplace problems they must solve.
Identify learners who display mastery, then provide relevant and more profound content where applicable.
Finally, if you only take three things away from this post, let them be
- Focus on building problem-solving skills
- Construct tests that evaluate these skills and
- Assess often and not just at the end of your course.
For more strategies for informing your designs, check out my post – Using Sound Instructional Strategies in Scenario-Based eLearning Design.