Confidence-Based Learning significantly reduces guessing. Guessing, or claiming “I’m Sure” and being incorrect, has negative scoring implications. If a learner answers confidently and incorrectly, the algorithms in Knowledge Factor’s system dock her more points than if she answered “I Don’t Know.”
Knowledge Factor’s excellent reporting functionality offers three categories of reports: User Progress, Performance, and User Activity. Within each of these categories, the following reports are available:
User Progress Reports
Individual Progress, All Iterations: The detail of every iteration for each assessment a learner has taken.
Individual Progress: The quadrant scores, percentage of total questions answered, and CBA Score for each learner in each module, based on the last time a learner saw each question, regardless of iteration.
Individual Progress, First and Last Time Iterated: Learners’ individual progress determined by comparing the quadrant scores, CBA Score, and confidence index for the first and last iteration by learner by module within a domain.
Mastery, User by Module: The number of iterations taken, number of questions in the module, number of questions seen by the learner, number of questions mastered, and mastery percentage, by module by learner.
Misinformation Red Alert: The percentage of incorrect answers by learner based on the last time the learner answered the question, regardless of iteration.
Course Performance: The percentage of questions in each quadrant at the course level. This report also points out key areas of misunderstanding. Results are based on the last time each learner answered each question.
Knowledge Quality by Competency: Quadrant scores and CBA scores by question at the module level for the last time learners answered the question.
User Activity Reports
Learner Iterations by Module: A count of the number of times a learner took each assessment/module within a domain.
User Iterations: The total number of assessments completed by a learner. This report shows iteration counts at the domain level, not divided by module.
Last Date Assessment Taken: The last time a learner took an assessment within a module.
The reports are also available at the organizational level. For example, you can generate reports that reveal, by learning objective, where pockets of doubt, misinformation, or lack of information exist in an entire department or enterprise. These “dashboard reports” go beyond just reporting how many questions employees answered correctly on an assessment, which is where most training programs stop.
Knowledge Factor, by definition, gives you not only the baseline data of learners’ knowledge on their first iteration, but it also gives you the Kirkpatrick Level 2 so you can measure the learning that occurred in each module. Lastly, Knowledge Factor can also incorporate Level 1 (reaction) surveys into each course.
Because the Knowledge Factor tool is delivered on the web, you don’t have to learn a specialized interface. Basically, an instructional designer creates questions in Knowledge Factor’s Microsoft Word template, and they are uploaded into the system.
The Knowledge Factor CBA tool has two major benefits. First, it allows learners to test out of knowledge they already know so they can focus on what they need to know. This targeted learning allows instructional designers to prove through online testing environment that their learners have the requisite knowledge. This is important not only because instructional designers can demonstrate knowledge but also because it might lead to another reason why learners are not performing at optimal level.
Second, the tool significantly reduces guessing because learners must answer not only what they think is the correct answer but also if they are confident in it. The beauty of that is a learner confident about what turns out to be a wrong answer is docked more points than if he marked “I Don’t Know.” Assessment items are randomized so each learner never sees exactly the same assessment.
As a company, Knowledge Factor is very customer oriented. They will work with you from beginning to end to work out anything you need, even if it’s not in their standard offering. And, if you don’t have instructional designers on staff, they offer those services too.
Drawbacks and caveats
In my experience, there aren’t any major drawbacks to the Knowledge Factor system. From an instructional designer’s viewpoint, it is a shift in the way course are developed. Having developed many assessments, I thought it wouldn’t take as long as it did to create the questions for the system. It takes time because each question must have a set of links to back up the correct and incorrect answers, but less time than it takes to develop a traditional online course. This is just a matter of having the links, but as any instructional designer worth her salt knows, gathering information and links can be a huge task. (It’s kind of like trying to get subject matter experts to review course content!)
One other word of warning. As with any tool, what you get out of it is equal to what you put into it [x = f(y)]. I suggest creating scenario-based questions that mirror your learners’ job context so the leap from course to application is smaller.
Also, even though learners can test out of a course, you are wise to test learners’ ability to apply the information in their job context. This is not a limitation of Knowledge Factor’s system but rather a limitation of most online learning. I recommend helping learners apply their knowledge on the job with a part of your curriculum.
Knowledge Factor operates as a learning system, similar to an LCMS, and its reporting function is similar to an LMS’s. Because questions are linked to learning objectives, each set of questions is an individual piece of content and can be turned on or off, depending on need and relevancy. This capability is especially useful if you have content that needs to be delivered to multiple audiences but may need customization by audience.
Knowledge Factor also integrates with LMSs so that information such as completion data and log-in information can be communicated between systems. My understanding is that this communication piece has to be created for each client, but the creation time is relatively short—a couple of weeks or so.
Knowledge Factor uses an approach to pricing similar to a wireless phone plan: system usage buckets by users. Instead of minutes per month, the company charges a rate for the number of users per month. The rate varies by volume and the size and the complexity of the program; the floor rate is $50 per user. Registered users have unlimited access to the program. Knowledge Factor charges a development and service fee for creating the initial program with the client and teaching the client how to create future programs more independently.
Knowledge Factor is a definite “buy.” Not only is their tool cutting edge—their customer service is superb. Knowledge Factor strives to be a company that builds tools that their customers need based on what their customers tell them they need
rather than on their idea of what customers need. The tool goes above and beyond the current trend of “rapid e-learning development” because it accurately measures the knowledge learners gained while in the module and course. As with any tool, though, the course is only as instructionally sound as the methodology used to get there. Simply dumping content into great tool doesn’t make the course instructionally sound.
Knowledge Factor Confidence-based Learning
Interface – 4 Stars ****
- Ease of use – 3 Stars ***
- Documentation – 4 Stars ****
- Value for the money – 3.5 Stars ****
- Overall rating – 3.5 Stars ****
Review by Katica Roy from Training Media Review
* – Note: Confidence-Based Assessment is
a registered trademark.