Hotel Online
News for the Hospitality Executive




Navigating the Labyrinth of Measuring Learning Objective Achievement

By Jim Hartigan
March 6, 2013

Training evaluations…there are many different approaches and methodologies: Kirkpatrick’s four levels, Phillips additional ROI calculation, Kaufman’s five levels, Tyler’s pre-test/post-test, Stake’s responsive evaluation, and on and on. A Google search of the term “training evaluation” returns over 223 MILLION results!  With so many different choices, it’s all-too-easy for a Learning and Development Professional to get lost in the dreaded Labyrinth of Perpetual Learning Objective Confusion (beware the deadly Instructional Designer Minotaur!)  Well fear not! We’re here to equip you with a map and enchanted sack full of magical tips to help you navigate your way to safety – and successful measurement of learning objectives!

The first question you will want to answer is “what do I want to know about my training course?”  Specifically, is it important to you:
  • How the learner felt about the course?  If so, consider utilizing “smile sheets” via online or paper and pencil surveys.
  • What knowledge was transferred to the learner? (And, frankly, who isn’t concerned about this one?)  To determine achievement of learning objectives, capture metrics based on integrated knowledge checks, quizzes, tests, or performance demonstration.
  • How efficient your development and delivery process is?  Measure the cost effectiveness of the course design, development, delivery by comparing it to some metric of student time (per seat hour, per student, per completed course, etc.).
  • What impact your training course had on your organization?  Measure the change in key benchmark data such as customer satisfaction, revenue, or a tracked behavior such as call handle time, conversion rates, etc.
All of these measures have importance in their own right, to be sure.  Happy students make for engaged students and referrals to others to take the course (“positive word of mouth” as we say in the marketing realm).  Efficiency in training design, development, and delivery is critical to the bottom line of every organization – and gaining funding for development of future courses.  At Orgwide we believe when it comes to revamping your training evaluations or beginning a training evaluation program, getting back to the basics is in order.  To that end, we believe the building of understanding, or successful knowledge transfer, enables changed behavior which drives organizational impact.  Therefore, the foundation of successful training is the creation and achievement of well written learning objectives.  As such, we incorporate the creation of our tests, quizzes, and knowledge checks early in the course development process.  So if you’re looking to improve your training evaluation process, consider starting with better measurement of learning objective achievement.

If you’ve been in the Learning and Development arena for any length of time, you’re likely familiar with the ADDIE method of training course design (Analysis, Design, Develop, Implement, and Evaluate).  Our method involves creating the tests, quizzes, and knowledge checks at the conclusion of the Design phase, after the learning objectives have been determined but before course development begins.  We find this process provides a number of benefits including:
  • Validation of the “goodness” of the learning objectives – If you have difficulty measuring the achievement of the learning objective, you may need to revisit the learning objective phase of the project to ensure that your objectives were crafted in an inherently measurable way, using demonstrable and quantifiable language.
  •  Focusing the development efforts – With clearly defined learning objectives and defined methods of measuring student achievement, content development can stay focused with the “end in mind” thereby eliminating non-essential materials which could confuse the student and lengthen the learning experience.
Of course, the best way to measure actual achievement of learning objectives is to conduct pre-testing of the learner to measure both before and after the training event.  If it is not possible to test learners before training, or when the learning objectives require demonstration of a newly acquired skill, you may need to test their performance after training (demonstrating the new skill) and ask the learner whether or not their understanding or skill came from the training session.  This “self-reported” method of measuring learning objective achievement is less than ideal, but better than forgoing the analysis completely.

Whether you are looking to improve your established training evaluation metrics or are incorporating new training metrics and evaluations with a full refurbishment of your training program, starting with the measurement of your course learning objectives is a “back to the basics” approach that will always pay dividends.  Until next time, remember to take care of the customer, take care of each other, and take care of yourself.

Jim Hartigan, Chief Business Development Officer and Partner joined OrgWide Services, a Training/e-Learning, Communications, Surveys and Consulting firm in April 2010 after nearly 30 years experience in the hospitality industry, including the last 18 as a senior executive with Hilton Worldwide. Jim’s last position was that of Senior Vice President – Global Brand Services where he provided strategic leadership and business development and support to the $22B enterprise of 10 brands and more than 3,400 hotels in 80 countries around the world. His team was responsible for ensuring excellence in system product quality, customer satisfaction, market research, brand management, media planning, and sustainability.

Jim Hartigan
Chief Business Development Officer & Partner
OrgWide Services
165 N. Main Street, Suite 202
Collierville, TN 38017
office: 901.850.8190  Ext. 230
mobile: 901.628.6586

Receive Your Hospitality Industry Headlines via Email for Free! Subscribe Here  

To Learn More About Your News Being Published on Hotel-Online Inquire Here

Recent Articles:

Garbage in – Garbage out… Five Tips to Improve Your Hiring Practices / Jim Hartigan / February 2013
Three Hotelier’s New Year’s Resolutions / Jim Hartigan / January 2013
Santa's Secrets to Overcoming an Economic Beating...Oh, yeah...(Happy) Holidays! / Jim Hartigan / December 2012
Handling Customer Complaints Means Taking the H.E.A.T.! / Jim Hartigan / November 2012
Serving Up Success: Waiting Tables Taught Me Customer Service? / Stephanie Hartigan / October 2012
The Similarities Between Marketing and Training – Start, Stop, and Continue / Jim Hartigan / October 2012
Customer Loyalty in as Little as 17 1/2 Minutes! / Bob Taylor and Jim Hartigan / August 2012
First Things First - How to Start a Successful Mentoring Program / Bob Taylor / May 2012
Why Can't We All Just Get Along? (It turns out, we're not supposed to!) / Bob Taylor / August 2011
What Do Hotel Leaders Really Do? / Bob Taylor / February 2011
Hotel Management – What’s Love got to do with it? / Jim Hartigan / February 2011
Coaching Hotel Team Members the OSKAR Way - Not like training a dog! / Jim Hartigan / January 2011
Hospitality Employee Selection and the Venus Fly Trap – A Study in Floral Business Acumen / Jim Hartigan / December 2010
The Right Tool for the job makes all the difference in the world! / Jim Hartigan / October 2010
Restore Honor to Your Organization: Succession Planning…the Ninja Way / Jim Hartigan / October 2010
Measure to Manage – The importance of KPIs ... and vampires / Jim Hartigan / September 2010
For Everything There is a Season - Including Budgets / Jim Hartigan / August 2010
Say What? Listen, Learn, and Act–Why Guess, When You Can Know Redux / Jim Hartigan / August 2010
Why Guess When You Can Know - Maximizing the Effectiveness of Your Workplace Surveys / Jim Hartigan / August 2010
Team Member Segmentation in the Workplace…or “If everyone brought potato salad to the picnic – it wouldn’t be much of a picnic!” / Jim Hartigan / July 2010
The Power of Effective Communication in the Workplace (and our Founding Fathers’ unrivaled Tweeting abilities) / Jim Hartigan / July 2010

To search Hotel Online data base of News and Trends Go to Hotel.OnlineSearch

Home | Welcome | Hospitality News
| Industry Resources

Please contact Hotel.Online with your comments and suggestions.