Learning Technology Tools: Measurement First

Technology continues to expand our frontiers. SpaceX launched rockets that can orbit the earth and then return to a landing pad, ready to be refueled and launched again. Submarines stay submerged for months on stealth missions, and Google is prototyping driverless cars. What an incredible time to experience new technologies!

It is also an exciting time for learning and development (L&D) groups because new technologies are changing the way we learn and helping L&D departments improve their reach.  Before taking a look at some of those new technologies, let’s consider what technology should do for L&D.

What should technology do for L&D?

Typically, technology enables business processes. It makes great leaps forward in our ability to do things faster and better. More specifically, it creates efficiencies, increases productivity, changes our world view. Here are some examples:

Increase efficiency: There are so many ways that technology achieves efficiency. Communication is probably top of mind for most of us.  Email, texting, phone calls, and related hands-free options help us share critical information with people around the world instantaneously without having to repeat ourselves. Simpler devices also abound like money counting machines in banks and bar code readers at grocery stories.  The evolution of printing presses, to typewriters, to word processing software is also a story of efficiency.

Increase productivity: Increased efficiency and increased accuracy are a sweet spot for improving productivity. Software packages like Excel, Lotus Notes and Hyperion were designed to increase productivity within the field of finance by making data processing more accurate and efficient. Likewise, online trading sites like Bloomberg, E*Trade, Schwab and many others provide rich information about publicly traded companies. By presenting complex information, these trading companies make individual investors more productive because they can make informed decisions about buying and trading stocks.

Solve problems: More than 125 years ago, one simple invention accelerated productivity during the industrial revolution – the light bulb. It changed the world. Productivity was no longer subject to ambient light. It allowed employees to work before sunrise and after sunset without candles or gas lights. Today, many manufacturing facilities still run three eight-hour shifts every day to maintain 24 hours of daily productivity. The light bulb also facilitated greater exploration of our world, whether it was in the deep, lightless salt mines beneath Kansas or the vast darkness of the ocean floor. Innovation helps us do something new or differently. Smartphones allow us to talk to each other face to face, take pictures, play video games, and even light our way when a light bulb isn’t handy.

All of these systems have one thing in common.  Measurement. They have features that measure, monitor and manage (react).  Space X rockets monitor position and speed in the X, Y and Z planes.  Similarly, submarines and driverless cars, monitor position, speed and other vehicles (and animals including humans) that might crash into them.  Measurement and feedback loops are essential parts of these technologies.

Measurement Systems: A Valuable Option Among New L&D technologies

New technologies hit the market every year with value propositions to make L&D processes better, faster or more cost-effective. Learner experience platforms, adaptive testing, augmented reality, virtual reality and natural language processing are among the new categories of technology that are gaining traction in L&D.

Recent advances in learning technologies are focused on two main categories: learners and analytics.

• Learner focus: Learning Experience Platforms make it easier for learners to access content from multiple and varied sources; VR, AR, and microlearning make it easier to acquire skills faster with richer learning experiences; communities of practice improve access to knowledgeable and skilled colleagues across business units and around the world.
  
  
• Analytics focus: When L&D professionals present information to L&D and business stakeholders, the data they share is most often descriptive—means, percentages, bar charts, pie charts, trends, etc. This information is useful, but the next generation of analytics has arrived; the type of information (predictive as well as descriptive) and the way analytics process data are different. This new generation makes it easier to gain insights by connecting to live data streams and analyzing results in real-time. This process replaces the older methods of downloading data into Excel, Tableau, SAS, or SPSS and doing advanced analytics there.  Natural Language Processing efficiently reads, codes, and summarizes themes in comments.  R and Python are programming languages that apply advanced and predictive analytics (correlation, regression, analysis of variance) to provide deeper insights about relationships among variables and significant differences across groups. In this way measurement systems provide reports about the data faster and with deeper insight than current human analysts. They help create a more comprehensive picture that addresses efficiency, effectiveness and outcomes.

Often new technologies are attractive because, frankly, they are THE next new shiny thing on the market. With unlimited budgets and a technology sandbox to play in, L&D can test new technologies at will.  However, for most organizations, constraints limit our ability to access new tools.  Dave Vance, former CLO at Caterpillar and current Executive Director of the Center for Talent Reporting, had this to say about new technologies:

“If I had an extra $50–$100,000 to improve my company’s learning function, I would not spend it on AR/VR, AI or the next shiny thing. I would spend it on the tools and processes that would close the gaps in fundamental operations. Am I measuring, monitoring and managing the right things? That comes first.”¹

If we are to take Vance’s advice, it is worth considering how to leverage technology to measure training effectiveness.  Here some criteria to use for qualifying measurement systems.

If we are to take Vance’s advice, it is worth considering how to leverage technology to measure training effectiveness.  Here some criteria to use for qualifying measurement systems.

• Gather data
• Store and code it
• Process it
• Report it
• Automate analytics insights

Why should a measurement technology do all of these things?  If done well, the technology will gather metrics about the quality of training that align to business outcomes. Such information provides insights about training’s impact on productivity. A system that can automate the reporting process will also provide timely insights to help the process become more efficient. Such a system can also solve the problems of determining course effectiveness and prioritizing continuous improvement efforts. When the right information is collected, L&D can ask and answer questions like: “Is self-paced micro-learning more effective and efficient than instructor-led training for this course?”; “Is the curriculum leading to knowledge and skill acquisition?”; “Are learners applying new skills and improving their performance?” Measurement moves L&D from guessing to knowing.

Conclusion

During every budget cycle, L&D organizations must choose where to spend their resources.  New-to-market and highly specialized learning systems are often enticing tools that may (or may not) help improve skills and achieve business goals.  Measurement is a critical part of knowing whether development programs are working and if not, how they should be improved.  Measurement systems deserve priority to help L&D know, rather than guess, whether their programs are driving business results.

References

1. Vance, D. (2019). Remarks made during the Center for Talent Reporting Conference 2019.