Whether you work in industry or education, your training initiatives should have well-defined goals, and produce measurable outcomes, for example, using learning analytics and data from your Moodle LMS.
By clearly establishing objectives, collecting data and analyzing trends, you will obtain a much clearer understanding of how well your training programs are performing. This will allow you to see which programs are working, what you need to improve, and how best to implement that change.
Increasingly, learning & development (L&D) organizations are coming to appreciate the power of a rigorous analytics strategy. A study published in 2016 found that 43.75% of corporations had increased their analytics budgets in the past year, 56.25% had kept it the same, and 0% had decreased it. This trend is only set to accelerate as analytics technology evolves and business leaders demand more accountability on training budgets.
Approaching your Learning Analytics Data
Ultimately, the goal of data analysis is to obtain actionable information. In order to do this, you need to establish a consistent framework that gives context to your raw learning data.
A complete analytics framework has two main components: strategy and techniques. Strategy refers to the high-level questions that will determine the key objectives, metrics and tools for your analytics program. Techniques are the best practices that will allow you to design and implement an effective program.
Let’s start with strategy. The three major questions you want to ask are:
1. Why do I need to analytics for my training program?
What are you trying to achieve in your training program? While all programs involve some element of knowledge acquisition or behavioural change, specifics can vary quite widely. For instance, your training program could be attempting to:
- Increase the success rate of outbound sales calls
- Reduce average time spent on customer support calls
- Maximize productivity in a manufacturing facility
- Ensure compliance with regulatory or industry standards
These are “big-picture” business goals that are not always straightforward to evaluate. As a result, many organizations lose sight of the original objective, and end up trying to measure too many things at once. By committing to a specific goal, you will be able to avoid this trap, and focus only on the metrics that matter.
2. What are the key metrics to measure towards my objectives?
The next step is to determine the key metrics that will allow you to measure progress toward your high-level objective. These metrics are known as “key performance indicators”, or KPIs for short. Some common learning KPIs include:
- Activity Scores
- Course Completion
- Grades
- Courses Purchased
- Quiz Attempts
- Site Logins
- Progress Tracking
- Time Spent Engaged
Often, to accurately evaluate progress, you will need to combine several of these metrics in some way. For example, your organization might need to ensure that a certain number of people are red-cross certified at any given time. This would require you to keep track of:
- People who have completed a red cross certification course
- The expiry date of each of those existing certifications
- The progress of a new batch of red-cross trainees, along with expected completion dates
This should allow you to monitor red-cross certifications, and predict the number of certifications in future time periods.
3. How to implement my analytics strategy?
Once you’ve determined the metrics you need to answer key questions, you have to decide how to implement your learning analytics program.
It is very important to have realistic expectations. Many analytics programs fail because they do not consider the organization’s current analytics capabilities. In particular, there are three common challenges you should watch out for:
- Technology: Do you have the tools required to collect, store, analyze and present your data?
- Data: Are you able to get access to all the relevant data in your LMS and other business intelligence systems?
- Skills: Do your people have the knowledge, judgment and experience to analyze your data effectively
Appropriately calibrating your expectations can mean the difference between a successful analytics program, and one that fails to get off the ground.
Techniques for Implementing Learning Analytics
1. Approach your learning analytics data through chunks of time
The learning process is fundamentally about change over time. So while single data points can be useful, they often provide more insight when analyzed as part of a larger trend. In practical terms, this means you will want to gather multiple data points between each evaluation milestone.
For instance, if you are evaluated quarterly, try collecting data every two weeks. This way, you won’t just have a single set of numbers to present to your business leaders. Instead, you will be able to accurately report your progress over multiple biweekly periods.2.
2. Make an ongoing data plan
To measure trends effectively, regular and consistent analysis of data is a must. Analytics involves comparisons of data over time. That comparison is only valuable if the same metrics are being measured at predictable intervals.
In order to ensure consistency, maintain an ongoing data plan so that you know what you’re going to measure, and when.
3. Have a vision for your data
Data is only useful if it is presented effectively to decision makers. So it’s critical to have a clear vision for how you are going to display, communicate and share your data. In addition, you have to ensure that your reports reach decision makers in a regular, timely manner.
This is one area where the quality of reporting tools really matter. A quality analytics platform will allow you to design beautiful custom reports, and share them automatically with stakeholders. Ideally, this will be done through a variety of channels, such as automated email, mobile device alerts, your LMS dashboard, and even social media.
4. Have a system in place for reflection and feedback
Analytics is a marathon, not a sprint, and you should seek to continually evolve your program over time. In order to do this, it’s important to regularly gather feedback from various stakeholders in your organization.
Be open to suggestions from a variety of sources. Often, the best ideas will come from people outside the L&D department. After all, once your trainees finish their training, they will start applying their new skills on the job, and who better to provide feedback than their co-workers?
What Gets Measured, Gets Improved
As an L&D professional, you are constantly striving to improve your existing training programs. Understanding how to use learning analytics and data effectively will supercharge that process. And establishing a strong analytics framework is the very first step in that journey.
For more advice on how to design a learning analytics framework, watch the full webinar, Capturing Data to Improve the eLearning Experience with Analytics:
