7 Steps for Successfully Implementing a Skill Gap analysis with Learning Analytics

Data gained from learning analytics can supercharge an organizational review, especially when looking for skill gaps or training needs. See how to apply learning analytics to your next analysis by using the ROMA framework.

Need some more background info? Read our article covering the basics of Why Your Organization Needs a Skill Gap Analysis.

Following our previous post on the what, how, and why’s of a skill gap analysis, here, we look at how learning analytics can be used to boost the results of any needs-based review, helping you to get the most out of your training program and learning management system.

The Rapid Outcome Mapping Approach (ROMA) framework provides seven simple steps for applying learning analytics to any process or program, regardless of industry. ROMA uses an evidence-based approach to guide you through initial data gathering stages to effect real and useful changes to policy. Below, we’ve applied the ROMA steps from The Essential Guide to Learning Analytics in the Age of Big Data to performing a skills gap analysis, creating a template for the next time you want to review the current state of your training or eLearning program.

Blog The gap between potential and actual market standing

The gap between potential and actual market standing (marketing91.com).

1. Clearly Define Your Policy Objectives

Identifying a skill gap is all about measuring an organization's ability to meet its objectives. To do this, you’ll need some kind of measure of performance. While this may sound simple, it’s important to determine the right measure for your specific needs—for the majority of businesses, this is likely to be financial, such as a profit margin. However, there may be more relevant ways to measure performance depending on organizational interests and the focus of your gap analysis. Turnaround time or customer satisfaction, for example, might be good indicators of service-based performance.

However you choose to define what counts as good performance, your objectives should support the overall goal of your company. This means that, even if your analysis is focused on a single department or team, the end goal of the analysis should be to promote core organization-wide goals, missions, or culture. Think about the best way to drive these large-scale objectives home—for instance, you might want to look at:

Communication patterns
Procedures
Documentation
Attitudes and perceptions
Behaviours and habits

A skills gap can appear in many forms, depending upon what exactly you’re analyzing. Often a gap will be related to workplace training, where closing the gap requires more detailed or technical resources. Alternatively, a skills gap might take the form of employee deficits, closed by hiring for new roles. Sometimes, however, gaps can be traced to specific workflows and processes, such as communication channels or sign-off protocols.

In all of these areas, accurate identification of inefficiencies or missed opportunities can lead to a reasonably easy solution. But there is always the possibility that the office process or policy you’re seeking to change will be an integral part of employee routine, which may be favoured by staff. In these cases, be prepared to defend the need for a policy change—both by consulting with staff throughout the process and by demonstrating your evidence.

Policy objectives created at this stage will become the yardstick by which you measure progress, so ensure they fully reflect company goals (G2 learn).

2. Map the Context

Mapping context means finding a coherent way of describing the space where your analysis is taking place—including the dynamics, relationships, and less tangible factors that may impact upon any changes you make. Understanding context helps to increase your ability to foresee issues, as well as resources and possible project champions.

As this insightful post on context mapping says, context is perspective-dependant. There may be a number of opinions and points of view on how different workplace factors do (and should) interact. Creating a map helps alleviate the confusion that can occur when different points of view come together, by giving everybody the same starting point.

Understanding project context is in many ways similar to mapping organizational growth, as well as succession mapping. In all three, the aim is to develop a picture of current affairs, in order to determine what changes would most help to close the needs gap.

Mapping context involves an examination of:

The context—Who, and what, will impact changes, and be impacted by new policy?

The evidence—What data supports the case for a skills gap?

The Links—Who within your organization forms a network around your policy? Who will champion or resist change?

One idea for visually mapping context (DBB).

3. Identify the Key Stakeholders

Stakeholders are those people—internal or external—who will be affected by the skill gap and will be tracked by your learning analytics program. There will be multiple stakeholders, from the teams and individuals who are actively producing work, to departments and clients downstream, who’s working relationships will be altered in some form or another.

Like mapping context, knowing your stakeholders helps you understand the true impact of organizational improvements, as well as consider how best to communicate the progress of your review.

Bear in mind that you may not be only looking for people here. A gap analysis can be performed on almost any significant entity, from running a content gap analysis, to the performance of a product or the reach of your brand. This means that stakeholders might be a program or product, as well as a team.

An important detail of skills gap analyses is that they can be performed on two concurrent levels: individual and company-wide. You can even initiate two gap analyses at the same time. A popular way of review performance is to compare a self-evaluated with an evaluation by the employee’s line manager or team leader. Knowing how employees and managers view any differences between performance and expectations helps inform future approaches to communication.

HR can work with team leaders to run analyses at multiple organizational levels (workable.com).

4. Identify Learning Analytics Purposes

When people ask what is learning analytics? What they’re often really looking for is a tangible example of how analytics can be implemented. So, to make it clear how learning analytics will help your gap analysis, it’s important to set out one or more valid uses for using learning analytics. For example, you might want to apply learner analytics to finding out more about:

Learner awareness

Monitoring and tracking
Research
Evaluation and planning
Reporting and communication

An analytics platform such as Lambda Analytics should be able to accommodate a range of goals and objectives, by tracking different aspects of how users interact with a training program or LMS. For example, by reporting on usage and completion rates across a variety of courses, you can begin to build a picture of how aware learners are of your resources.

There are many frameworks that can help you conceptualize your skill gap analysis, and develop an implementation strategy. A framework can be used to define current performance, or to target policy changes (smartsheet.com).

5. Develop a Strategy

An implementation strategy helps keep things on track by increasing your ability to predict and control how the skill gap closes. To do this, a strategic plan tracks the evolution of workflows with timelines, reviews, and deadlines (in some ways, a mini-gap analysis inside the official analysis).

The difficulty with developing a strategy to close skill gaps is that you’ll rarely be dealing with a static environment. As new needs or roles emerge, a gap might actually increase. With only 30% of CEOs saying their employees are able to easily adapt to significant organizational changes, it’s even possible that your strategy itself will require supplemental plans to help staff deal with change.

Statistics like these suggest that skill gaps are not usually indicative of an isolated issue within your organization, but a perpetual part of company life. Therefore, instead of expecting a single analysis to solve all issues, think of improvements in more holistic terms. Aim to develop an atmosphere of continuous learning, where resources are intended to support employees across their careers.

With attention becoming an ever more valuable commodity, today’s workplace is more equipped than ever to facilitate continuous learning. See how organizations are developing an engagement-based culture in this post: 2019’s Top 6 Workplace Trends for Establishing a Culture of Continuing Education.

6. Analyze Your Resource Capacity

Without the right resources to implement your findings, a skill gap analysis becomes nothing more than a hypothetical exercise (which might still be useful!) Especially when using learning analytics to provide insights on a skill gap, it’s worth taking the time to run some simple projections on what costs might be needed, and what might need to be acquired, to put your recommendations into practice. Not preparing resources can leave you short-handed when it comes time to act on your findings, which in turn can lead to your team losing the enthusiasm you’ve spent time building up. For an analytics-led gap analysis, most organizations will need to resources in areas including:

Project management and evaluation
Database development
Learning technologies administration
IT support and interface development
Analytics design and development
Learning analytics reporting
Institutional reporting and business intelligence

7. Develop a Monitoring and Learning System (Evaluation)

As discussed above, it’s important to understand skill gap analyses and learning analytics as continual elements of your review process. As your new content or policy changes begin to take effect, continue to monitor progress by looking for Key Performance Indicators (KPIs). Understanding progress towards your original goals will help you to fine-tune and tailor interventions, and may lead to insights that will form the basis of future analysis projects.

Three areas for KPIs

Training KPIs for User Engagement: It’s no surprise to find that, in general, the more learners immerse themselves in course content and communicate with their peers, the better their performance is likely to be. In turn, the more that users engage, the more beneficial LMS content becomes for the entire group of learners. Studies have shown that discussion forum performance is a strong indicator of overall online course performance.
KPIs for Learner Attendance, Completion, and Progress: Beyond user engagement data is a whole range of fundamental metrics that can be used to interpret learner performance. Average pass rates, quiz scores, test results, and completion times are all simple indications of learner progress. When compiled into an LMS dashboard, they provide a clear visual insight into the appropriateness of course content and the performance of students.
KPIs for Company-Wide and Bottom Line Performance: At the end of most business-related LMS courses is an important question: how much value is LMS training adding to our team? It may be possible to relate the adoption of an LMS program to overall profit and revenue KPIs. For example, you might ask if any quantifiable change in revenue correlates to a change in LMS learner performance and engagement. However, the bottom line KPIs most likely to be associated with LMS eLearning are HR-related: those connected to employee culture and performance. For example, have employees reported higher satisfaction ratings after using an LMS? Are business goals being achieved more frequently, more quickly, or with more consistency?

For a deeper look at KPIs: What Are The Most Important Training KPIs for eLearning?

Ready to keep climbing down the Learning Analytics rabbit hole? Here’s what we’ve got for you: