Orchestrator is our smart strategic alignment tool that helps L&D and HR teams connect knowledge initiatives with business objectives, by levereging AI technology. It brings visibility, structure, and purpose to your learning programs, ensuring every training investment supports a measurable business goal.
Getting Started with Insights in Orchestrator
Insights is a module within Orchestrator that helps you measure and understand the impact of your knowledge initiatives. It connects learner reactions and feedback to objectives and initiatives, giving you early, data-driven signals about how your learning programs are performing.
Insights is designed to answer questions like:
Are learners finding the learning content valuable or not?
Do learners feel confident applying what they’ve learned?
Which objectives and initiatives are progressing—and which are at risk?
ℹ️ Orchestrator and Insights are available for our Core customers. Reach out to your Customer Success Manager or Customer support to get started!
Impact measurement sequence
In the Insights page, we are introducing the full measurement sequence for knowledge impact in the entire chain from learning to impact. The measurement sequence contains following:
Reaction Indicators: These are early signals captured while learners are completing courses. They are based on learner reactions to course content and provide fast feedback on relevance and readiness.
Activation Indicators: Proof of real-work application, showing whether learners are applying knowledge in real work situations, validated across learners, managers, and capability owners. (Coming soon)
Outcome Results: Business KPI tracking over time, including baseline, current, and target values. (Coming soon)
Together, these layers create a clear line of sight from learning → behavior → business results.
The goal is to provide a shared source of truth for knowledge leaders and business leaders regarding how knowledge initiatives are affecting the business in relation to the business KPIs.
What You Can Do with Insights
With Insights, you can:
Get a quick overview of how objectives and initiatives are performing
Track impact indicators at different stages of learner journey
Follow score trends over time at objective, initiative, or course level
Identify which courses and initiatives drive positive or negative outcomes
Build a shared understanding of learning impact across L&D and business teams
Reaction Indicators
Reactions indicators are early signals of impact, while learners are doing the courses, they are the leading indicators of knowledge impact. The basis of these indicators are reactions of the learners on course material.
As part of Reaction indicators, we are deducing two scores as leading indicators: Learner Sentiment Score and Learner Confidence Score.
Learner Sentiment Score
The Learner Sentiment Score captures how learners react to the learning material throughout a course. While taking a course, learners are asked to evaluate their experience by answering a question “Did you learn something new?”, with responses given on a three-point scale. Learners respond to the sentiment question at the end of a course page, and these responses form the basis of the Learner Sentiment Score.
Learners respond to in the end of course page to the sentiment evaluation question, while the answers are used to deduce a Learner Sentiment Score. 👇
These responses are aggregated across all learners, weighted by response volume, and normalized to a 0–100 score.
The score is visualized in charts to help you:
Track sentiment trends over time
Identify which materials resonate with learners
Spot content that may need improvement
Learner Confidence Score
The Learner Confidence Score measures how confident learners feel about applying the knowledge and skills gained from a course. At the end of the course, learners answer an evaluation question: “Do you feel confident applying the knowledge and skills from this course? ", with responses given on a three-point scale.
These responses are aggregated and normalized to a 0–100 score, and visualized in the dashboard. The Learner Confidence Score provides early insight into learners’ readiness to apply knowledge and the potential for real-world knowledge impact.
In the end of the course learners are probed with a confidece evaluation question, whose answers are used to deduce a Learner Confidence Score👇
How Scores Are Calculated
Learner Sentiment and Confidence scores are designed to reflect the overall learner response, while taking into account how many learners have reacted and how they reacted.
A) Learner reactions used as input
Learner reaction are collected based on responses of learners that are enrolled to courses that are connected to initiatives. So if a learner is a part of a course but not added to an initiative, their response will not be taken into account in the Learner reaction score
Learners’ responses are collected from two sources:
On course pages: Learners can react to a question at the end of each course page, which is the data collected to calculate Learner Sentiment Score.
At the end of a course: At the end of courses that are linked to an initiative, learners are presented to react to a question, and those reactions present the data collected to calculate Learner Confidence score.
Learners respond using a three-point scale, where each reaction counts as a data point:
Yes = 3
Neutral = 2
No = 1
B) Aggregation across learners and courses
All learner reactions within the selected scope (objective, initiative, course or all content) are combined into a single aggregated score.
This means:
Courses with more learner reactions have a greater impact on the score
The score reflects the experience of the majority of learners, not just an average of courses
C) Normalization to a 0–100 scale
The aggregated result (originally on a 1–3 scale) is converted into a 0–100 index.
This normalization:
Makes scores easier to read and compare
Allows consistent comparison across objectives, initiatives, and time periods
A higher score always indicates a more positive learner response.
D) How to interpret the scores
Use scores as directional indicators:
80–100 → Strong positive response
60–79 → Mostly positive, with room for improvement
40–59 → Mixed response, worth investigating
Below 40 → Clear signal that content may need attention
Always look at:
Trends over time, not single data points
Reaction distribution (Overall rating)
Contributing courses or initiatives to understand what’s driving the score
How Scores Are Visualized
The impact scores are displayed in interactive charts that allow you to:
Follow score development over time
View aggregated scores at objective, initiative, or course level
Understand how individual initiatives or courses contribute to overall impact
Stacked charts make it easy to see which elements drive positive or negative results, giving L&D teams a strong, data-driven foundation for decision-making.
Current Score
When you land on the Insights page, you’ll see a current score displayed as a value out of 100.
This score represents the current aggregated score, based on learners’ reactions on learner material across the selected scope (objective, initiative or default state without filter). The score is converted into a 0–100 index to provide a quick snapshot of overall response.
Overall Rating
Below the current score, the Overall rating shows the distribution of learner reactions. This helps explain what’s driving the aggregated score and provides context behind the number.
Current score displayed as a value out of 100, in the image below being "80/100"👇
Dashboard
The insigths dashboard represents a set of charts and filters that help you explore learning impact data by scope and time. It allows you to visualize the data and filter out the data sets of interest by scope (Objective vs Initiative) or time.
Default View
When no filters are applied, scores are based on aggregated learner reactions across all objectives and initiatives.
Filtering by Objective or Initiative
Objective filter: Shows data from learner reactions across all initiatives and courses linked to the selected objective
Initiative filter: Shows data only from learner reactions within the selected initiative
Filters allow you to narrow focus and understand impact at different strategic levels.
Filter in the Insights page, allows users to filter out data according to objective or initiative👇
Charts
The dashboard is divided into two main sections, referring to Learner Sentiment Score or Learner Confidence Score. Each section contains charts that visualize how aggregated scores change over time as learners progress through courses.
Objective Score Chart: This chart shows aggregated score changes over time for each objective, based on learner reactions across all courses linked to initiatives of the objective. Use it to spot which objectives are improving or declining.
Initiative Score Chart: This chart shows aggregated score over time for each initiative, based on learner reactions across all courses connected to an initiative. It helps you compare initiatives and track overall impact.
Filtering by an initiative or objective provides an additional chart, "Stacked chart":
Contributing Courses - Stacked Chart: This chart is shown when you filter on a specific initiative. It shows how individual courses contribute to the aggregated initiative score over time, making it easier to see which courses are driving positive or negative learner response.
Contributing Initiatives- Stacked Chart: This chart is shown when you filter on a specific objective. It shows how individual initiatives contribute to the aggregated objective score over time, making it easier to spot positive or negative contributions from initiatives.
To get the most value from Insights
Do the following steps to get the most value from Insights
Ensure courses are connected to initiatives and objectives
Encourage learners to complete courses and respond to evaluation questions (course page reactions and evaluation question at the end of the course)
Use trends in the dashboards—not single data points—to guide decisions
Revisit Insights regularly to track progress over time