In another article (“Boosting leadership coaching with Digital Role Play“) I already explored how powerful SkillGym can be when it comes to supporting executive and leadership coaching activity.
We have seen that there are several ways to leverage artificial intelligence (A.I.) for the purpose of upskilling managers in conversations during and in between coaching sessions.
In this article, we will explore case studies to see in practical terms how professionals around the world are already taking advantage of this amazing solution.
CASE STUDY 1: LEVERAGE DIGITAL TOOLS TO SHOW MEASURABLE IMPACT
Joseph is an executive coach dedicated to helping leaders grow their interpersonal skills.
He is 46 years old, a professional certified coach and has over 10 years of experience in the field.
He mostly works with companies belonging to the Fortune 500 and he coaches executives in their forties.
His coaching format is one of the most classical:
- Define the plan by investigating the “as is”
- Set goals and then work on the various aspects of the plan by listening
- Ask questions
- Challenge with dedicated role play sessions and support growth with powerful questions
His traditional means of measuring the outcome of his work is by conducting either a structured feedback session or, where possible, a 360 focusing specifically on the area of interpersonal skills.
Each case is a little different, of course, but this is how he describes his most typical approach.
The initial indicators that something was changing began around nine months earlier, when some of his clients started to ask with increasing frequency and a common thread in their requests that, “They needed to see the needle move,” said Joseph of a commonly occurring phrase.
The most common requests were about finding a way to relate his activity to some performance indicator that could be tracked along the way. “At the beginning it was only one or two…but then it went on the rise,” he continued.
“At the beginning it was only one or two…but then it went on the rise.”
When those requests started to become more than just a wish, he realized that he needed to put some thought into how to shift the way he was delivering.
He took the subject seriously regarding a coaching program that was oriented more toward the “move the needle” idea. He wanted to be able to identify a balance of significant indicators to be defined, agreed upon and consistently monitored by means of some sort of digital tool.
The initial idea was to find a way to be able to steer the program along the way based on the signals coming from such metrics and to be able to simultaneously show positive and encouraging progress together.
He needed a more predictable final outcome than from the traditional ways of evaluation in a coaching program.
Since Joseph was deeply into interpersonal skill development and was absolutely convinced of the importance of role play as a coaching strategy, he researched in this field for techy tools that could help.
His choice was for SkillGym due to its powerful way of delivering a library of stories to play with and enough data to be extracted from the digital tracking of its usage.
In this way, Joseph could simultaneously provide the coachee with consistent practice (since, according to him, improvement comes from the application of knowledge to practical situations) and collect enough information to build the metrics needed to “move the needle.”
In fact, he noticed that the platform was providing many types of different data, even more than what he could possibly handle.
And certainly, more than what he needed for his first attempt with this new strategy.
Thus, he decided to focus his attention on just two key metrics that seemed suitable for providing meaningful information about the impact of his work.
The first was the level of CONFIDENCE with which the coachee was dealing with the conversation in the role play and the second was the level of AWARENESS with which the coachee was able to judge his own performance against the quality measured by the platform.
The way he packaged the Confidence measure was quite simple: the higher the overall score of the digital role play at the end of the conversation, the higher the confidence in the task.
He just made sure that such score was in fact measuring the quality of the impact of the coachee’s behaviors on the approach of the virtual character.
As for the AWARENESS, Joseph thought about integrating a certain number of questions to be asked to the coachee at the end of each role play, so as to compare his self-assessment to the evaluation of the platform. The higher the match, the higher the awareness.
After a few weeks of trial, Joseph was able to pivot his traditional format, making room for a consistent ritual based on practice, measurement and adjustment of the exercise.
The idea was basically to define the main goal with the client and then link it to the two indicators of his program: confidence and awareness.
These would then be measured consistently across the sessions by asking the coachee to play specific role plays that were scheduled in between each meeting.
During the coaching session, Joseph and the coachee would discuss the outcome of the simulation by always referring to those two metrics and comparing them with two different benchmarks.
- One was the result achieved by the same coachee during the previous session (the so called “self-benchmark”) where the scope was to measure a trend of improvement.
- The other one was a comparison with the average result achieved by a cluster of comparable coachees, as registered anonymously by the platform across all users enrolled by several other coaches in the country.
This second indicator served to set a baseline and indicate where “the rest of the world” was at.
After each session, Joseph and the coachee had enough data to help define what the next role play should be to sustain a challenge, thus dynamically adjusting the schedule of practice.
After 6 months from the beginning of the new program, Joseph was ready to collect some comprehensive feedback from the field.
As expected, comments were mixed and generally fell into two categories: Signs of appreciation and alerts of possible issues (most of these signals had already been collected and managed in some manner by Joseph along the way). But Jospeh used the final brainstorming to draw some important conclusions on how to better define this new approach.
One coachee became enthralled with the amount of data the platform was providing beyond the two key indicators chosen by Joseph and started to ask Joseph for help in learning how to influence such measures.
Joseph decided to clearly define his role here as a coach, not a trainer, for the coachee.
His reported thought was, “I am here to raise questions to help you find your way, not to explain my way to you,” in a wise attempt to delineate the fine line between training and coaching people.
Of course, some leaders complained about the extra effort of playing schedule role plays. To this, Joseph remarked that “moving the needle” requires two ingredients: metrics (of course) and, most importantly, effort to practice. “No pain, no gain.” After that, most of them found the time.
“I am here to raise questions to help you find your way, not to explain my way to you.”
CASE STUDY 2: LEVERAGE DIGITAL CAPABILITY FOR REMOTE PRACTICE
Melanie is 42 and she is an executive coach (professional certified coach) supporting emerging leaders and high-potential employees with career development advice.
Her coaching format includes two live coaching sessions of 90 minutes every month, over a cycle of six months.
Each session with the coachee always includes one role play focusing on interpersonal skills and, of course, the relevant debrief time.
For quite a few years, especially when working with emerging young leaders, Melanie had been receiving many requests about evolving the traditional way of delivering her coaching advice.
In particular, her clients were increasingly asking about two seemingly divergent needs:
- On one side, they wanted more practical activity, including more role play time to try to discuss situations. They felt that practicing was an efficient way to unleash new skills.
- On the other side, however, they were struggling to fit a meeting into an ever-busier schedule. They were asking to reduce the time spent together!
“Go digital” was a common request. They were used to accomplishing a lot of activities with the help of digital technology, which was helping them to compress more into their time while still being present, which was still not happening in their coaching sessions.
Melanie decided to try a different way of packaging her service by moving part of her activity online.
The idea was to cut by half her physical one-to-one presence while doubling or even tripling the opportunity for her coachees to practice. Of course, she knew that she still had to stay in the picture, which led her delivering added value by making herself available online to support the remote practice activity and other matters as needed.
The idea of looking at SkillGym was quite natural: She had to remove herself from the practice delivery of role play, otherwise she could never add more role plays without doubling her personal effort, which could never be sustainable over time. She decided to adopt SkillGym because of the three following features:
- She needed a flexible way to let her coachee book practice sessions along the way. She wanted something where her coachee could automatically receive invitations to schedule one role play and decide when to fit it in their schedule. At the same time, she wanted to make sure that, if the coachee did not show up for the scheduled role play, the platform was smart enough to send reminders and propose to reschedule.
- She wanted a solution robust enough to allow for comprehensive monitoring of the practice activities that the coachees were involved in. She wanted something that allowed her to review the role play at any time and to understand where any help was needed.
- Finally, she thought that a good remote collaboration system within the platform could allow for an asynchronous way to deliver powerful questions. There was no need to discuss a role play face to face.
It is quite interesting to look at how the coachee became engaged in the new remote and self-paced practice activity. Basically, the idea was that:
- The fictional character (let’s call him “Mark”) would interact with the coachee well before starting the role play.
- He would directly write an email like: “Hi, I am Mark, and I would like to ask you for a quick feedback session on XYZ. I know you are busy, so please click the link below to open a doodle with a few slots, let’s see if we can make it.”
- If the coachee did not respond, Mark would send another attempt.
- Finally, on the scheduled day, Mark would write a confirmation email like “Hi, it’s Mark… and it’s today at 5 p.m., just to let you know I’ll be there.”
- Sometimes if the coachee did not show up, Mark would write again to say, “Hey don’t worry, I guess you got trapped… let’s reschedule, I was quite busy too.”
The result was stunning: 85% of the time, the coachee was reliably there on time. And for another 10%, just a second try was enough to find the right fit in their “busy” schedule.
The majority of the coachees explained to Melanie that they felt obliged to respect an appointment. They were acting as if it was a real schedule with a real person! Very efficient.
Also, it is remarkable how certain smart features of SkillGym could solve the fact that Melanie was not there when the role play was played.
She was able to access a recorded version of the interview, move through it and, with the help of augmented reality, review any blind spots in the conversation.
Blind spots may be aspects such as the reaction to stressed body language, the way a certain behavior was applied or even the recurrence of specific patterns and approaches.
It took seven minutes on average to open a role play, review it, find a spot and post one or two questions on the chat board.
The coachee would be notified that a message was posted and access it when they were available, read the question and reflect on their own. The option to reply as needed was available, of course.
In this way, Melanie could manage up to three times the number of role plays with the same amount of effort without needing to block time slots in both her and the coachee’s schedules.
In the end, her new format became very popular with her clients, especially the millennials.
She reduced her in-person sessions down to 1 per month with the same duration of 90 minutes.
During that time, they both had more time to discuss how to apply certain behaviors, as witnessed in the role play, beyond the boundaries of the specific context.
She increased the quantity of practical exercises without interfering with her leaders’ busy schedules and she was still perceived as available and helpful when needed, with even more laser-focused powerful questions working as eye-openers on blind spots.
Needless to say, the comments coming from her clients were very enthusiastic: EFFICIENT was the most popular keyword. They felt like their time was more valuable.
“GAME” was another popular keyword.
The Digital Role Play was perceived as relaxing time during which they could learn and be entertained at the same time.
There were, however, two issues that Melanie reported to have encountered in the process. And apparently, they were two quite divergent issues.
- First, some of the coachees started to think that the role play was rich enough with information that “they could even go alone” believing “I don’t need a coach anymore.” Ultimately, however, it was not what played out. In the final review, she was rated very high with regard to remote support, and for the question, “Do you think remote support is a key element of your online experience,” the score was also high.
- Second, many other coachees were asking for more asynchronous help, sometimes forcing Melanie to turn her advice session into a longer remote discussion. But it was always focused on the subject and, in the end, Melanie says, the possibility to differ the answer made it easy to keep the pace.
To summarize, Melanie gave us the following very simple indeed claim to define this shift to “Blended Coaching,” she did nothing more than moving online those activities that, once well supported by technology, did not require her presence anymore.
As an aside, it is interesting to note that coachees reported in the final survey that, on average, they felt much more comfortable with self-paced role play not only in terms of practical organization of their time, but also from a point of view of lower pressure perception when making mistakes.
What is important, however, are the three key lessons that Melanie learned, which helped her to keep solidly on the path forward :
- The coach can’t be cut out: coachees still have the need for discussion
- Remote practice can be fun
- Sometimes, real time isn’t necessary: waiting a day for my review of the role play is OK
CASE STUDY 3: LEVERAGE DIGITAL TOOLS TO DEVELOP FEEDBACK SKILLS
Tom is a professional leadership coach with 3 years of experience in Digital Role Play.
Tom was assigned to Kate, a middle manager of a mid-size company, in charge of a team of 20 people.
He was called because the annual performance appraisal suggested that Kate was below average in her commitment and feedback skills.
Her reports were much less positive than the other colleagues in the company in terms of Kate’s commitment to providing on-time and straightforward feedback about their activity. Kate was aware of this situation and quite aware of the reasons behind this.
Some of the things she told Tom during their first session were: “Often…I know I should give feedback…but I always find other jobs to do” and “I don’t feel comfortable.”
She basically reported to Tom that giving feedback was somehow outside her comfort zone.
So, Tom and Kate decided on one specific goal for their coaching sessions: helping Kate become more confident, expand her comfort zone and include the key activity and responsibility of regularly providing her reports with consistent feedback on their performance.
Tom decided to work on this subject by introducing SkillGym to let Kate practice with typical performance feedback situations and to discuss these conversations during their coaching sessions.
“Often…I know I should give feedback… but I always find other jobs to do” and “I don’t feel comfortable.”
Tom’s idea was simple: Confidence comes through practice and role play represents a very good form of “safe” practice to warm up and prepare for real life.
Tom was expecting two key advantages from using digital role play: Kate could play “alone” so that the safe environment would be perceived as even safer, and they would have a lot of material to work with by using powerful coaching questions.
Tom scheduled a consistent practice schedule for Kate in between coaching sessions. As a follow-up activity after six months, they met about their coaching program.
The idea was that a regular number of simulated feedback sessions would avoid overwhelming her normal schedule with too much exercise.
Kate ended up playing a total of 22 conversations over the six months, each one taking an average of 17 minutes to complete. Tom and Kate discussed eight of those conversations during their coaching sessions.
Before the coaching session, Tom spent around 10 minutes reviewing the metrics of the conversations played by Kate in search of interesting triggers to discuss during the session. During each session, either Kate or Tom proposed reviewing some passages of a specific conversation.
Tom’s method included:
- Listening to some passages
- Reviewing the underlying behaviors
- Asking Kate questions about why and how she applied those behaviors
- Letting Kate focus on the key points behind her decisions
- Stimulating her with questions that helped Kate to find a better strategy for the next conversation
On average, this role play debrief took around 20 minutes in each session where they decided to focus on a conversation.
The joint review showed a couple of recurrent elements that became powerful triggers in the debrief. The “virtual reports” of Kate consistently appreciated her approach, especially when it was oriented toward supporting them to do better.
On the other hand, Kate noticed that some of her specific behaviors, such as asking open questions, consistently triggered the same type of welcoming body language in the counterpart.
Of course, Tom leveraged those triggers with powerful questions such as: “What would you do to get those same reactions in your next role play?” or “Why do you think your report appreciated that question?”
Also, and this is very important, Tom started to ask if the same reaction was emerging during Kate’s real-life feedback sessions, which were becoming increasingly frequent.
In fact, as Kate’s confidence grew (see the blue line over the months, crossing the evolving perceptions of Kate as noted by Tom) she was involved in more occasions with real-life structured feedback sessions with her reports (the green bars).
In fact, the more Kate was involved in building her confidence through role playing, the more Tom noted two different types of perceptions and comments coming from Kate. She was more confident because she became aware that her activity was perceived as useful from her reports during the conversation.
Additionally, she appeared to have increasing control of the situation, just like if playing the role play on a consistent base was building in Kate’s mind a collection of Deja-vu situations that she was becoming increasingly familiar with.
The more she played, the more she knew what to expect in the next conversation. According to Kate, this exercise was “One accelerator of experience” for her.
Tom, who was still providing services to the same company, ran into Kate again later on and he was happy to know that the following year, her performance appraisal highlighted a much more confident Kate with regard to her skills and commitment in giving consistent feedback to her reports.
According to Kate, this exercise was “One accelerator of experience” for her.
Tom and Kate reported three main elements of Tom’s strategy to be relevant to the result:
- The first was consistent practice. Kate noticed that, even though at the beginning she was somehow “forcing” herself to find the time to play, the more she played the easier it was for her to consider this task as something she could manage.
- The second was the use of A.I.-based role play, which gave Kate the feeling that even playing the same role play more times, each time something different was going on. This, she said, helped to keep striving for the result, instead of playing using a known plot.
- And finally, the ability to reflect on what happened looking at the situation from outside. With Tom’s help, Kate learned to visualize where she could improve and, most of all, where she was providing value to her reports.
At the same time, Tom reported one main issue that at the beginning risked impacting his strategy. In fact, Kate often told Tom that she had no time in her busy schedule to play.
After the initial few weeks…when Kate actually didn’t play at all, Tom decided to ask Kate to consider each Role Play as a real feedback session and to schedule it on her weekly busy schedule.
This approach changed the situation. From that point on, Kate planned each conversation with 2 weeks advance notice, which meant those conversations became a part of her routine.
Tom learned to stress the idea of scheduling virtual conversation on his coachees’ schedules right at the beginning instead of suggesting free practice.
CASE STUDY 4: LEVERAGE DIGITAL TOOLS TO RAISE SELF-AWARENESS
Linda is a leadership coach working inside a large corporation.
She has over one year of experience using Digital Role Play and she reported a very interesting case study with Jeffrey, a first-time manager in the same corporation, in charge of a team of 12 people.
She was assigned to Jeffrey as part of his induction activities. Even during the first coaching session, when Linda and Jeffrey played one face-to-face role play, Linda noticed that Jeffrey had little experience with Leadership Conversations.
In fact, in the debrief, Jeffrey had the tendency to overestimate his own performance, while Linda noted several weaknesses that he did not. Some of the quotes by Jeffrey that Linda noted: “I did well, she doesn’t want to understand,” justifying why he did not manage to reach the conversational objective; or “I have my way,” “I know what he needs.”
All in all, it was clear to Linda that Jeffrey had a problem with self-awareness regarding the real outcome of his critical conversations.
So, they decided to focus on this issue as one of their objectives and to find a way to help Jeffrey develop a more critical approach to his way of judging his own performance and behaviors.
Linda decided to work on this subject by introducing SkillGym to let Jeffrey practice with critical conversations, with the specific goal of paying attention to the difference between Jeffrey’s perception and that of the virtual report.
In fact, SkillGym made it possible to listen to the virtual report while debriefing with a colleague about the conversation he had with Jeffrey.
On top of that, Linda would dig inside some parts of the conversation using the replay features to review the details of Jeffrey’s awareness of specific behaviors. Linda decided to use SkillGym mainly in face-to-face sessions.
She wanted to observe Jeffrey’s behaviors on her own during the simulation, to have more elements of discussion and to make up her mind about his approach to provide her with a clearer vision to ask interesting questions during the joint review.
The method was very simple:
- Linda would present the situation.
- Jeffrey would take a few minutes to prepare the meeting and then play the meeting on his computer.
- Linda would attend discretely.
- After the conversation and BEFORE getting feedback from the simulation, Jeffrey would face a structured self-assessment time.
- The digital role play selected specific moments of the conversation, asking Jeffrey to reflect on his own approach and performance and to rate them on a scale of 1 to 4.
- Immediately after, Jeffrey would listen to the opinion of the virtual report on the same topics.
After this exercise, which lasted on average 15 minutes, Linda would take another 10 minutes discussing the outcome with Jeffrey.
She started asking about Jeffrey’s perception on the virtual report’s feedback and why it was diverging from Jeffrey’s self-evaluation.
This part was very important because it forced Jeffrey to compare his position with that of an objective tool.
Then Linda would expose Jeffrey to some metrics coming from the system.
In particular, the Digital Role Play calculated the AWARENESS INDEX, measuring the distance between the two evaluations (Jeffrey’s and the simulation’s). This index was taken by Linda and Jeffrey to measure their progress, with the goal in mind to move from an initial 52% to a more reasonable 80%.
Then Linda would focus on some specific passages from the role play and by replaying the conversation, she asked to Jeffrey to explain the relationship between some of his behaviors and the relevant impact in the virtual report.
This was mind blowing for Jeffrey. He could:
- Frame the situation
- Focus on what happened from a third perspective, and
- Start to see the point of view of the other person
Specifically, Linda found one interesting pattern to work on by asking a series of powerful questions. She helped Jeffrey to understand that most of the time, he was receiving one answer from the virtual report that was intended to be one example to explain the report’s opinion. Jeffrey tended to consider it as a mere justification.
At the same time, triggered by this perception, Jeffrey tended to answer harshly and, would still consider his own answer as the right and balanced way to refocus his report’s priorities.
Then, when the counterpart manifested a frustrated reaction, Jeffrey’s interpretation was that of an attack. Once Jeffrey understood this recurrent pattern, it became easier for him to focus on certain perceptions and work on them to rebalance his point of view.
After just a few sessions where Linda and Jeffrey applied this coaching strategy, Linda started to notice a shift in Jeffrey’s approach.
In fact, as a first step, he started to improve his skill of self-evaluating his overall performance. Later on, this ability evolved as Jeffrey developed a more critical approach regarding the details of his own behaviors.
Jeffrey became increasingly in tune when listening to the weak signals coming from his virtual report.
In five months, Linda was happy to notice this result
Jeffrey’s appraisal on his own performance during critical conversation was much more mature and almost matching her evaluation.
Linda and Jeffrey reported that three main ingredients from SkillGym impacted the result:
- The first was Linda’s strategy of “reflect and listen.” In fact, she asked Jeffrey to self-evaluate his performance and, immediately after, to compare such evaluation to that of the simulation. It was the starting point for change. Noticing a diverging score was the way to look for a solution.
- The second was the use of video-based role play. Part of Jeffrey’s commitment was that of investigating the body language of his counterpart in search of weak signals showing a diverging perception of the quality of the ongoing conversation. Playing a digital role play, where the counterpart was a professional filmed actor, was a great element in this exercise.
- And finally, the opportunity to reflect on what happened; reviewing together the situation from outside helped Jeffrey to visualize where he could improve and, most of all, where he was misperceiving the reality.
When asked about lessons learned, Linda reported that she faced the following issue with Jeffrey in the beginning: He was trying to justify the mismatch in evaluation score saying that, “Anyway this is not a situation I would normally experience.”
Something that showed he would not accept the evaluation of the Digital Role Play.
Linda decided to ask Jeffrey to simply start from that situation and immediately after, find comparisons to real life situations that were similar.
Jeffrey started to notice that, no matter what, he could see a lot of similarities between that virtual situation and those of his real life daily experience. Linda learned that it was much better to use this approach from the beginning in order to leverage a de-contextualized situation to reconnect to real life immediately after.
What’s next
Wow, what a long series of case studies. Sorry if I went too long, and thank you for staying with me so far! 🙂
If you are searching for a Digital Role Play solution that meets your needs, please check out our website, which contains plenty of inspiring content including pre-recorded webinars and articles. Of course, we would welcome the opportunity to continue this conversation with you; all you need to do is book a 1-hour discovery call with us.
Enjoy the rest of your day.
