By: Tim Schwamb (SPC) and Cindy Currie (SPC)
The PCI SAFe® Training Team recently held both a SAFe® Product Owner/Product Manager 5.0 course and a SAFe® Scrum Master 5.0 course virtually for one of our government customers, and we wanted to share some of our successes and lessons learned.
Overall, the classes went extremely well – better than we anticipated – because we spent a significant amount of time planning upfront. As such, we’ll hit six vital points: The platform, the breakout activities, end-of-lesson quizzes, the PI Planning simulation, learner feedback, and post-training coaching.
The Platform: We use Microsoft Teams (our inhouse platform), although we are aware that Zoom has been used extensively by others. One significant advantage to Teams is that you can set up all of your virtual rooms, including breakout rooms, ahead of time. Learners are assigned to a breakout room before the course begins. They can then log in to both the primary classroom and their breakout room and switch between the two rooms quickly by placing one of the rooms “on hold.” We found Teams to be very powerful, although using it is not as intuitive as Zoom, especially for government students. We also found that Teams worked best when students logged in with a personal e-mail address and personal computer rather than using a government e-mail address on a government computer through the DoD VPN. Thus, the “Tech Check” session we delivered beforehand proved critical to get everyone’s technical issues worked out. Several of the problems we saw involved the students not following the technical set-up steps we sent to them, but there were other issues as well. Doing the Tech Check session the day before the class ensured class could start right on time the next day.
Breakout Activities: While some of the activities suggest discussing in pairs, we divided up all of them into either whole-class discussions or the pre-designated breakout session teams of 5-7 (the same groups used for the PI Planning simulation). This tactic worked well. Further, we placed a dedicated instructor in each breakout room to act as an activity facilitator so that students didn’t lose focus on what they were supposed to do during each activity. We spent a considerable amount of time ahead of the class developing activity stubs for each of the breakout sessions because SAI’s Remote Training Aids weren’t out quite yet. Even if using the SAI stubs, however, it’s important that each instructor fully understands each activity so they can lead their breakout team through it effectively. We also had each breakout room instructor/facilitator share out their screen with the activity stub and fill in the information provided by the team. This maneuver eliminated the need for the learners to find the stubs and download them, thus ensuring that we were able to use the short timebox for the breakout activities more efficiently.
End-of-Lesson Quizzes: In an actual classroom, it’s relatively easy to gauge how well learners are grasping the material through their nods or puzzled looks. That is nearly impossible in a virtual environment, so we developed our end-of-lesson quizzes using Survey Monkey. We included basic questions to see whether learners grasped the basic concepts, as well as more comprehensive questions like they might see on the SAFe® exam to see if they could begin putting multiple concepts together and applying them. After learners completed each quiz, we used Survey Monkey’s analytics to see average score, high-miss questions, and the like so we could go back over concepts not well understood. Learners completed these activities in class (we added timeboxes to our class schedule to do this), and we reviewed the results with the learners. We then gave learners the reason why specific answers were correct and others were not correct to help bolster their understanding of the material. (This also gave learners an insight into choosing the “best” answer to a question when more than one answer seems correct.)
PI Planning Simulation: By far, the most challenging portion of teaching a virtual SAFe® class was the PI Planning simulation. We used Rentouch’s PI Planning App for the platform and included their training videos in the course welcome e-mail, along with a training session of our own during the Tech Check session. That preparation helped significantly, but navigating a new tool for the first time is still tricky. Throw in a scenario that is also new, and there’s a lot for the learners to take in. They struggled to stay within the timebox and get a draft plan with PI Objectives, Risks, and the stories laid into the iterations. Discussion between teams regarding dependencies was non-existent, although we are actively working on how to facilitate dialogue between groups for future deliveries, including the possibility of working with the PI Planning App’s built-in Program Board.
Nevertheless, the learners were able to understand the point of PI Planning and get a feel for how it works in the end. We suggest giving out the Features and starter stories to the class and then spending extra time before the breakout to ensure the scenario is reasonably understood. We also recommend extending the 40-minute timebox for Breakout #1 to 60 minutes to provide extra capacity margin to absorb the additional uncertainty induced in a virtual class.
Learner Feedback and Relentless Improvement: While we captured and documented feedback given to us in real-time by learners during class, we decided to get a bit more formal about it too. So, to formally gather learner feedback, we administered a short Survey Monkey survey at the end of each day. The survey included three questions: 1) What went well today? 2) What could have gone better? 3) What suggestions do you have for improvement? Overall, the feedback was very positive and constructive, and we were able to make improvements daily based on it.
We reviewed a summary of their feedback with learners at the beginning of each new day and let them know what improvements we were putting on that day based on their feedback. And, we also captured our observations daily on things that we wanted to improve, so we incorporated as much as possible into each new day.
Post-Training Coaching: The training team is now holding once-a-week post-training coaching sessions for our students in each of these classes. We designed the sessions to answer any questions they have regarding the role-specific SAFe® certification exams. We also spend time working through a practice exam together so that students gain insight into parsing queries for keywords so that they understand what the question is genuinely asking them and can zero in on the correct answer. Students are finding these sessions extremely helpful and appreciate the extra support.
We’re looking forward to hearing about all the newly SAFe® 5.0 certified PO/PMs and SSMs!
Overall, we are delighted with how the classes went (and our learner feedback supports this) and look forward to more virtual classes while in-person classes aren’t feasible. Each one will only get better as we relentlessly improve! And we are already working on our next course for virtual delivery! Have you held a virtual SAFe® class? What lessons have you learned?