MXT signed the UK Government’s first ever Immersive Technology Services framework agreement in 2017 to provide all things Virtual, Augmented, and Mixed Reality to Highways England. Training using immersive technology offers the opportunity to immerse trainees, such as new Traffic Officers, wherever they might be and to train them on a limitless number of highly realistic scenarios. Emergency Services Times (EST) spoke to Toby Pettinger, the Managing Director of MXT, to find out more.
Emergency Services Times (EST): What are the driver training requirements of Highways England and how are MXT fulfilling that need?
Toby Pettinger (TP): Highways England employs around 1200 Traffic Officers to maintain the safety and satisfaction of road users and the free flowing of vehicles. MXT are responsible for developing and delivering a wide range of Virtual, Augmented, and Mixed Reality solutions across the organisation and have developed a Mixed Reality driving simulator for the Traffic Officer community. They need to be confident in achieving the expected high standards of Highways England in their daily objectives; spotting defects, assisting road users, monitoring traffic flow, and usually being the first responders to incidents. At this point they need to execute their operating procedures in a myriad situations, including the implementation of a rolling road block to bring traffic to a safe standstill. MXT enable this training, as if on a live motorway, without the need to be on a live motorway. We have developed a Mixed Reality driving simulator that can be used to do this on any chosen stretch of road, in any chosen weight of traffic, at any time of day, and in any weather conditions. This is training that otherwise cannot be done to the same level.
EST: What is the difference between Virtual Reality (VR) training and Mixed Reality (MR) training?
TP: Virtual Reality is the technology that enables the immersion of the user (or trainee) in a computer-generated environment and most of MXT’s work is in VR. Mixed Reality, still a fairly nascent technology, enables that same immersion but with the added benefit of the capability to blend it with reality. This means that, within the CGI simulation, the user can, for example, see themselves, their trainer, the vehicle in which they are being trained. Since this is pretty seamless it means that the CGI can be concentrated where it is really needed and the user has a real sense of presence in the simulation.
The specific headset that we work with is the Varjo XR-3. This is the latest Mixed Reality headset from the Finnish innovators Varjo. They’re the leaders in the field, working with fighter pilots at Saab, astronauts on the International Space Station, Volvo, and Kia. As well as us!
EST: The blue light sector understands the importance of realism in training – how close to ‘real’ do you believe a Mixed Reality training experience can be?
TP: You’ll have to judge when you come to see it! This is exactly what we spend a large proportion of our time on. Blending immersive sound, realistic visuals and physical simulation with elements of reality gets us pretty close. Nobody should pretend it’s the same as reality but I believe that we’re not far off getting to that point. The latest Graphics Processing Units (GPUs) and capabilities of key development engines such as Unity and Unreal are light years from where we were when we opened our UK studio, in 2016. Virtual and Mixed Reality capabilities are already leaps and bounds ahead of the training tools available to-date.
EST: Emergency services are familiar with driving simulators where the set-up is a chair in front of flat screens showing a number of driving scenarios – how does the MXT experience compare?
TP: The use of flat screens is portable, known, and less expensive than simulator centres or physical training on private roads. However, flat screens are a compromise; look to either side and the illusion is broken, even between the screens the graphics are often unnaturally distorted. There is a cognitive dissonance between what you see on the flat screen and what you are experiencing physically. And why limit yourself to set scenarios when flexible systems are likely to suit multiple requirements? Immersive simulation focuses the user entirely on the experience, feels realistic and intuitive, and so facilitates far better outcomes.
MXT build systems, such as responsive traffic and a nimble road editor, so that you can use a single simulator for as many scenarios as you want. And training can be data-driven; we use eye-tracking technology, embedded in the headsets, that can be used to review performance in terms of emotional response, distraction, focal points, and gaze duration. For the tiny proportion of people that still prefer flat screens we also offer flat screen functionality just in case…!
We partner with Motion Systems to build motion rigs, which we complement with seats, steering wheels and pedals familiar to the users. The rigs integrate well into our systems and provide pitch, roll and yaw, meaning that the user feels acceleration, deceleration, road noise, and the motion of turning left and right. I mentioned cognitive dissonance earlier; our aim is always to eradicate that and a good, well-integrated, motion rig is a critical component to address that need. As you’ll see they are fairly easy to transport and the simulator can work without them if the need arises.
“Immersive simulation focuses the user entirely on the experience, feels realistic and intuitive, and so facilitates far better outcomes.”
EST: What are the benefits of MXT training for Training Officers?
TP: As with many frontline services, and especially blue and yellow light, traditional training methods can only get you so far before it’s time to work operationally. Experience is invaluable but it’s not feasible to train new recruits in live situations to the degree that they are completely confident on Day 1. Training on a live motorway is costly and risky, creating a rolling roadblock on a motorway purely to train is definitely out of the question! Similarly, training police officers by handing them a firearm, and pointing them at an armed suspect, or a firefighter by handing them a bucket and directing them to a burning warehouse. Simulator centres, training towers, physical recreations of environments are all very expensive and limited in different ways. Simulation in Virtual and Mixed Reality is a powerful tool to add to a trainer’s toolkit. Visceral, realistic, flexible and endlessly repeatable in any location, the simulations that we develop offer experiential learning that builds confidence and is often the only option for training in hazardous situations without any risk.
EST: How might a Training Officer use the Road Editor to recreate his or her region’s road network?
TP: It’s as simple as identifying the stretch of road you want. Our Road Editor can take data direct from open source map data and, with a few tweaks from the team, you can be driving along it in a few days. It’s not yet as simple as drag-and-drop but it’s far quicker and more accurate than recreating a stretch of road.
EST: Does the system recreate other road users in its training scenarios?
TP: Yes we take traffic data from SUMO into our Traffic System and synchronise it with our vehicle assets (cars, lorries etc) and sound so that the user is driving in realistic traffic.
EST: How does the ‘traffic’ respond to the actions of the officer undertaking the training?
TP: Realistically. It will overtake, slow down, and, following feedback from Traffic Officers, even try to sneak past when you’re attempting a rolling road block. To-date one thing we’ve specifically been asked to exclude is the possibility of crashes so it doesn’t do that.
EST: Does the system allow for different environments as well as different roads?
TP: Absolutely. We have a variety of ‘Standard’ environments but for accuracy we are sometimes asked to build bespoke surroundings; Stonehenge was a recent build for the A303.
EST: What involvement did Highways England Traffic Officers have in creating training scenarios?
TP: Immersive technologies are a powerful training tool. Although MXT would consider ourselves experts in developing those tools we never presume to know how the training needs to be structured, delivered or evaluated and we’d never presume to know what the best methods will be. Working closely with the people that will be using the tools, the people that do the training and those being trained, is fundamental to their success. So early on in a project we identify those key stakeholders, we build user testing groups, and they are involved all the way along.
EST: How applicable is the MXT trainer to other emergency service driver trainers?
TP: We find that, not just within Highways England, but in many other clients the concept of ‘interoperability’ is important; how can what we’re building today be used tomorrow and how can it be used more widely than for today’s requirement? As such we build in a strategic way; we have developed a large asset database of animations, characters, buildings (and similar), the road editor and traffic system I mentioned earlier, and much else that can be used in many situations and for many possible requirements. In the context of the driving simulator any organisation requiring any kind of driving simulation in any setting can use this, whether emergency service driver trainers or anyone else that sits behind a wheel and drives a vehicle.
EST: Can the Mixed Reality training experience be used for training other than driver training? Eg Incident Management immersive training, for example.
TP: Yes. We have built experiences on phones, laptops, tablets, in VR, AR and MR headsets. We’ve built a free-to-roam 500m2 multi-user immersive training facility. And we’ve covered anything from health and safety, archaeology, customer experience, and animal handling, to gamification, employee engagement and STEM events. Incident Management is a critical requirement at Highways England for which we built the immersive training facility. This is for training multiple users in managing incidents on the motorway. The trainer has control of the scene and can add or remove issues, such as a spillage or a loose animal, before joining the trainees in the virtual simulation to train them.
