Megan Smith wearing a VR headset on in an urban VR scenario. (Photo by Michael Bell)
There’s potential for emotions to run high whenever an RCMP officer responds to a call. Their decisions can sometimes mean life or death.
So how can cadets best prepare to de-escalate a situation when a suspect feels trapped, angry, and afraid while bystanders are near? How do they stay focused around distractions? How do they keep a level head when responding to potentially dangerous situations?
These are some of the questions the RCMP’s Greg Krätzig thinks about, and what ultimately led him to reach out to the University of Regina to study scenario training using virtual reality (VR).
Krätzig, the director of research and strategic partnerships at the RCMP Depot Division, says that every new RCMP officer must complete an intensive six-month training program at the training academy in Regina, Saskatchewan.
“While the program covers skills and academic training, it also provides realistic, evidence-based training that helps prepare cadets to keep our communities safe and secure,” says Krätzig.
He adds that while most police-public interactions are peaceful and positive, “unfortunately there are also situations that require the police to use force in a rapidly changing situation.”
It’s moments like these that necessitate exceptional decision-making skills, and this is the area that Krätzig and his research team are delving into.
The RCMP has used simulation technology (driving simulators and simulators for marksmanship and judgment training) for many years. But with the rapid development of VR and through discussions with other safety and security partners, Krätzig envisioned using VR as a meaningful way to deliver training that was not previously possible.
But now it is.
Krätzig knew that while the RCMP didn’t have the necessary expertise to work with VR technology, the University of Regina had a rich pool of talent. And that’s where he first met Megan Smith.
“We started talking at a University networking event called Digital Future. It provided an opportunity for professionals in various fields interested in technological research and development to swap ideas and, perhaps, to also develop collaborative partnerships,” explains Krätzig.
At the time, Smith, an associate professor of creative technologies with the University’s Faculty of Media, Art, and Performance, was working on a Social Science and Humanities Research Council-funded project where she was using new media art processes to build an augmented reality performance. It involved her riding a networked stationary bike through Google Street View across Canada.
“That was a large endeavour, and included developing educational kits along with workshops to teach people how to make their own networked bikes at home,” Smith explains.
Her experience in physical computing (building interactive art works from hardware and software),art, computer science, and engineering, was a perfect match for Krätzig’s vision for the RCMP.
“I’m always working on how to push visual and sensory experiences and he was looking at implementing virtual reality technologies to enhance the cadet training experience,” says Smith. “We met at the right time.”
A year after their first conversation, Krätzig says he reached out again to Smith once he received funding from the Canadian Safety and Security Program and Defence Research and Development Canada. “It was a perfect way to make our project a reality.”
VIRTUAL REALITY
Krätzig and Smith knew that VR could add an important aspect to cadet training. They imagined the cadets immersed in VR scenarios where they would have to respond to a host of situations via their VR headsets.
“Developing smart devices for the RCMP, and placing something in their hands that feels and performs like a real pistol or Conductive Energy Weapon will actually help place the trainees into that kind of immersive situation.”
Smith’s role was to build remote controls that would work in the VR world, but would look and feel like real weapons that an officer would normally carry on their belts.
“A lot of the VR industry is still putting regular remote controls in people’s hands to manage situations and scenarios. But a remote control doesn’t mimic what would be in an officer’s hands in a real-life situation. And that can break their presence and detract from the immersive experience,” Smith explains.
To overcome this obstacle, Smith wanted to see if she could imitate the weapons that the RCMP use in real life, as well as build their tactile assets into a VR remote control. She started by researching and developing the RCMP’s duty pistol as a virtual controller. Then, using a 3D printer, she experimented with the technology. Several iterations later, she produced a VR pistol that includes a functioning trigger. Now she’s working on adding additional weight to improve the realism.
Smith says, “It’s important to have a few physical cues in real life that translate into the virtual reality space in order to keep you entrenched in the virtual environment.
“You don’t need to replicate an entire space, but you do need a few sensory similarities to keep the cadet embedded in the situation.
“Developing smart devices for the RCMP, and placing something in their hands that feels and performs like a real pistol or Conductive Energy Weapon will actually help place the trainees into that kind of immersive situation.”
To do this, Smith installed specialized motion-tracking sensors in all the 3D-printed equipment she has built so that the trainees can see the tools move within the VR environment.
Smith also works closely with Rhett Krushen, a multi-media designer with the RCMP. He creates the virtual environments that the RCMP cadets are immersed in while using the weapons Smith has created.
These highly immersive visual environments are designed with sound and interactive characters. “When a user puts on a headset, they may find themselves in places like a drug house, a city corner or a rural setting,” Smith explains.
While a student interacts with the virtual world, Krushen can manipulate and change what happens in real time. “If a cadet is de-escalating a situation properly, the circumstances also change in accordance,” says Smith.
The virtual environments are designed to be as realistic as possible, and are even able to incorporate sensations such as wind, smells, and tactile feedback. For example, in a shooting simulation, the user would hear a gunshot, which would be accompanied by the smell associated with that.
“It’s not real life, but you don’t actually need to make it real to stimulate the sense of urgency and responsibility you require to react,” says Smith.
The end result is a realistic looking and feeling field-training experience.
But can VR adequately prepare someone for real-life situations?
Smith says it’s difficult to describe how immersive it is to someone who hasn’t put on a headset, but says, “Once you experience it, you understand how engaging and productive it is and what possibilities exist in VR for deep training.”
Krätzig agrees that VR has to be experienced to truly be understood.
“VR has the capacity to elicit powerful reactions,” he says. Describing his own experience using a commercially available VR horror game, he says, “My heart was racing. I got the shivers and the hairs on the back of my neck stood up. I was looking behind me because I heard things all around me – it’s what you would expect in the real world.”
(l to r) A chronology of VR-compatible pistol prototypes researched and developed by Smith. (Photo by Michael Bell)
Krätzig says there are many advantages to using the technology and that’s why the RCMP has begun to research reactions to VR scenarios.
“We have done some research with commercially available games measuring ECG (electrocardiogram) data and the physiological responses are incredible.”
“Building these tools has been like solving a puzzle. The pieces exist, and I’m doing the work to find them, then put them together to create a useful, realistic tool, in a way that hasn’t yet been done,” says Smith.
Krätzig says when they eventually put cadets in these environments, they will experience the emotional response they would experience in the real world. “The VR will add a lot more to their mental library when confronted with stressful situations.”
But for the VR scenarios to be effective training tools, the actual tools found on officers’ belts – pistols, Conductive Energy Weapons, and pepper spray – must be realistic. “The work [Smith] has done in developing the hardware has been remarkable. In fact, she’s light years ahead of industry in this regard,” says Krätzig.
To make them, Smith combines commercially available developer components with custom designed parts, fitting them into the 3D-printed weapons. Working with commercial VR hardware development kits and optical sensors, she is getting excellent object tracking results.
“At this stage of the design, I have also developed a way to actually see how cadets manipulate the trigger on their weapon.” This is a crucial part of the technology she’s developed as it allows facilitators to teach the cadets how to react better if needed.
“Building these tools has been like solving a puzzle. The pieces exist, and I’m doing the work to find them, then put them together to create a useful, realistic tool, in a way that hasn’t yet been done,” says Smith.
"She’s a recognized global leader who has done remarkable work on a shoestring budget,” says Krätzig.
However, high demand for the components has resulted in supply issues. But using gaming technology that already exists means that in the long run the cost of VR equipment will be much cheaper than conventional training methods. And more portable, too.
The whole setup runs off of a mid-range consumer gaming laptop.
“It’s not overpriced,” Smith says. “My goal is to have a kit that can go out to RCMP units across Canada for under $5,000, which includes the headset, VR weapons, and computer.”
She also says that training modules can be downloaded on site, allowing for more rapid and mobile training capabilities.
The VR training can also be incredibly responsive.
“I envision that as procedures or laws change, the RCMP virtual training can be quickly altered to keep pace, ensuring, for instance, that rural and remote areas receive training updates in a timely fashion,” says Smith.
Smith is also working on integrating user biofeedback into the system to measure physiological response, refine simulations in real-time, and identify problem areas, allowing trainers to look into how users manage their stress.
Now this innovative work is garnering Smith international attention.
Law enforcement agencies and partners worldwide have expressed interest in her technology. “I am very up to date on VR use within a policing environment, and the commercial products that are selling for hundreds of thousands of dollars cannot come close to what she has developed. She’s a recognized global leader who has done remarkable work on a shoestring budget,” says Krätzig.
Given interest in the team’s work from various international groups and professionals in other fields of study, such as medicine, Smith says she is now in the early stages of possibly commercializing the process. But, for now, preparing cadets is the primary goal with the VR training project.
Krätzig is duly impressed with what the team has already accomplished, and is excited for the future. “The VR training methods will be an effective complement to the RCMP’s current programming.”
And Smith agrees.
“I never thought I’d be doing this kind of work when I first came to the University of Regina. But being part of this project has been an incredible experience because it’s reciprocal: I do development work, then take it to the RCMP, who offer feedback, then go back to my lab and make informed changes.” Working directly with the RCMP allows her to thoroughly understand and quickly respond to their technology needs.
“This entire process really demonstrates how VR can lead to exceptional training experiences,” says Smith, who adds that her team will continue to work on creating smarter tools, which ultimately will allow for more effective instruction and better learning experiences for everyone, now and into the digital future.
A doorway to culture
Reconnecting with cultural and traditional ways of knowing and being is increasingly seen as a significant part of the healing and learning process for Indigenous peoples, whose culture has been historically and systemically oppressed through colonization.
Read More
Funding
Funding U of R research with impact
Read More