I recently conducted research at my time at University College London that answered this very question. My last two posts dealt with some of the dynamics involved in aviation risk assessment. More specifically dynamics that affect risk perception amongst aircrew and aviation security (AVSEC) experts. Now I will outline how I answered the question above.
The study was based on five scenarios which were based off actual onboard security events that recently occurred at a major commercial airline in Europe. The five scenarios were specifically designed to require the participants to conduct risk assessments of potential onboard security threats. Scenarios that required little risk assessment and would elicit a definitive response regardless who the participant was such as a breach of the flight deck door or an actual bomb detonation onboard were not used. Each scenario was also designed to have the participants think of different security considerations that would commonly be encountered on any commercial airline.
The participants in the study included pilots, cabin crew and aviation security experts employed by commercial airlines in Asia, Europe, Middle east and North America. A diverse range of ages and experiences was included in the group. The airlines that were represented included international short and long haul airlines as well as some regional carriers.
The main aim of the study was to determine if aircrew assess risk differently than AVSEC experts when dealing with a potential onboard threat. As suspected, the empirical results do suggest that aircrew perceive threats differently than AVSEC experts. The three questions in each scenario that most directly deal with risk perception (likelihood the incident represents an actual security threat, likelihood the incident causes severe injury or death to passengers or crew and likelihood the incident would cause severe damage to aircraft or other property) showed that there was a statistically significant difference in perception between aircrew and AVSEC experts using a Kruskal-Wallis H test in all scenarios except two of the questions in scenario five, which had to do with a passenger feeling uncomfortable about another passenger’s appearance.
In all five scenarios, AVSEC experts reported the likelihood of threat, likelihood of injury and likelihood of damage was much lower than both the pilot and cabin crew groups. All three groups were in agreement about scenario five though, which had to do with a passenger feeling uncomfortable about another passenger’s appearance, that it had the least likelihood of being an actual threat, causing injury or death or injury or causing severe damage.
When examining the question that asked if the participants considered the incident to be a false alarm, again the AVSEC experts indicated they would consider the incident to be false more so than cabin crew in all five scenarios and more than pilots in all scenarios except scenario five. All three groups considered a false alarm in scenario five the most. Cabin crews considered each scenario to be false the least with the exception of scenario two, which could be expected as it related to a disorderly passenger assaulting a member of cabin crew.
The findings suggest the significant difference in risk perception between experts and non-experts observed in other fields are also present in aviation. Aircrew, as non-experts in security, perceived security risk much greater than the AVSEC experts. Some of the pilot participants said they would divert the flight for all of the scenarios that occurred during flight. Twenty-one pilots said they would likely divert the aircraft in at least one of the scenarios. Some of those pilots did indicate their company policy would require diversion for scenario two, which was an actual assault of a crewmember. None the less, several pilots would take the extreme measure of diversion in the other scenarios.
Pilots made the following comments about scenario one “Divert, better safe than sorry”, “Immediately divert” and “When it comes to bombs, there is no false alarm to take a chance, I’ll declare an emergency (divert)”. One cabin crew even said “we would treat all threats with the same amount of caution regardless if they are legitimate or not”. This statement indicates there was little risk assessment done by that crewmember, as they indicated they would react the same way even if the threat were deemed false. Regarding scenario three, pilots made similar comments. One pilot said, “The flight would be diverted and federal officials would be called to investigate” and another said “Call company, divert”. Even amongst pilots there is a wide variance in their risk assessments. In scenario four, one pilot would refuse to fly, while another said they would merely “throw it (suspicious drawing) away”.
In contrast, AVSEC experts made comments such as “ensure the assessor has full facts as opposed to hearsay”, “verify”, “gather more information” and “use threat assessment guidelines” for scenario one. For scenario three AVSEC experts also repeated, “Assess in accordance with threat assessment methodology” and made comments such as “This type of incident is almost certainly a non-event” and “Suicide bombers don’t announce the action. Others seek attention and wish to disturb”. In every scenario AVSEC experts indicated they would engage in a methodical and established risk assessment procedure and base their actions or recommendations off of those findings.
The data confirms the dissimilar actions each group would take and suggest the risk assessment being done by aircrew and actions taken when it comes to potential onboard security threats are far different than that of AVSEC experts. To highlight this, one pilot stated, “As pilot in command it is my duty to ensure there is a safe outcome of each flight”, even though that pilot also stated that particular scenario was an “unlikely threat”. Once again confirming whilst the aircrew may be conducting risk assessments, those assessments are not always dictating the actions they take especially if they are false.
Other findings of this study showed similar differences between aircrew and AVSEC experts. In 33 out of 55 (60%) of questions measuring responses between aircrew and AVSEC experts, a Kruskal-Wallis H test showed that there was a statistically significant difference between the groups.
AVSEC experts indicated they were more familiar with all five scenarios than pilots and cabin crew, which was expected, as it is their area of expertise. Pilots reported being more familiar with all five scenarios than cabin crew, which could have to do with the higher average experience in the pilot group. Pilots and cabin crew both indicated they were most familiar with scenario two which had to do with a disorderly passenger assaulting a member of cabin crew. This may have to do with the fact disorderly passengers are a more common occurrence than the incidents in the other scenarios and there has been much media coverage of recent incidents involving disorderly passengers or air rage.
When the participants were asked if they would consider possible economic loss to their airline for actions they would take when deciding how to handle each scenario, there was also a significant difference between aircrew and AVSEC experts. Since pilots are the final decision makers onboard, this study was more interested in their responses. Although, it is interesting to note that cabin crew were more likely to consider economic loss to their airline when handling the incident than pilots in all scenarios. In every scenario except scenario two, AVSEC experts were more likely to consider economic loss to their airline than pilots.
When examining if pilots and AVSEC experts would be influenced by delays when deciding how to handle each scenario, all five scenarios showed a significant difference. Surprisingly, pilots were significantly less influenced by the possibility of delays than either cabin crew or AVSEC experts when dealing with a potential onboard threat. This could be attributed to the pilots training. There have been several high-profile aviation accidents where a human factors phenomenon known by pilots as “Get-There-Itis” has played a role. This is when a pilot’s judgment is impaired through a fixation on the original goal or destination, combined with a disregard for any alternative course of action (Velazquez 2018). Because of this, all pilots have been trained to overcome “Get-There-Itis” for safety reasons. This conditioning would also be carried over when dealing with an onboard security incident.
When it comes to the influence of passenger demands, only scenario five showed a significant difference between aircrew and AVSEC experts. Again it was surprising, aircrew would be less influenced by passenger demands in scenario five than AVSEC experts even though the scenario had to do with passengers feeling uncomfortable about another passenger. In all five scenarios cabin crews would be influenced more than pilots, which was expected, as they are the group that deals most directly with the passengers. All groups also indicated out of the five scenarios, they would be most influenced by passenger demands in scenario one, which dealt with the passenger that overheard another passenger speaking of a bomb.
In all of the scenarios except scenario five, the aircrews indicated the incident would have more than a minor effect on increasing their workload. As past research shows, any increase in workload, especially the pilots, can be detrimental to performance and can create safety issues during flight.
In all five of the scenarios, aircrew said they believed their training would be at least “somewhat helpful” in dealing with the incident. This finding is interesting considering the suggestion of past research that aircrew believe their security training to be inadequate. Whilst the aircrew in this study seemed to be confident their training is helpful, the evidence that aircrews assess and respond differently than AVSEC experts indicates that confidence may be misconceived.
Also, in all five scenarios, AVSEC experts believed their company SOP would influence the handling of the incident more so than pilots. This is not to say pilots were not cognisant of the SOP’s when dealing with the scenarios. In fact, thirteen pilots referenced “SOP’s” when describing how they would handle the scenarios.
The fact that pilots indicated they were less influenced by their SOP’s than the AVSEC experts is an interesting finding as it could indicate several things. Pilots either do not trust or completely understand security SOP’s, as past research suggesting the conflict between safety and security in aviation would confirm (Pettersen & Bjørnskau 2015), or pilots do not believe security SOP’s cover all of the scenarios put forth in this study.
Finally, in all of the scenarios pilots indicated they would be slightly to extremely influenced by advice from AVSEC experts when dealing with the situation. Interestingly, in every scenario, AVSEC experts’ responses were higher than pilots. Even the cabin crew’s responses were higher in every scenario, but scenario five. The responses by the pilots indicate they at least consider advise from an expert, although it remains to be seen exactly how influential that advice really is due to the fact both cabin crew and AVSEC experts had stronger average responses to that question. Nineteen pilots actually stated they would seek advice from or at least contact their airline’s security department when dealing with the scenarios in the study.
The topics I have previously discussed regarding the different dynamics involved in aviation security risk assessment played a factor in these results. The contradiction between safety and security in aviation and the dynamic between expert and layperson have significant influences on the risk perception gap between aviation security experts and aircrew. Now that we know this gap exists and some of the reasons why, I will discuss in the next post how we combat this risk perception gap.
Comments