Cognitive Reliability and Error Analysis Method (CREAM) is widely used in human reliability analysis (HRA). It defines nine common performance conditions (CPCs), which represent the factors that may affect human reliability and are used to modify the cognitive failure probability (CFP). However, the levels of CPCs are usually determined by domain experts, which may be subjective and uncertain. What’s more, the classic CREAM assumes that the CPCs are independent, which is unrealistic. Ignoring the dependence among CPCs will result in repeated calculations of the influence of the CPCs on CFP and lead to unreasonable reliability evaluation. To address the issue of uncertain information modeling and processing, this paper introduces evidence theory to evaluate the CPC levels in specific scenarios. To address the issue of dependence modeling, the DecisionMaking Trial and Evaluation Laboratory (DEMATEL) method is used to process the dependence among CPCs and calculate the relative weights of each CPC, thus modifying the multiplier of the CPCs. The detailed process of the proposed method is illustrated in this paper and the CFP estimated by the proposed method is more reasonable.
Reliability evaluation for large complex systems is of great importance. Reliability assessment involves examining various factors that could lead to a certain system’s failure or malfunction and estimating the probability of those occurrences. The outcome of the assessment can assist people in determining whether they can depend on the system to function correctly, or need to make modifications to enhance its reliability. Reliability assessment has received wide attention and should be conducted in many fields, such as the power industry [
Human reliability analysis (HRA) plays an important role in reliability evaluation for large complex systems. It qualitatively analyzes the impact of human error on system failures and quantitatively calculates human error probability (HEP), to reduce the occurrence of human failure events (HFE). HRA has become an indispensable part of the reliability evaluation for complex systems, such as nuclear power plants [
Cognitive Reliability and Error Analysis Method (CREAM), introduced by Hollnagel [
However, in the process of calculating cognitive failure probability (CFP) based on CREAM, the determination of common performance condition (CPC) levels mainly depends on the opinions of experts, which may contain uncertain information. How to effectively express and deal with the uncertainty of expert opinions is a problem that urgently needs to be solved. In addition, the classical CREAM does not take into account the dependence among CPCs. Therefore, the dependence of CPCs may be repeatedly calculated, resulting in overestimation or underestimation of the results.
Yang et al. [
Many researches were conducted to model uncertain information [
DEMATEL is an approach that uses a matrix to describe the relationships among elements of a system. Through indepth analysis of the logical relationship between every two elements of the system, it can calculate the total influence of a certain element on other elements and the total degree to which a certain element is influenced by other elements in the system, thus determining the causal relationship and the importance of each element in the system, which is widely used to extract key elements [
In this paper, we propose an improved CREAM method based on DS evidence theory and DEMATEL. DS evidence theory is used to express and process the uncertain information in the assessment of CPC levels. DEMATEL is used to model the dependence among CPCs and calculate CFP.
This paper is organized as follows.
CREAM’s quantitative HEP prediction methods include primary methods and extended methods. The basic method involves determining the expected effect of common performance conditions (CPCs) on the performance reliability to get a rough probability interval, while the extended method can obtain specific probability values. This paper mainly describes the extended method. The extended method divides human cognitive functions into four categories: observation, interpretation, planning, and execution. Each type of cognitive function has several failure types. Hollnagel explains 13 general failure types and the fundamental values and upper and lower bounds of the failure probability, as shown in
Cognitive function  Generic failure type  Lower bound (0.5)  Basic value  Upper bound (0.95) 

Observation  O1. Wrong object observed  3.0E4  1.0E3  3.0E3 
O2. Wrong identification  2.0E2  7.0E2  1.7E2  
O3. Observation not made  2.0E2  7.0E2  1.7E2  
Interpretation  I1. Faulty diagnosis  9.0E2  2.0E1  6.0E1 
I2. Decision error  1.0E3  1.0E2  1.0E1  
I3. Delayed interpretation  1.0E3  1.0E2  1.0E1  
Planning  P1. Priority error  1.0E3  1.0E2  1.0E1 
P2. Inadequate plan  1.0E3  1.0E2  1.0E1  
Execution  E1. Action of wrong type  1.0E3  3.0E3  9.0E3 
E2. Action at wrong time  1.0E3  3.0E3  9.0E3  
E3. Action on wrong object  5.0E5  5.0E4  5.0E3  
E4. Action out of sequence  1.0E3  3.0E3  9.0E3  
E5. Missed action  2.5E2  3.0E2  4.0E2 
CPC 
CPC name  Level  Cognitive function  

Observation  Interpretation  Planning  Execution  
1  Adequacy of organization  Very efficient  1.0  1.0  0.8  0.8 
Efficient  1.0  1.0  1.0  1.0  
Inefficient  1.0  1.0  1.2  1.2  
Deficient  1.0  1.0  2.0  2.0  
2  Operating conditions  Advantageous  0.8  0.8  1.0  0.8 
Compatible  1.0  1.0  1.0  1.0  
Incompatible  2.0  2.0  1.0  2.0  
3  Adequacy of MMI and operational support  Supportive  0.5  1.0  1.0  0.5 
Adequate  1.0  1.0  1.0  1.0  
Tolerable  1.0  1.0  1.0  1.0  
Inappropriate  5.0  1.0  1.0  5.0  
4  Availability of procedures/plans  Appropriate  0.8  1.0  0.5  0.8 
Acceptable  1.0  1.0  1.0  1.0  
Inappropriate  2.0  1.0  5.0  2.0  
5  Number of simultaneous goals  Fewer than capacity  1.0  1.0  1.0  1.0 
Matching current capacity  1.0  1.0  1.0  1.0  
More than capacity  2.0  2.0  5.0  2.0  
6  Available time  Adequate  0.5  0.5  0.5  0.5 
Temporarily inadequate  1.0  1.0  1.0  1.0  
Continuously inadequate  5.0  5.0  5.0  5.0  
7  Time of day  Daytime (adjusted)  1.0  1.0  1.0  1.0 
Nighttime (unadjusted)  1.2  1.2  1.2  1.2  
8  Adequacy of training and experience  Adequate, high experience  0.8  0.5  0.5  0.8 
Adequate, low experience  1.0  1.0  1.0  1.0  
Inadequate  2.0  5.0  5.0  2.0  
9  Crew collaboration quality  Very efficient  0.5  0.5  0.5  0.5 
Efficient  1.0  1.0  1.0  1.0  
Inefficient  1.0  1.0  1.0  1.0  
Deficient  2.0  2.0  2.0  5.0 
Step 1. Analyze human error events and determine which cognitive activities are involved. Then, each cognitive activity is analyzed to determine the most probable types of cognitive function failure in each cognitive activity, and their corresponding basic CFP from
Step 2. Evaluate the situational environment of each cognitive activity to determine the levels of CPCs.
Step 3. Calculate the cognitive failure probability (CFP) of each cognitive activity. Assuming that there are
where CFP
Step 4. Determine the total human error probability (HEP) [
where CFP
DempsterShafer evidence theory is effective to handel uncertainty, and has been extended to complex domain [
The mass
where
and the mass function of proposition C is
Dempster’s rule use the conjunction operation as its numerator and a normalization factor of
When making decisions, a belief function needs to be transformed into a probability function [
where
The basic theory of DEMATEL is introduced in the following steps [
Step 1. A group of experts evaluated the relationship between every two alternatives, resulting in a direct relation matrix
Step 2. The matrix
where
Step 3. The total relation matrix
The sum of the elements in each row of matrix
Step 4. The value of
To make the proposed method more intuitive and easy to understand, the steps of which are constituted in
Select several experts with expertise and experience in nuclear power plants to form a team to participate in the assessment. There are three experts involved.
Experts may not be completely certain about the level of CPC in a specific scenario, so their judgments are often accompanied by ambiguity and uncertainty. Evidence theory allows experts to assign different levels for a CPC and suggest ratios to represent the relative probabilities of different levels.
To express confidence in their judgments, experts use a scale ranging from 0 to 1, corresponding to seven confidence levels, where 1 represents “absolutely confident”, 0.8 indicates “mostly confident”, 0.6 indicates “fairly confident”, 0.4 indicates “only some confident”, 0.2 indicates “mostly not confident”, 0 indicates “no confidence at all” and other values (i.e., 0.1, 0.3, 0.5, 0.7 and 0.9) represent confidence levels between these seven levels. It allows experts to make flexible judgments in the face of uncertain situations.
Case  CPC level  Confidence 

Case 1  {Efficient}  1 
Case 2  {Efficient, Inefficient}  1 
Case 3  {Efficient}:{Inefficient} = 3:1  0.8 
Case 4  {Inefficient}  0 
Case 5  {Very efficient, Efficient, Inefficient, Deficient}  1 
The levels of each CPC constitutes a discernment frame. For instance, consider the discernment frame
where
For example, the BBA of Case 3 in
Based on the judgments in
Case  BBA 

Case 1  m{Efficient} = 1 
Case 2  m{Efficient, Inefficient} = 1 
Case 3  m{Efficient} = 0.6, m{Inefficient} = 0.2, m( 
Case 4  m( 
Case 5  m( 
For each CPC, the judgments given by the experts are fused, and the fusion rules are based on
After the fusion of BBA, the confidence of the final result can be obtained, as calculated in
In the proposed method,
The fused BBA can be used to calculate the probability of each level of CPCs using the above equation. The multipliers
Experts with rich prior knowledge converted the language tag variable into fuzzy numbers to evaluate the relationship between every two CPCs. These values describe the relationship between sets of paired CPCs, the bigger the value the stronger the dependence. If the value equals 0, it means that no dependence exists between these two CPCs. In this paper, numerical value (comparison scale) varying from 0 to 9 is adopted [
Dependence degree  Numerical value 

High  9 
Medium to high  7 
Medium  5 
Low to medium  3 
Low  1 
Zero  0 
Convert experts’ judgments into an initial input matrix in DEMATEL, known as the direct relation matrix. The relative weight
The value of
We adopt the processing method of the
where
The CPC multipliers
The basic principle of
Calculate CFP according to
CREAM defines cognitive functions as the basis for thinking and decisionmaking into four categories: observation, interpretation, planning, and execution. Each typical cognitive activity can then be described in terms of which combination of the four cognitive functions it requires. As shown in
In this study, “observation” in four cognitive functions and “Wrong identification” in failure types are taken as the assessment object.
Three experts participate in the evaluation, all of whom are selected in the HRA field and have professional experience and expertise in nuclear power plants.
According to
CPC 
CPCs  Experts  CPC level  Confidence 

1  Adequacy of organisation  Expert 1  {Very efficient}  0.8 
Expert 2  {Very efficient, Efficient}  1  
Expert 3  {Efficient}  0.6  
2  Working conditions  Expert 1  {Advantageous}:(Compatible} = 3:1  0.8 
Expert 2  {Compatible}  0.7  
Expert 3  {Advantageous, Compatible}  1  
3  Adequacy of MMI and operational support  Expert 1  {Adequate}  1 
Expert 2  {Adequate}  1  
Expert 3  {Adequate}  1  
4  Availability of procedures/plans  Expert 1  {Appropriate, Acceptable}  1 
Expert 2  {Acceptable}  0.8  
Expert 3  {Inappropriate}  0.5  
5  Number of simultaneous Goals  Expert 1  {Fewer than capacity}  0.4 
Expert 2  {Matching current capacity}  0.5  
Expert 3  {Matching current capacity}  0.4  
6  Available time  Expert 1  {Adequate}  0.6 
Expert 2  {Temporarily inadequate}  0.4  
Expert 3  {Adequate}  0.7  
7  Time of day  Expert 1  {Daytime}  0.7 
Expert 2  {Daytime}:{Nighttime} = 2:1  0.9  
Expert 3  {Daytime}  0.6  
8  Adequacy of training and experience  Expert 1  {Adequatehigh experience}  0.8 
Expert 2  {Adequatehigh experience}  0.7  
Expert 3  {Adequatelow experience}  0.8  
9  Crew collaboration quality  Expert 1  {Very efficient}:{Efficient} = 1:1  0.8 
Expert 2  {Efficient}  0.8  
Expert 3  {Efficient}  0.7 
According to
CPC 
CPCs  Experts  BBAs 

1  Adequacy of organisation  Expert 1  m({Very efficient}) = 0.8, m( 
Expert 2  m({Very efficient, Efficient}) = 1  
Expert 3  m({Efficient}) = 0.6, m( 

2  Working conditions  Expert 1  m({Advantageous}) = 0.6, m({Compatible}) = 0.2, m( 
Expert 2  m({Compatible}) = 0.7, m( 

Expert 3  m({Advantageous, Compatible}) = 1  
3  Adequacy of MMI and operational support  Expert 1  m({Adequate}) = 1 
Expert 2  m({Adequate}) = 1  
Expert 3  m({Adequate}) = 1  
4  Availability of procedures/plans  Expert 1  m({Appropriate, Acceptable}) = 1 
Expert 2  m({Acceptable}) = 0.8, m( 

Expert 3  m({Inappropriate}) = 0.5, m( 

5  Number of simultaneous Goals  Expert 1  m({Fewer than capacity}) = 0.4, m( 
Expert 2  m({Matching current capacity}) = 0.5, m( 

Expert 3  m({Matching current capacity}) = 0.4, m( 

6  Available time  Expert 1  m({Adequate}) = 0.6, m( 
Expert 2  m({Temporarily inadequate}) = 0.4, m( 

Expert 3  m({Adequate}) = 0.7, m( 

7  Time of day  Expert 1  m({Daytime}) = 0.7, m( 
Expert 2  m({Daytime}) = 0.6, m({Nighttime}) = 0.3, m( 

Expert 3  m({Daytime}) = 0.6, m( 

8  Adequacy of training and experience  Expert 1  m({Adequatehigh experience}) = 0.8, m( 
Expert 2  m({Adequatehigh experience}) = 0.7, m( 

Expert 3  m({Adequatelow experience}) = 0.8, m( 

9  Crew collaboration quality  Expert 1  m({Very efficient}) = 0.4, m({Efficient}) = 0.4, m( 
Expert 2  m({Efficient}) = 0.8, m( 

Expert 3  m({Efficient}) = 0.7, m( 
For each CPC, fuse the BBAs of the three experts by using
CPC 
CPCs  Fused BBA 

1  Adequacy of organisation  m({Very efficient}) = 0.62, m({Efficient}) = 0.23, 
2  Working conditions  m({Advantageous}) = 0.31, m({Compatible}) = 0.59, m({Advantageous, Compatible} = 0.1 
3  Adequacy of MMI and operational support  m({Adequate}) = 1 
4  Availability of procedures/plans  m({Acceptable}) = 0.8, m({Appropriate, Acceptable}) 
5  Number of simultaneous Goals  m({Fewer than capacity}) = 0.2, m({Matching current capacity}) = 0.3, m({More than capacity}) = 0.2 m( 
6  Available time  m({Adequate}) = 0.82, m({Temporarily inadequate}) 
7  Time of day  m({Daytime}) = 0.93, m({Nighttime}) = 0.05, m( 
8  Adequacy of training and experience  m({Adequatehigh experience}) = 0.76, 
9  Crew collaboration quality  m({Very efficient}) = 0.04, m({Efficient}) = 0.94, 
In order to integrate the experts’ judgments, the fused BBA of each CPC is converted into a probability value according to
CPC 
CPCs  Probability 

1  Adequacy of organisation  P({Very efficient}) = 0.70, P({Efficient}) = 0.30 
2  Working conditions  P({Advantageous}) = 0.36, P({Compatible}) = 0.64 
3  Adequacy of MMI and operational support  P({Adequate}) = 1 
4  Availability of procedures/plans  P({Acceptable}) = 0.9, P({Appropriate}) = 0.1 
5  Number of simultaneous Goals  P({More than capacity}) = 0.286, P({Matching 
6  Available time  P({Adequate}) = 0.92, P({Temporarily 
7  Time of day  P({Daytime}) = 0.95, P({Nighttime}) = 0.05 
8  Adequacy of training and experience  P({Adequatehigh experience}) = 0.80, 
9  Crew collaboration quality  P({Very efficient}) = 0.04, P({Efficient}) = 0.96 
CPC 
CPCs  Multipliers ( 

1  Adequacy of organisation  1 
2  Working conditions  0.928 
3  Adequacy of MMI and operational support  1 
4  Availability of procedures/plans  0.98 
5  Number of simultaneous Goals  1.286 
6  Available time  0.54 
7  Time of day  1.01 
8  Adequacy of training and experience  0.84 
9  Crew collaboration quality  0.98 
Experts evaluate the dependence degree between every two CPCs and suggest fuzzy semantic labels. By converting them into numbers according to
CPC 
CPC 
CPC 
CPC 
CPC 
CPC 
CPC 
CPC 
CPC 


CPC 
0  3  0  3  0  7  0  1  7 
CPC 
3  0  0  1  0  3  0  0  7 
CPC 
1  3  0  1  5  5  0  1  5 
CPC 
5  1  1  0  3  7  0  1  7 
CPC 
5  5  3  5  0  7  0  1  5 
CPC 
7  1  1  3  1  0  1  1  5 
CPC 
3  3  0  1  3  3  0  1  1 
CPC 
7  1  1  1  0  5  0  0  7 
CPC 
1  0  0  3  0  5  0  0  0 
According to DEMATEL’s process (see Definition 2.3), the initial input matrix (i.e., direct relation matrix) is transformed into matrix
CPC 
Modified value ( 
Weight ( 


1  1.5477  2.5234  −0.9757  10.0849  0.7912 
2  0.9935  1.1436  −0.1501  10.91044  0.8559 
3  1.7105  0.4408  1.2697  12.33023  0.9673 
4  1.9623  1.6691  0.2932  11.35377  0.8907 
5  2.4256  0.7395  1.6861  12.74663  1 
6  1.5698  3.2987  −1.7288  9.33172  0.7321 
7  1.2579  0.1387  1.1193  12.1798  0.9555 
8  1.6398  0.4777  1.1620  12.22258  0.9589 
9  0.7834  3.4590  −2.6757  8.384855  0.6578 
In this paper,
CPC 
Multipliers ( 
Weight ( 
Modified multipliers ( 

1  1  0.7912  1 
2  0.928  0.8559  0.9384 
3  1  0.9673  1 
4  0.98  0.8907  0.9822 
5  1.286  1  1.2860 
6  0.54  0.7321  0.6632 
7  1.01  0.9555  1.0096 
8  0.84  0.9589  0.8466 
9  0.98  0.6578  0.9868 
On the “Observation” stage, when the failure mode is “error identification”, the basic value of the error probability is 0.007. The CFP of the observation stage is calculated as 0.0046 according to
To visualize the effect of the proposed method, we calculate CFP without considering the dependence among CPCs, denoted as CFP^{*}. The multiplier in
The advantage of the proposed method is that it can deal with the uncertainty information generated by experts in assessing CPC levels, and take into account the influence of the dependence among CPCs on CFP to avoid the overestimation or underestimation of CFPs. The CFP obtained by the proposed method is more reasonable.
In this paper, we improve a method of calculating CFP in CREAM, by taking into account the specific levels of CPCs and dependence among CPCs. Although the CREAM method provides the level and level factor of each CPC, each level only corresponds to a crisp value, which limits the flexibility of experts in evaluating CPC levels. In this paper, DS evidence theory is introduced to allow experts to make ambiguous judgments and suggest confidence in their judgments, which blurs the boundary between levels. Moreover, the participation of several experts reduces the uncertainty and subjectivity of judgment. Based on expert opinions, we construct the BBA of each CPC and convert it into a probability value to modify the CPC level factor and obtain a multiplier of each CPC.
The classic CREAM assumes that CPCs are independent of each other, which is unreasonable. Failing to consider the dependence among CPCs leads to the repeated calculation of the influence of the related part on CFP, resulting in the overestimation or underestimation of CFPs. To address this issue, we discount the multipliers with the relative weights to obtain the final modified multipliers. For relative weights, we convert expert opinions into an initial input matrix, process the dependence among CPCs using the DEMATEL method, and obtain the relative weights of CPCs. After discounting the multiplier, the modified multipliers are obtained. The CFP calculated is more reasonable and in line with the real situation.
The author greatly appreciates the editor’s encouragement and the anonymous reviewers’ suggestions to improve our paper.
The work is partially supported by Shanghai RisingStar Program (Grant No. 21QA1403400), Shanghai Sailing Program (Grant No. 20YF1414800), Shanghai Key Laboratory of Power Station Automation Technology (Grant No. 13DZ2273800).
The authors confirm contribution to the paper as follows: study conception and design: Xiaoyan Su; data collection: Yuntong Pu, Xiaolei Pan; analysis and interpretation of results: Xiaoyan Su, Hong Qian; draft manuscript preparation: Shuwen Shang, Zhihui Xu. All authors reviewed the results and approved the final version of the manuscript.
The authors declare that they have no conflicts of interest to report regarding the present study.
The authors declare that they have no conflicts of interest to report regarding the present study.