Extending 2AFC rating task
Estimation of both of the mean and variance of a
signal stimulus
Yasuharu Okamoto, 2023.05
The script files have been revised for CmdStanPy, 2025.12
2AFC (two alternative forced choice) task is one of the popular methods of SDT (signal detection theory) (Wixted, 2019). As to measuring memory, Brady et al. (2023) recommend 2AFC, but they point out that, in 2AFC task, equal variance is assumed, and unequal variance models provide a good account of ROC curves. The unequal variance SDT model has been widely used (DeCarlo, 2010). To apply an unequal variance model for 2AFC task, an extension of 2AFC task was proposed by Okamato (2023; https://osf.io/ts2eq/)
In the proposed extension of 2AFC task, two pairs of stimuli, (noise,noise) and (signal,signal), were added to the pairs, (noise,signal) and (signal,noise) , of the standard 2AFC task, and rating of judgment was introduced. From rating data of the pair (noise,noise) , boundaries of categories of rating can be estimated. From rating data of the pair (signal,signal) and estimates of category boundaries, variance of sensation of the signal stimulus can be estimated. From these estimates and data of pairs of (noise,signal) and (signal,noise), mean of sensation of the signal stimulus can be estimated. These are intuitive explanation, which justifies the procedure. For a given data, statistical analysis is done by Bayesian method as follows.
Stochastic Model
Let sensations of the noise stimulus 
and the signal one 
be represented by 
and 
, respectively. They are assumed to have normal distributions as
follows:
Set the origin and the unit so that
In two alternative forced choice (2AFC)
task, two stimuli are presented in one trial, at position 1 and position 2.
These positions are temporal or spatial ones, e.g., the first one and the
second one, or the left side and the right side. Denote these positions using
angle brackets. For example, <S1,S2> means that
S1 is presented at the position1 and S2 is at the position 2. <
> means that sensation of the stimulus at position 1 is 
and sensation of the
stimulus at position 2 is 
.
When the bias of position 2 relative to
position 1 is denoted by 
, we have:
If 
, then 
If 
, then 
, then 
If 
, then 
The bias parameter may include sensory ones, e.g., constant time or space error, and response inclination.
Now, set category boundaries of rating as follows:
where number of categories is 
.
When the following condition holds
rating 
for the pair <> is 
., i.e.
For example, ratings in the case of 
may be as follows:
denotes “It is sure that the signal stimulus was presented at positon 1”
denotes “I think that the signal stimulus was presented at positon 1”
denotes “Maybe, the signal stimulus was presented at positon
1”
denotes “I don’t know/Both stimuli seemed to be the
same”
denotes “Maybe, the signal stimulus was presented at positon
2”
denotes “I think that the signal stimulus was presented at positon 2”
denotes “It is sure that the signal stimulus was presented at positon 2”
Denote a cumulative normal distribution
with mean 
and variance 
by
As a convention, set
Then we have the following equations
When the stimuli 
are presented, we have
Hence,
When the stimuli 
are presented, we have
Hence,
When the stimuli 
are presented, we have
Hence,
When the stimuli 
are presented, we have
Hence,
Category boundaries are set to be symmetric about the origin 0.
In case of K even, we set as follows
In case of K odd, we set as follows
According to this difference, Stan script are prepared as in Listing 1 (K even) and in Listing 2 (K odd). The script files used in this website are archived into the file samplefiles.zip, which can be downloaded and used freely.
Example of Analysis of the extended 2AFC rating data
I show how to use the script files separately in cases where K is even and where K is odd.
In the case that Number of categories K is even
Analyze the data in Table 1.
|
Table 1. Data of 6 categories |
||||
|
Categories |
n-n |
n-s |
s-n |
s-s |
|
1: surely position 1 |
2 |
1 |
6 |
6 |
|
2: positon 1 |
5 |
1 |
10 |
3 |
|
3: probably position 1 |
15 |
13 |
24 |
12 |
|
4: probably position 2 |
21 |
21 |
7 |
21 |
|
5: position 2 |
6 |
7 |
3 |
6 |
|
6: surely position 2 |
1 |
7 |
0 |
2 |
The data of Table 1 was stored in an Excel file (data6cat.xlsx). Notice the orders of rows and columns. Rows are ordered from surely position 1 to surely position 2., columns from n-n, n-s, s-n to s-s.
Figure 1
Put the script files in Listings1, 2 and 3 in the same folder, in which the input data file is also put.
Run the script AnalEx2AFCR.py by the following command in an environment with CmdStanPy installed.
(stan) ****/samplefiles$ python AnalEx2AFCR.py
The input data file name is asked.
(stan) ****/samplefiles$ python AnalEx2AFCR.py
Input data
file (*.xlsx) = data6cat.xlsx
In the above example, the name of Figure 1 is set.
After setting the input file name, building of the Stan script starts. After completion of building, MCMC sampling starts. After sampling, then trace plot of MCMC is displayed (Figure 2).
Figure 2
Close the window of Figure 2, a graph of Figure 3 is displayed.
Figure 3
Posterior distribution of 
is shown with the
median.
Close the window of Figure 3, then the
posterior distribution of 
is displayed (Figure 4).
Figure 4
Posterior probability that 
is 0.94. It seems that
the equal variance assumption does not hold up.
Close the window of Figure 4, then posterior distributions of category boundaries are shown (Figure 5).
Figure 5
In case of K even, the middle criterion is fixed at the origin 0, and shown as a red semicircle.
Close the window of Figure 5, then the posterior distribution of is shown (Figure 6).
Figure 6
Close the window of Figure 6, then a graph which shows relation between the cumulative proportion of category responses and the predicted values is shown (Figure 7).
Figure 7
Close the window of Figure 7, the program ends.
In the case that Number of categories K is odd.
Analyze the data in Table 2.
|
Table 2. Data of 7 categories |
||||
|
Categories |
n-n |
n-s |
s-n |
s-s |
|
1: surely position 1 |
2 |
0 |
5 |
0 |
|
2: positon 1 |
7 |
3 |
9 |
7 |
|
3: probably position 1 |
9 |
10 |
14 |
13 |
|
4: don’t know/same |
17 |
14 |
11 |
7 |
|
5: probably position 2 |
11 |
9 |
10 |
13 |
|
6: position 2 |
1 |
10 |
1 |
9 |
|
7: surely position 2 |
3 |
4 |
0 |
1 |
The data of Table 2 is saved in Excel file data7cat.xlsx (Figure 8)
Figure 8
Put the script files of Listings 1, 2 and 3, and the input data file in the same folder.
Run the script of Listing 3 by the following command in an environment with CmdStanPy installed.
(stan) ****/samplefiles$ python AnalEx2AFCR.py
The input data file name is asked.
(stan) ****/samplefiles$ python AnalEx2AFCR.py
Input data file
(*.xlsx) = data7cat.xlsx
After the input data file name is set, building of the Stan script starts. After building, MCMC sampling starts. Results of the sampling is displayed as trace plot (Figure 9).
Figure 9
Close the window of Figure 9, posterior
distribution of is displayed (Figure 10)
Figure 10
Close the window of Figure 10, the
posterior distribution of is displayed (Figure
11).
Figure 11
Posterior probability that is 0.546, so the
assumption of equal variance seems to hold up.
Close the window of Figure 11, then posterior distributions of category boundaries are shown (Figure 12).
Figure 12
Close the window of Figure 12, then the
posterior distribution of is shown (Figure 13).
Figure 13
Close the window of Figure 13, then a graph which shows relation between the cumulative proportion of category responses and the predicted values is shown (Figure 14).
Figure 14
Close the window of Figure 14, the program ends.
Brady, T. F., Robinson, M. M., Williams, J. R., & Wixted, J. T. (2023). Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research. Psychonomic Bulletin & Review, 30, 421-449.
DeCarlo, L. T. (2010). On the statistical and theoretical basis of signal detection theory and extensions: Unequal variance, random coefficients, and mixture models. Journal of Mathematical Psychology, 54, 304-313.
Wickens, T. D. (2002). Elementary Signal Detection Theory. Oxford University Press.
Wixted, J. T. (2019). The forgotten history of signal detection theory. Journal of Experimental Psychology: Learning, Memory, & Cognition, 46, 201-233.
Listing 1. Stan script for the extended 2AFC rating task in case of K even (SDT_2AFC_KCat_even.stan)
data {
int K;
array[K] int nn_cond;
array[K] int ns_cond;
array[K] int sn_cond;
array[K] int ss_cond;
}
transformed data {
int hK;
vector[K%/%2] alpha;
hK
= K %/% 2;
for (i
in 1:hK) {
alpha[i] = 1.0;
}
}
parameters {
real mu_s;
real<lower = 0.00001> sgm_s;
real b;
simplex[hK]
preC;
}
transformed parameters {
vector[K] nn_theta;
vector[K] ns_theta;
vector[K] sn_theta;
vector[K] ss_theta;
vector[K+1] nn_cum_p;
vector[K+1] ns_cum_p;
vector[K+1] sn_cum_p;
vector[K+1] ss_cum_p;
vector[K-1] C;
real vsum;
real sgm_nn;
real sgm_ns;
real sgm_ss;
C[hK]
= 0.0;
vsum
= 0.00005;
for (i
in 1:hK-1) {
vsum += 0.9999*preC[i];
C[hK+i] = vsum / (1.0 - vsum);
C[hK-i] = -C[hK+i];
}
sgm_nn
= sqrt(1.0 + 1.0);
sgm_ns
= sqrt(1.0 + square(sgm_s));
sgm_ss
= sqrt(square(sgm_s)*2);
nn_cum_p[1] = 0.0;
ns_cum_p[1] = 0.0;
sn_cum_p[1] = 0.0;
ss_cum_p[1] = 0.0;
nn_cum_p[K+1]
= 1.0;
ns_cum_p[K+1]
= 1.0;
sn_cum_p[K+1]
= 1.0;
ss_cum_p[K+1]
= 1.0;
for (i
in 2:K) {
nn_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | b, sgm_nn);
ns_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | mu_s + b, sgm_ns);
sn_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | -mu_s + b, sgm_ns);
ss_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | b, sgm_ss);
}
for (i
in 1:K) {
nn_theta[i] = nn_cum_p[i+1]
- nn_cum_p[i];
ns_theta[i] = ns_cum_p[i+1]
- ns_cum_p[i];
sn_theta[i] = sn_cum_p[i+1]
- sn_cum_p[i];
ss_theta[i] = ss_cum_p[i+1]
- ss_cum_p[i];
}
}
model {
mu_s
~ normal(0.0, 1000.0);
sgm_s
~ uniform(0.00001, 1000.0);
preC
~ dirichlet(alpha);
b ~ normal(0.0,
1000.0);
nn_cond
~ multinomial(nn_theta);
ns_cond
~ multinomial(ns_theta);
sn_cond
~ multinomial(sn_theta);
ss_cond
~ multinomial(ss_theta);
}
Listing 2 . Stan script for the extended 2AFC rating task in case of K odd (SDT_2AFC_KCat_odd.stan)
data {
int K;
array[K] int nn_cond;
array[K] int ns_cond;
array[K] int sn_cond;
array[K] int ss_cond;
}
transformed data {
int hK;
vector[(K%/%2)+1] alpha;
hK
= K %/% 2;
for (i
in 1:hK+1) {
alpha[i] = 1.0;
}
}
parameters {
real mu_s;
real<lower = 0.0001> sgm_s;
real b;
simplex[hK+1] preC;
}
transformed parameters {
vector[K] nn_theta;
vector[K] ns_theta;
vector[K] sn_theta;
vector[K] ss_theta;
vector[K+1] nn_cum_p;
vector[K+1] ns_cum_p;
vector[K+1] sn_cum_p;
vector[K+1] ss_cum_p;
vector[K-1] C;
real vsum;
real sgm_nn;
real sgm_ns;
real sgm_ss;
vsum
= 0.00005;
for (i
in 1:hK) {
vsum += 0.9999*preC[i];
C[hK+i] = vsum / (1.0 - vsum);
C[hK+1-i] = -C[hK+i];
}
sgm_nn
= sqrt(1.0 + 1.0);
sgm_ns
= sqrt(1.0 + square(sgm_s));
sgm_ss
= sqrt(square(sgm_s)*2);
nn_cum_p[1] = 0.0;
ns_cum_p[1] = 0.0;
sn_cum_p[1] = 0.0;
ss_cum_p[1] = 0.0;
nn_cum_p[K+1]
= 1.0;
ns_cum_p[K+1]
= 1.0;
sn_cum_p[K+1]
= 1.0;
ss_cum_p[K+1]
= 1.0;
for (i
in 2:K) {
nn_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | b, sgm_nn);
ns_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | mu_s + b, sgm_ns);
sn_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | -mu_s + b, sgm_ns);
ss_cum_p[i] = 0.00005 + 0.9999*normal_cdf(C[i-1] | b, sgm_ss);
}
for (i
in 1:K) {
nn_theta[i] = nn_cum_p[i+1]
- nn_cum_p[i];
ns_theta[i] = ns_cum_p[i+1]
- ns_cum_p[i];
sn_theta[i] = sn_cum_p[i+1]
- sn_cum_p[i];
ss_theta[i] = ss_cum_p[i+1] - ss_cum_p[i];
}
}
model {
mu_s
~ normal(0.0, 1000.0);
sgm_s
~ uniform(0.0001, 1000.0);
preC
~ dirichlet(alpha);
b ~ normal(0.0,
1000.0);
nn_cond
~ multinomial(nn_theta);
ns_cond
~ multinomial(ns_theta);
sn_cond
~ multinomial(sn_theta);
ss_cond
~ multinomial(ss_theta);
}
Listing 3. Python script which uses the Stan scripts in Listings 1 and 2 (AnalEx2AFCR.py)
import numpy
as np
import scipy.stats as ss
import matplotlib.pyplot as plt
import seaborn as sb
from cmdstanpy
import CmdStanModel
import arviz
as az
import pandas as pd
inflnm = input('Input
data file (*.xlsx) = ')
data_pd = pd.read_excel(inflnm).values
print(data_pd)
n_n = data_pd[:,1]
n_s = data_pd[:,2]
s_n = data_pd[:,3]
s_s = data_pd[:,4]
print('n_n =', n_n)
print('n_s =', n_s)
print('s_n =', s_n)
print('s_s =', s_s)
K = len(n_n)
if K < 3:
print('Number
of categories should be larger than 2.')
sys.exit()
if len(n_s) != K:
print('Numbers
of categories are not consistent.')
sys.exit()
if len(s_n) != K:
print('Numbers
of categories are not consistent.')
sys.exit()
if len(s_s) != K:
print('Numbers
of categories are not consistent.')
sys.exit()
Data = {'K':K,
'nn_cond':n_n, 'ns_cond':n_s, 'sn_cond':s_n, 'ss_cond':s_s}
stan_code = 'SDT_2AFC_KCat_odd.stan'
if K % 2 == 1 else \
'SDT_2AFC_KCat_even.stan'
model = CmdStanModel(stan_file=stan_code)
fit = model.sample(data=Data)
print(fit.diagnose())
print(fit.summary())
infdata = az.from_cmdstanpy(fit) # Arviz InfereceData型へ変換
az.plot_trace(infdata, var_names=['mu_s', 'sgm_s'])
plt.tight_layout()
plt.savefig('Fig_trace.png')
plt.show()
fit = fit.draws_pd()
# Pandas DataFrame型への変換
print(fit.keys())
mu_s_med = np.median(fit['mu_s'])
sb.kdeplot(fit['mu_s'])
plt.xlabel(r'$\mu_s$')
plt.title(r'$\mu_s$(Med)
= {0:.3f}'.format(mu_s_med))
plt.savefig('Fig_mu.png')
plt.show()
sgm_s_med = np.median(fit['sgm_s'])
p_sgmL1 = np.mean(fit['sgm_s'] >
1.0)
sb.kdeplot(fit['sgm_s'])
plt.title(r'$\sigma_s$(Med)
= {0:.3f}, P($\sigma_s$>1) = {1:.5f}'.format
(sgm_s_med, p_sgmL1))
plt.xlabel(r'$\sigma_s$')
plt.savefig('Fig_sgm.png')
plt.show()
Csmpls = []
for k in range(K-1):
Csmpls.append(fit[f'C[{k+1}]'])
Csmpls = np.array(Csmpls).T
print('Shape_Csmpls
=', np.shape(Csmpls))
s_title = ''
if K % 2 == 1:
for k
in range(K-1):
sb.kdeplot(Csmpls.T[k])
c_med = np.median(Csmpls.T[k])
s_title += f'C{k+1}={c_med:.2f}'
if k
< K-2:
s_title += ', '
else:
for k
in range(K-1):
if k != (K//2) - 1:
sb.kdeplot(Csmpls.T[k])
c_med = np.median(Csmpls.T[k])
s_title += f'C{k+1}={c_med:.2f}'
if k < K-2:
s_title += ', '
else:
plt.plot([0], [0],
marker = 'o', markersize = 15, c = 'r')
s_title += f'C{k+1}=0, '
plt.title('Median estimates\n' + s_title)
plt.savefig('FigC.png')
plt.show()
b_med = np.median(fit['b'])
sb.kdeplot(fit['b'])
plt.title('b(Med) = {0:.3f}'.format(b_med))
plt.savefig('Fig_b.png')
plt.show()
cum_n_n = np.cumsum(n_n)
pcum_n_n = cum_n_n/cum_n_n[-1]
cum_n_s = np.cumsum(n_s)
pcum_n_s = cum_n_s /
cum_n_s[-1]
cum_s_n = np.cumsum(s_n)
pcum_s_n = cum_s_n /
cum_s_n[-1]
cum_s_s = np.cumsum(s_s)
pcum_s_s = cum_s_s /
cum_s_s[-1]
nn_cum_p_smpls = []
ns_cum_p_smpls = []
sn_cum_p_smpls = []
ss_cum_p_smpls = []
for k in range(K+1):
nn_cum_p_smpls.append(fit[f'nn_cum_p[{k+1}]'])
ns_cum_p_smpls.append(fit[f'ns_cum_p[{k+1}]'])
sn_cum_p_smpls.append(fit[f'sn_cum_p[{k+1}]'])
ss_cum_p_smpls.append(fit[f'ss_cum_p[{k+1}]'])
nn_cum_p_smpls = np.array(nn_cum_p_smpls).T
ns_cum_p_smpls = np.array(ns_cum_p_smpls).T
sn_cum_p_smpls = np.array(sn_cum_p_smpls).T
ss_cum_p_smpls = np.array(ss_cum_p_smpls).T
est_pcum_n_n = np.median(nn_cum_p_smpls,
axis=0)[1:-1]
est_pcum_n_s = np.median(ns_cum_p_smpls,
axis=0)[1:-1]
est_pcum_s_n = np.median(sn_cum_p_smpls,
axis=0)[1:-1]
est_pcum_s_s = np.median(ss_cum_p_smpls,
axis=0)[1:-1]
plt.plot(est_pcum_n_n,
pcum_n_n[:-1],
label = 'n_n')
plt.plot(est_pcum_n_s,
pcum_n_s[:-1],
label = 'n_s')
plt.plot(est_pcum_s_n,
pcum_s_n[:-1],
label = 's_n')
plt.plot(est_pcum_s_s,
pcum_s_s[:-1],
label = 's_s')
plt.plot([0,1], [0,1], c = 'k', ls = '--',
label = 'Obs.=Est.')
plt.xlabel('Est.Cum.P')
plt.ylabel('Obs.Cum.P')
plt.legend()
plt.title('Cumulative Proportions')
plt.savefig('FigCumEstObs.png')
plt.show()