One Hot Encoding and Common Quantification
Yasuharu Okamoto
As a preprocessing method, One Hot Encoding is well known in machine learning. It encodes a categorical attribute with dummy variables. Dummy variables encode categorical values individually.
On the other hand, common quantification preprocesses categorical attributes, so that the numerical values given to categorical values (locally) maximize the variance of the numerical values given to the categorical attribute. The numerical values given by common quantification are given by the following eigen decomposition
We get K-1 sets of common quantification for a categorical attribute with K categorical values.
For the details, see Okamoto (2015).
Comparison of One Hot Encoding and common quantification was reported by 岡本安晴（2020）. In this website, the programs, which are based on those used in 岡本安晴（2020）, are prepared, and some results of the comparison are presented.
Programs are written in Python. Program source files and the sample data file are archived in files_q_one.zip , which can be freely downloaded.
Installation of Python is explained in this website.
The programs in this website use the sample data file (Figure 1), which is created from Table 8.2.1 of 岡本安晴（2019）.
Figure 1 Sample data (DataPsy.xlsx)
An input data file should be created as an Excel file.
In the first row, names of the attributes are set. The attribute DataID is used to identify a case of the data. Attribute Q1 to Q5 are categorical ones. When the data were collected, number of categories of the attributes were three or four. But, after removing unused categories, number of categories becomes the same number three. When the number of categories of an attribute is K, categorical values of that attribute should be represented by integers from 1 to K.
Attributes Q6 and Q7 are numerical ones.
In the second row, set the numerical values which show how to preprocess the corresponding attributes. Number 0 means no preprocessing. Number 1 dictates standardization. Number larger than 1 indicates the number of categorical values, according to which common quantification or One Hot Encoding is applied to the attribute.
After the second row, data values are set.
Using the sample data (Figure 1), an example of application of common quantification is shown first, then for comparison, an application of One Hot Encoding is shown.
To assign numerical values to categorical ones by the common quantification, the comq function is used:
#
# X: The categorical
variable to bu quantified
# NCat: The number of categories in the
variable
#
NCase: The number of cases
#
def comq( X, NCat, NCase ):
G =
[0]*NCase
# A list for the indicator
matrix
for
i in range(0, NCase):
if (X[i] < 1) or (X[i] > NCat):
print("Category is out of the range...X[%d] = %d"%(i,X[i]))
raise "CatRange"
G[i] = []
for j in range(0, NCat):
if (X[i] == j + 1):
G[i].append(1)
else:
G[i].append(0)
G =
np.array(G)
onesRow = np.full((1, NCase), 1)
onesCol = np.full((NCase, 1), 1)
f =
onesRow @ G
H =
G - (onesCol @ f) / NCase
HpH
= H.transpose() @ H
w, v
= eigen(HpH)
# Eigen decomposition of HpH
= H'H
n_quant = 0;
# n_quant: the number of
non-zero eigen values
while (w[n_quant] > 1.0e-7):
n_quant = n_quant + 1
eigen_val = w[0: n_quant]
eigen_vec = v[0: n_quant]
matOmega = eigen_vec.transpose()
#
#
Calculation of the standardized scores of quantification
#
z =
H @ matOmega
invLmbdN
= np.full((n_quant, n_quant), 0.0)
for
i in range(n_quant):
invLmbdN[i, i] = 1.0 / ((eigen_val[i] / NCase) ** 0.5)
z =
z @ invLmbdN
return n_quant, eigen_val, matOmega, z
#
n_quant: The
number of quantification variables
#
eigen_val:
The vector of eigen values of quantification
#
matOmega:
The matrix of the eigen vectpors (the weights)
#
of quantification
Script files and the sample data file are archived in files_q_one.zip , which can be freely downloaded.
For common quantification, run MainCommonQ.py.
The name of the input data file is asked (Figure 2).
Figure 2
Set the Excel data file name. In Figure 2, the name of the file in Figure 1,“DataPsy.xlsl”, is set.
When the input file name is set, data is read and the values read are printed out.
After reading the data file, the name of the output text file is asked (Figure 3).
Figure 3
Set any text file name, which you like.
After entering the output text file name, name of the output Excel file is asked
Set any name with the file name extension “.xlsx”
After the output Excel file name is set, the transformation of the input data starts.
When the transformation completes, the message that the output Excel file and the output text file are saved is displayed, then the program ends (Figure 3).
After the program ends, open the output Excel file (Figure 4).
Figure 4
You can see that the transformation of the input file is successful. Each of categorical attributes has 3 categorical values, so 3-1=2 common quantification, Qj-0 and Qj-1 for the categorical attribute Qj, are obtained.
The output text file is as follows:
Input data file = DataPsy.xlsx
Categorical variable: Q1
quantified.Number of quantification variables = 2
Eigen value = 13.84615
Eigen vector =
0.40825
-0.81650
0.40825
Eigen value = 12.00000
Eigen vector =
-0.70711
-0.00000
0.70711
Categorical variable: Q2
quantified.Number of quantification variables = 2
Eigen value = 14.78034
Eigen vector =
-0.61920
0.77052
-0.15132
Eigen value = 6.24530
Eigen vector =
0.53222
0.27013
-0.80235
.
.
.
Values of 
s and 
s of Equation (1) are shown.
If we analyze the data obtained by common quantification, data in Figure 4, by Principal Component Analysis (PCA), we obtain the result shown in Figure 5.
Figure 5
Negative relations between Q1-1 and Q2-0 and between Q4-0 and Q6 are indicated.
The relation between Q1-1 and Q2-0 is shown in Table 1, in which the order of columns are arranged according to the weights. We can see the negative relation easily in Table 1.
|
Table
1 Cross table of Q1-1 and Q2-0 Weights
are shown in (). |
|||
|
|
Q2(Cat-1) (-0.61920) |
Q2(Cat-3) (-0.15132) |
Q2(Cat-2) (0.77052) |
|
Q1(Cat-1)-(0.70711) |
1 |
1 |
10 |
|
Q1(Cat-2) (0.00000) |
3 |
2 |
10 |
|
Q1(Cat-3) (0.70711) |
6 |
2 |
4 |
To see the negative relation between Q4-0 and Q6, simple linear regression analysis is applied to Q4-0 and Q6.
We obtain the result in Figure 6.
Figure 6
To compare common quantification with One Hot Encoding, a program for One Hot Encoding was developed.
The OneHotEncoding function, by which One Hot Encoding is done, is as follows:
#
# X: The categorical
variable to bu quantified
# NCat: The number of categories in the
variable
#
NCase: The number of cases
#
def OneHotEncoding( X, NCat, NCase ):
G =
[0]*NCase
# A list for the indicator
matrix
for
i in range(0, NCase):
if (X[i] < 1) or (X[i] > NCat):
print("Category is out of the range...X[%d] = %d"%(i,X[i]))
raise "CatRange"
G[i] = []
for j in range(0, NCat):
if (X[i] == j + 1):
G[i].append(1)
else:
G[i].append(0)
G =
np.array(G)
return G
Script files and a sample data file are archived in files_q_one.zip , which can be freely downloaded.
To do One Hot Encoding, run MainOneHotEncoding.py.
The input data file name is asked (Figure 7).
Figure 7
Set the Excel data file name. In Figure 7, the name of the file in Figure 1,“DataPsy.xlsl”, is set.
When the input file name is set, data is read and the values read are printed out.
After reading the data file, the name of the output text file is asked (Figure 8).
Fgiure 8
Set any text file name, which you like.
After entering the output text file name, the output Excel file name is asked
Set any name with the file name extension “.xlsx”
After the output Excel file name is set, the transformation of the input data starts.
When the transformation completes, the message that the output Excel file and the output text file are saved is displayed, then the program ends (Figure 8).
After the program ends, open the output Excel file (Figure 9).
Figure 9
We can see that the transformation completed successfully.
When we analyze the data in Figure 9 by PCA, we get the result in Figure 10.
Figure 10
We can see relations among individual category values Q1-0 to Q5-2 and numerical attributes Q6 and Q7.
岡本安晴（2019）．いまさら聞けないPythonでデータ分析」丸善出版．
岡本安晴（2020）One Hot Encodingと共通数量化、日本行動計量学会第48回大会抄録集、212−215．