(Matlab) Codes and Datasets for Subspace Learning (Dimensionality Reduction)

We provide here some matlab codes of subspace learning algorithms, as well as some datasets in matlab format. All these codes and data sets are used in our experiments. The processed data in matlab format can only be used for non-commercial purpose.

If you have some problems or find some bugs in the codes, please email: dengcai AT gmail DOT com

If you find these algoirthms and data sets useful, we appreciate it very much if you can cite our related works: (Publications sort by topic)

Deng Cai, Xiaofei He, and Jiawei Han, "SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis", IEEE TKDE 2008. Bibtex source
Deng Cai, Xiaofei He, Jiawei Han, and Hong-Jiang Zhang, "Orthogonal Laplacianfaces for Face Recognition", in IEEE TIP, 2006. Bibtex source
Deng Cai, Xiaofei He, and Jiawei Han, "Document Clustering Using Locality Preserving Indexing", in IEEE TKDE, 2005. Bibtex source
Deng Cai, Xiaofei He and Jiawei Han, "Semi-Supervised Discriminant Analysis", ICCV'07. Bibtex source
Deng Cai, Xiaofei He, Yuxiao Hu, Jiawei Han and Thomas Huang, "Learning a Spatially Smooth Subspace for Face Recognition", CVPR'07. Bibtex source
Deng Cai, Xiaofei He and Jiawei Han, "Spectral Regression for Efficient Regularized Subspace Learning", ICCV'07. Bibtex source
Xiaofei He, Shuicheng Yan, Yuxiao Hu, Partha Niyogi, and Hong-Jiang Zhang, "Face Recognition Using Laplacianfaces", in IEEE TPAMI, 2005. Bibtex source
Xiaofei He and Partha Niyogi, "Locality Preserving Projections", NIPS 16, 2003. Bibtex source

Learning algorithms

All the codes have been rewritten. You can download the old version at here .

Non-negative Matrix Factorization on Manifold (Graph)
Topic Modeling on Manifold (Network)
Spectral Regression (SRLDA, SRKDA, RLPP...)
Spectral Regression provides an efficient subspace learning framework. I strongly recommend using SR instead of the following ordinary subspace learning algorithms.
Some general functions
- EuDist2: Calculate the Euclidean distance matrix of two data matrix.
- constructKernel: Construct the kernel matrix.
PCA: Principal Component Analysis
KPCA: Kernel Principal Component Analysis
LGE: (Regularized) Linear Graph Embedding (Provides a general framework for graph based subspace learning. This function will be called by LPP, NPE, IsoProjection, LSDA, MMP ...)
OLGE: (Regularized) Orthogonal Linear Graph Embedding (Provides a general framework for graph based subspace learning (orthogonal basis vectors). This function will be called by OLPP. It is also very easy to develop ONPE, OIsoProjection, OLSDA, MMP...)
TensorLGE: Tensor Linear Graph Embedding (Provides a general framework for graph based tensor subspace learning. This function will be called by TensorLPP. It is also very easy to develop TensorNPE, TensorIsoProjection, TensorLSDA, TensorMMP...)
KGE: (Regularized) Kernel Graph Embedding (Provides a general framework for graph based kernel subspace learning. This function will be called by KernelLPP. It is also very easy to develop KernelNPE, KernelIsoProjection, KernelLSDA, KernelMMP...)
- Deng Cai, Xiaofei He, Yuxiao Hu, Jiawei Han and Thomas Huang, "Learning a Spatially Smooth Subspace for Face Recognition", CVPR'07. Bibtex source
LDA: (Regularized) Linear Discriminant Analysis (Generally, LDA can also use LGE as a subroutine. However, we can use the special graph structure of LDA to obtain some computational benefits.)
KDA: (Regularized) Kernel Discriminant Analysis (Generally, KDA can also use KGE as a subroutine. However, we can use the special graph structure of KDA to obtain some computational benefits.)
- Deng Cai, Xiaofei He and Jiawei Han, "SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis", IEEE TKDE 2008. Bibtex source
- Deng Cai, Xiaofei He and Jiawei Han, "Spectral Regression for Efficient Regularized Subspace Learning", ICCV'07. Bibtex source
LPP: Locality Preserving Projection (You need to download LGE.m as well as constructW.m).
OLPP: Orthogonal Locality Preserving Projections (You need to download OLGE.m as well as constructW.m)
TensorLPP: Tensor Locality Preserving Projections (You need to download TensorLGE.m as well as constructW.m)
KernelLPP: Kernel Locality Preserving Projections (You need to download KGE.m as well as constructW.m)
constructW: Function used to construct the affinity matrix.
- Deng Cai, Xiaofei He, Jiawei Han, and Hong-Jiang Zhang, "Orthogonal Laplacianfaces for Face Recognition", in IEEE TIP, 2006. Bibtex source
- Xiaofei He, Shuicheng Yan, Yuxiao Hu, Partha Niyogi, and Hong-Jiang Zhang, "Face Recognition Using Laplacianfaces", in IEEE TPAMI, 2005. Bibtex source
- Xiaofei He and Partha Niyogi, "Locality Preserving Projections", NIPS 16, 2003. Bibtex source
NPE: Neighborhood Preserving Embedding (You need to download LGE.m)
IsoProjection: Isometric Projection (You need to download LGE.m)
dijkstra.dll (for Windows)
dijkstra.mexglx (for Linux): dijkstra algorithm (You can download the source code at here)
LSDA: Locality Sensitive Discriminant Analysis (You need to download LGE.m)
SDA: Semi-Supervised Discriminant Analysis (Graph Embedding Way)
SDA_MR: Semi-Supervised Discriminant Analysis (Manifold Regularization Way)
- Deng Cai, Xiaofei He and Jiawei Han, "Semi-Supervised Discriminant Analysis", ICCV'07. Bibtex source
MMP: Maximum Margin Projection
LaplacianScore: LaplacianScore for feature selection
Other functions:
- Clustering Evaluation: Evaluate the clustering result by accuracy and normalized mutual information
  Deng Cai, Xiaofei He, and Jiawei Han, "Document Clustering Using Locality Preserving Indexing", in IEEE TKDE, 2005. Bibtex source
- bestMap
- hungarian
- MutualInfo
  %===========================================
  fea = rand(50,70);
  gnd = [ones(10,1);ones(15,1)*2;ones(10,1)*3;ones(15,1)*4];
  
  res = kmeans(fea,4);
  
  res = bestMap(gnd,res);
  %============= evaluate AC: accuracy ==============
  AC = length(find(gnd == res))/length(gnd);
  %============= evaluate MIhat: nomalized mutual information =================
  MIhat = MutualInfo(gnd,res);
  %===========================================

Data sets in matlab format

Reproducing experimental results

Deng Cai, Xiaofei He, Wei Vivian Zhang, and Jiawei Han, "Regularized Locality Preserving Indexing via Spectral Regression", in CIKM'07, 2007. ( pdf )
Deng Cai, Xiaofei He, and Jiawei Han, "Spectral Regression for Efficient Regularized Subspace Learning", in ICCV'07, 2007. ( pdf )
%=========Unsupervised=====================
options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'KNN';
options.WeightMode = 'Cosine';
options.k = 5;
W = constructW(fea,options);

options = [];
options.W = W;
options.ReguType = 'Ridge';
options.ReguAlpha = 0.1;

[eigvector] = SR_caller(options, fea);

[nSmp,nFea] = size(fea);
if size(eigvector,1) == nFea + 1
                newfea = [fea ones(nSmp,1)]*eigvector;
else
                newfea = fea*eigvector;
end
%=========Supervised=====================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.gnd = gnd_Train;
options.ReguType = 'Ridge';
options.ReguAlpha = 0.1;

[eigvector] = SR_caller(options, fea_Train);

[nSmp,nFea] = size(fea);
if size(eigvector,1) == nFea + 1
                newfea = [fea ones(nSmp,1)]*eigvector;
else
                newfea = fea*eigvector;
end
%===========================================
D. Cai, X. He, Y. Hu, J. Han, and T. Huang, "Learning a Spatially Smooth Subspace for Face Recognition", in CVPR'07, 2007. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.Regu = 1;
options.ReguAlpha = 0.1;
options.ReguType = 'Custom';
load('TensorR_32x32.mat');
options.regularizerR = regularizerR;

[eigvector, eigvalue] = LPP(W, options, fea_Train);
[eigvector, eigvalue] = LDA(gnd_Train, options, fea_Train);
...

newfea = fea*eigvector;
%===========================================
D. Cai, X. He, J. Han, and H. Zhang, "Orthogonal Laplacianfaces for Face Recognition", IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3608-3614, November, 2006. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.Regu = 0;
options.PCARatio = 1;
bSuccess = 0;
while ~bSuccess
[eigvector, eigvalue, bSuccess] = OLPP(W, options, fea_Train);
end

newfea = fea*eigvector;
%===========================================
X. He, D. Cai, and P. Niyogi, "Tensor Subspace Analysis", in NIPS'05, 2005. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.nRepeat = 10;
fea_Train = reshape(fea_Train',32,32,size(fea_Train,1));
[U, V, eigvalue_U, eigvalue_V, posIdx] = TensorLPP(fea_Train, W, options);

[nSmp,nFea] = size(fea);
fea = reshape(fea',32,32,nSmp);
nRow = size(U,2);
nCol = size(V,2);
newfea = zeros(nRow,nCol,nSmp);
for i=1:nSmp
newfea(:,:,i) = U'*fea(:,:,i)*V;
end
newfea = reshape(newfea,nRow*nCol,nSmp)';
newfea = newfea(:,posIdx);
%===========================================

Return to Deng Cai's Homepage