GPS Demo: predicting user involvement¶

Without data fusion¶

As a baseline test, only the UserLocAct_missing tensor will be used. We model the tensor by a rank \(R=2\) CPD and add regularization to prevent overfitting. Mathematically, we solve:

In Tensorlab we use the sdf_nls method to solve this model. The cpd_nls method can be used as well if regularization is not required. As usual, the SDF model is defined in three steps: variable definition, factor matrix definition and model choice for each dataset.

First three variables are defined: u for the users, l for the location and a for the activities. For each variable, we define a random initial guess. A better guess can be used if available from prior knowledge, or from previously computed models.

R = 2;
model.variables.u   = rand(nbUsers,R);
model.variables.l   = rand(nbLocations,R);
model.variables.a   = rand(nbActivities,R);

Next, we define the factor matrices. To illustrate the use of constraints, we impose nonnegativity constraints on all factor matrices.

model.factors.U   = {'u', @struct_nonneg};
model.factors.L   = {'l', @struct_nonneg};
model.factors.A   = {'v', @struct_nonneg};

Finally, the tensor is modeled. A CPD will be used here. We also add L2 regularization by defining a second factorization. In the case of regularization, the data part can be omitted if it is zero. Note that regL2 is implemented to add a term \(\norm{\cdot}^2_{\scriptsize{\text{F}}}\) for each factor matrix separately.

model.factorizations.ula.data = UserLocAct_missing;
model.factorizations.ula.cpd  = {'U','L','A'};

model.factorizations.reg.regL2 = {'U','L','A'};

We can investigate the resulting model using the language parser sdf_check:

sdf_check(model, 'print');

If no errors have been made, the following overview is printed:

Factor               Size                 Comment
------------------------------------------------------------------------
U                    [146x2]
L                    [168x2]
A                    [5x2]

Factorization        Type       Factors              Comments
------------------------------------------------------------------------
ula (id = 1)         cpd        U,L,A
reg (id = 2)         regL2      U,L,A

In the Matlab Command Window, the factors are links. These links can be used to investigate the model further. For example, when clicking on the factor U, the steps in its construction are revealed:

Expansion of factor U (id = 1) (view):
  (1,1)  u               [146x2]    struct_nonneg        Factor         [146x2]

We can now call the sdf_nls routine to solve the problem using extra optimization options. The one that most affects the results here is the RelWeights option, which gives a weight to each factorization, in casu the tensor and the regularization. The lambdareg parameter has experimentally been chosen equal to 0.01:

options = struct;
options.Display    = 10;
options.MaxIter    = 500;
options.TolFun     = 1e-8;
options.TolX       = 1e-4;
options.CGMaxIter  = 150;
options.RelWeights = [1 lambdareg];

% call the solver
[sol,out] = sdf_nls(model,options);

To measure the quality, we use performance curves and the area-under-curve (AUC) value. Here we compare the predicted values pred with the removed values. First the predicted values are computed by reconstructing the tensor using the computed factor matrices. The result can be seen in Figure 1.

pred = cpdgen({sol.factors.U,sol.factors.L,sol.factors.A});
lab  = UserLocAct(sel);
pred = pred(sel);

[X,Y,~,AUC] = perfcurve(lab(:), pred(:),1);
plot(X,Y);

With data fusion¶

Now, we enrich the model by using the other datasets as well. We will need an extra variable f to model the features in the LocFea tensor.

model.variables.f = rand(size(LocFea, 2), R);

Also, because the data in the tensor and the matrices may be scaled differently, we introduce extra variables duu, daa and dul to capture the scaling differences in the UserUser, the ActAct and the UserLoc matrices. We do not need an extra variable for the LocFea matrix as the scaling differences are captured in F.

model.variables.duu = rand(1,R);
model.variables.daa = rand(1,R);
model.variables.dul = rand(1,R);

Like before, we create the corresponding factor matrices. We use a more dynamic approach to apply constraints here. If constr = {}, cat(2, 'u', constr) is equivalent to {'u'}; if constr = {@struct_nonneg}, cat(2, 'u', constr) becomes {'u', @struct_nonneg} like we used above. This more dynamic notation becomes useful when experimenting with different types of constraints.

constr = {};
model.factors.u   = cat(2,'u',constr);
model.factors.l   = cat(2,'l',constr);
model.factors.a   = cat(2,'a',constr);
model.factors.f   = cat(2,'f',constr);
model.factors.duu = cat(2,'duu',constr);
model.factors.daa = cat(2,'daa',constr);
model.factors.dul = cat(2,'dul',constr);

Alternatively, the transform field can be used. Instead of defining the factors, we can write:

model.transform = {{'u', 'l', 'a', 'f', 'duu', 'daa', 'dul'}, ...
                   {'U', 'L', 'A', 'F', 'Duu', 'Daa', 'Dul'}, ...
                   @struct_nonneg};

We define the models used for the different matrices. Here we choose a CP model for all matrices and apply L2 regularization to all factor matrices. Note that the factor matrices Duu, Daa and Dul can be added without any problem, because a matrix is a tensor with the third dimension equal to 1.

model.factorizations.ula.data = UserLocAct_missing;
model.factorizations.ula.cpdi = {'U','L','A'};

Model.factorizations.uu.data = UserUser;
model.factorizations.uu.cpd  = {'U','U','Duu'};

model.factorizations.lf.data = LocFea;
model.factorizations.lf.cpd  = {'L','F'};

model.factorizations.aa.data = ActAct;
model.factorizations.aa.cpd  = {'A','A','Daa'};

model.factorizations.ul.data = UserLoc;
model.factorizations.ul.cpd  = {'U','L','Dul'};

model.factorizations.reg.regL2 = {'U','L','A','F','Duu','Daa','Dul'};

Note that we used cpdi instead of cpd for the ula factorization. The cpdi factorization type activates a specialized kernel for incomplete tensors, which usually improves the convergence behavior compared to cpd.

We can again use the sdf_check method to get an overview of the model:

sdf_check(model, 'print');

Factor               Size                 Comment
------------------------------------------------------------------------
U                    [146x2]
L                    [168x2]
A                    [5x2]
F                    [14x2]
Duu                  [1x2]
Daa                  [1x2]
Dul                  [1x2]

Factorization        Type       Factors              Comments
------------------------------------------------------------------------
ula (id = 1)         cpdi       U,L,A
uu (id = 2)          cpd        U,U,Duu
lf (id = 3)          cpd        L,F
aa (id = 4)          cpd        A,A,Daa
ul (id = 5)          cpd        U,L,Dul
reg (id = 6)         regL2      U,L,A,F,Duu,Daa,Dul

The sdf_nls method is used to compute the results for the model. The relative weights have been determined using grid search.

options.RelWeights = [1 0.005 0.001 0.005 0.001 1e-6];
[sol,out] = sdf_nls(model,options);

The ROC curve of the result can be seen in Figure 1. Using the extra information clearly improved the area-under-curve (AUC).