52 5. PROTOTYPE-WISE INTERPRETABLE COMPATIBILITY MODELING
learning,
L
nmf
D
G
p
PH
p
2
F
C
G
u
UH
u
2
F
: (5.8)
It is intuitive that the top and bottom of one compatible prototype should be more com-
patible than those of the incompatible ones. erefore, we define the intrinsic compatibility for
each prototype p
l
(u
r
) as follows:
s
p
l
D
Qp
t
l
T
Qp
b
l
; s
u
r
D
Qu
t
r
T
Qu
b
r
; (5.9)
where s
p
l
and s
u
r
are the intrinsic compatibility for the compatible prototype p
l
and incompatible
prototype u
r
, respectively. Qp
t
l
, Qp
b
l
, Qu
t
r
, and Qu
b
r
are the hidden representations of p
t
l
, p
b
l
, u
t
r
, and
u
b
r
, respectively, which can be acquired based on Eq. (5.2).
To seamlessly integrate the latent prototype learning and compatible modeling, for each
sample .i; j; k/, we particularly define its most similar compatible and incompatible prototypes
p
l
and u
r
with the Euclidean distance, whose indexes l
and r
can be derived as follows:
8
ˆ
ˆ
<
ˆ
ˆ
:
d
p
.i; j; l/ D
f
t
i
f
b
j
p
t
l
p
b
l
2
; d
u
.i; k; r/ D
f
t
i
f
b
k
u
t
r
u
b
r
2
;
l
D arg min
l
d
p
.i; j; l/ ; r
D arg min
r
d
u
.i; k; r/:
(5.10)
In a sense, we expect that the intrinsic compatibility of the compatible prototype p
l
should be
higher than that of the incompatible one u
r
. erefore according to the BPR, we thus have the
following adaptive objective function:
L
proto
bpr
D
X
.i;j;k/2D
S
ln
s
p
l
s
u
r
;
(5.11)
where s
p
l
and s
u
r
can be obtained with Eq. (5.9). Interestingly, with L
item
bpr
and L
proto
bpr
, the com-
patibility modeling between fashion items and the prototype learning can be mutually promoted.
Ultimately, we obtain the final objective function as follows:
L D L
item
bpr
C L
proto
bpr
C L
nmf
;
(5.12)
where and are the non-negative trade-off hyperparameters to weigh the different compo-
nents of the objective function.
5.3.5 INTERPRETABLE ATTRIBUTE MANIPULATION
In order to transform the incompatible fashion item pairs into the compatible ones, we first
employ the L
p
compatible prototypes as templates to identify the discordant attributes. In par-
ticular, for the given negative (incompatible) top-bottom pair .t
i
; b
k
/, we particularly find the
5.3. METHODOLOGY 53
most similar compatible prototype p
l
according to Eq. (5.10). For simplicity, we divide p
l
into Z parts as follows:
p
l
D
h
p
1
l
I : : : I p
Q
l
I p
QC1
l
I : : : I p
Z
l
i
; (5.13)
where Z D 2Q. e first Q parts refer to the attribute representations of the top in prototype
p
l
, while the last Q parts correspond to that of the bottom in p
l
. In the same manner, the
negative top-bottom pair .t
i
; b
k
/ can be represented as follows:
g
ik
D
h
f
t
i
I f
b
k
i
D
h
g
1
ik
I : : : I g
Q
ik
I g
QC1
ik
I : : : I g
Z
ik
i
: (5.14)
Moreover, we define the attribute-wise difference between .t
i
; b
k
/ and p
l
as follows:
d
e
i; k; l
; z
D
g
z
ik
p
z
l
2
M
z
;
(5.15)
where d
e
.i; k; l
; z/ denotes the attribute difference between .t
i
; b
k
/ and p
l
regarding the z-
th attribute. We then identify the most discordant attribute that causes the incompatibility as
follows:
z
D arg max
z
d
e
.i; k; l
; z/: (5.16)
ereafter, to suggest the alternative item and make the compatible pair, we replace the
attribute representation g
z
ik
of .t
i
; b
k
/ with p
z
l
and hence obtain the manipulated semantic
attribute representation as follows:
Og
ik
D
8
<
:
h
O
f
t
i
I f
b
k
i
; if z
Q;
h
f
t
i
I
O
f
b
k
i
; if z
> Q;
(5.17)
where
O
f
t
i
and
O
f
b
k
are the manipulated semantic attribute representations of top t
i
and bottom b
k
,
respectively, with which we can retrieve the new fashion items to make a compatible match-
ing. In particular, if the discordant attribute manipulation needs to be taken on the top t
i
(i.e.,
z
Q), we can retrieve new tops t
i
0
’s by ranking the Euclidean distance d
p
’s between
O
f
t
i
and
the semantic attribute representations of training tops in the decent order. Otherwise, we can
retrieve new bottoms b
k
0
’s by ranking d
p
’s between
O
f
b
k
and the representations of training bot-
toms. e workflow of attribute manipulation is shown in Figure 5.3, and the algorithm of the
proposed method is summarized in Algorithm 5.2.
54 5. PROTOTYPE-WISE INTERPRETABLE COMPATIBILITY MODELING
g
ik
g
ik
gˆ
ik
p
L
p
p
l*
p
l*
p
1
p
2
Incompatible Pair
Eq. (
5.10)
Eq. (5.16)
Retrieval
Manipulated PairCompatible Prototypes
1
2
.
.
.
Q
Q + 1
.
.
.
z
1
2
.
.
.
Q
Q + 1
.
.
.
z
z*
Figure 5.3: Illustration of the workflow of attribute manipulation.
Algorithm 5.2 Interpretable Clothing Matching and Retrieval.
Input: D
S
D f.i; j; k/g, , , L
p
, L
n
Output: Parameters in MLP, parameters P, H, and U in NMF.
1: Initialize neural network parameters in MLP and NMF.
2: repeat
3: Randomly draw .i; j; k/ from D
S
4: Calculate l
and r
according to Eq. (5.10).
5: Update ‚ and ˆ according to Eq. (5.12).
6: until Converge
7: Calculate the incompatible attribute in the negative top-bottom pair according to Eq. (5.16)
8: Manipulate the incompatible attribute to get the new semantic attribute representation and
retrieve the new fashion item.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.