11
C H A P T E R 3
Data-Driven Compatibility
Modeling
3.1 INTRODUCTION
In this chapter, we aim to investigate the practical problem of clothing matching, without loss
of generality, by particularly answering the question “which bottom matches the given top.” e
problem we pose here primarily requires modeling human notion of the compatibility between
fashion items. However, modeling such subtle notion regarding compatibility is non-trivial due
to the following challenges. First, the compatibility between fashion items usually involves color,
material, pattern, shape, and other design factors. In addition, human notion of compatibility is
not absolute but relative, as people can only tell that a pair of items is more suitable to each other
than other items. erefore, how to accurately measure the compatibility between items consti-
tutes a tough challenge. Second, existing studies mainly focus on measuring the compatibility
based on images of items but fail to take the contextual information of fashion items into consid-
eration. As a matter of fact, similar to visual images, contextual descriptions also present the key
features of fashion items and thus can be helpful in distinguishing compatible fashion items. For
example, as shown in Figure 3.1, it may be hard to predict whether the “Hybrid tank top shirt” is
compatible with the “Fairy Tulle Black Skirt” with the current computer vision techniques due
to their compatible color. However, if we further take the contextual descriptions into account,
we can safely draw the conclusion that the lady skirt is not very suitable for the neutral top shirt.
erefore, how to model the intrinsic relatedness between the visual and contextual modalities
(a) Hybrid tank
top shirt.
(b) Fairy tulle
black skirt.
Figure 3.1: Illustration of the importance of contextual modality in compatibility measurement.
12 3. DATA-DRIVEN COMPATIBILITY MODELING
(a) Hybrid tank
top shirt.
(b) Fairy tulle
black skirt.
(a) Hybrid tan
k
top shirt.
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
(b
) Fairy
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
tu
ll
ll
ll
ll
ll
ll
ll
ll
ll
ll
ll
ll
e
e
e
e
e
e
e
e
black sk
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
ir
t.
t.
t.
t.
t.
t.
t.
t.
t.
t.
t.
t.
10,000
1,000
100
10
1
10,000
1,000
100
10
1
Number of Tops
Number of Bottoms
0 05 510 1015 1520 2025 2530 30 35 40 45 50 55 60
Number of Paired Bottoms Number of Paired Tops
(a) Top distribution. (b) Bottom distribution.
Figure 3.2: Distribution of tops and bottoms in our dataset. e Y-axis is in the logarithmic
scale.
of the same fashion item and further boost the performance is another crucial challenge. Last
but not least, according to our pilot study on Polyvore, only 1,181 (7.94%) of 14,871 fashion
tops have been paired with more than 2 bottoms, as can be seen from Figure 3.2. Such a sparse
relationship between fashion items makes the matrix factorization-based methods [94, 122] not
applicable and hence poses another challenge for us.
To address these challenges, we present a content-based neural scheme for clothing
matching (i.e., matching tops with bottoms), as shown in Figure 3.3. To deal with the sparse
relationship between tops and bottoms, the proposed scheme learns a latent compatibility space
to unify the complementary fashion items that come from the heterogenous spaces. In particu-
lar, the proposed scheme seamlessly integrates the multi-modal data (i.e., visual and contextual
modalities) of fashion items to comprehensively model the compatibility among fashion items.
Moreover, considering that the factors affecting the compatibility among items can be highly so-
phisticated, we employ the autoencoder neural model to exploit the latent compatibility space.
Meanwhile, to take full advantage of the rich implicit semantics regarding the compatibility
among fashion items on Polyvore, we further employ the advanced BPR framework [100] to
exploit the pairwise preference between complementary fashion items (i.e., tops and bottoms).
Ultimately, we propose a dual autoencoder network (BPR-DAE) for compatibility modeling,
which jointly models the coherent relationship between different modalities of fashion items
and the implicit preference among them.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.