A Graph Kernel Approach for Detecting Core Patents and Patent Groups
CORE PATENT GROUPING
A Graph Kernel
Approach for
Detecting Core
Patents and
Patent Groups
Dohyun Kim, Korea Institute of Science and Technology Information
and Myongji University
Bangrae Lee, Korea Institute of Science and Technology Information
and University of Seoul
An approach
Hyuck Jai Lee, Korea Institute of Science and Technology Information
to discovering
Sang Pil Lee and Yeongho Moon, Korea Institute of Science and Technology
Information and Korea University of Science and Technology
core patents and
clustering patents
uses a patent citation
network in which
A
long with the influential nodes detection problem, the graph nodes clustering or community detection problem is one of the main topics in network
core patents are
analysis. Researchers in diverse fields, including computer science and physics,
represented as an
have studied the problem.1–3 A cluster or community has dense node-to-node
influential node and
connections within the community but sparse
connections with nodes from other communities.4 For example, a community detection
method in a citation network finds similar
patent (or paper) subgroups. However, the
methods used to discover influential nodes
and detect communities have previously been
studied separately, especially in a citation network.3,5–8 Simultaneously exploring influential
nodes and communities allows for the
easy discovery of significant nodes in each
patent groups as a
cluster of nodes.
44
Myong K. Jeong, Rutgers University
community and the recognition of the distribution of similar nodes around the significant
nodes, which isn’t true for existing community
detection methods or centrality measures.
Moreover, community detection in a citation
network has generally been conducted using
document similarity measures such as bibliographic coupling9 and co-citation.10 The bibliographic coupling measure counts the number
of common references used by two documents,
while a co-citation measure is defined as the
1541-1672/14/$31.00 © 2014 IEEE
Published by the IEEE Computer Society
IEEE INTELLIGENT SYSTEMS
frequency with which two documents
are cited together.11 The bibliographic
coupling and co-citation are the most
popular citation-based similarity measures of documents. Although many
studies12-13 have shown that bibliographic coupling and co-citation can be
used to cluster patents or papers successfully, these two similarity measures
have the drawback that they inevitably overlook the case that two complementary documents are connected by
an edge. Hence, our proposed method
focuses on simultaneously discovering
influential nodes (that is, core patents)
and detecting communities (that is, patent groups) in a patent citation network over all nodes. Additionally, the
proposed method tries to alleviate the
complementary edge issue.
For the purpose of the present study,
we introduce a kernel k-means clustering algorithm with a graph kernel. A
graph kernel in network analysis helps
to compute implicit similarities between
patents in a high-dimensional feature
space.3 Then, the kernel k-means clustering algorithm performs patent clustering
based on a similarity matrix obtained
from the graph kernel. Namely, kernel
k-means clustering based on graph kernel searches clusters and their centers
in the feature space, not in the original
input space. The cluster center is obtained to minimize the sum of distances
from other nodes within a cluster in feature space. This means the center is the
most tightly connected to other patents
by citations. For this reason, we define
the center of each cluster as the cluster’s core patent. The concept is similar
to that of closeness centrality, which explores a node with the minimum length
sum of shortest path to other nodes
from the target node. The difference is
that the kernel k-means clustering uses
the distance in feature space, which is
calculated using a graph kernel, instead
of the length of the shortest path. A similar approach has been applied in the
JULY/AUGUST 2014
declarative modeling field for searching
representative designers.14
Counting the number of citations of
each patent can be one of the alternative approaches to finding the cluster
center in each cluster. However, this
approach counts only directly connected edges and thus can’t distinguish
between similar and complementary
documents. In general, the similar documents have one or more directly or indirectly connected common references,
while complementary documents have
only direct citations. Therefore, the
proposed graph kernel-based approach
prevents the complementary patents,
which aren’t similar, from being classified as similar by considering even indirectly connected edges. That is, the
proposed graph kernels, including the
exponential diffusion kernel and von
Neumann diffusion kernel, compute
the summation of rth power of adjacent
matrix A(r = 1, …, ∞) of which value
means the weighted sum over all paths
from one node to another.15 This approach leads to lower similarity scores
for complementary edges and higher
similarity scores for similar edges and
then selects the most tightly connected
patents to others as a cluster center. After all, the proposed approach alleviates
the complementary edge issue.
Note that, when using kernel k-means
clustering, the center of each cluster is
detected, and each cluster is built based
on each cluster’s center. This procedure
is performed iteratively until convergence is complete. We consider the finally obtained cluster centers as the
core patents and the determined clusters as the patent communities. From
this perspective, building clusters and
detecting influential nodes are considered a simultaneous work undertaken
during the clustering stage.
However, in kernel k-means clustering, the obtained center in each cluster
isn’t always consistent with one of the
given patents. Therefore, the center in
www.computer.org/intelligent
each cluster is regarded as a virtual,
core patent, and we might consider
the closest patent from the center as a
real, core patent.
Proposed Method
The proposed method consists of
• computation of a graph kernel function over all nodes in a network,
• clustering of nodes using kernel
k-means clustering,
• computation of the distance from
each node to its cluster center, and
• identification of an influential node
for each cluster based on the distance from each center.
In the following text, these steps are
explained in detail.
Graph Kernel
Similarity-based clustering approaches,
such as k-means clustering, generally use
the similarities or distances between observations as shown in input space. The
k-means clustering algorithm can be extended using a kernel function, which
helps to compute implicit similarities
between observations in a high-dimensional feature. The k-means clustering algorithm using a kernel function is
called kernel k-means clustering.
In kernel k-means clustering for a
network analysis, a similarity measure
between graph nodes integrating indirect paths should be calculated. Among
many kernels, graph kernel functions
aim to calculate the similarity between graph nodes. Therefore, kernel
k-means clustering in network analysis
uses graph kernels; however, the kernel function for kernel k-means clustering is required to meet the following
conditions:16
• kernel should be a positive, semidefinite matrix; and
• kernel should be a similarity measure, not a dissimilarity measure.
45
CORE PATENT GROUPING
Therefore, some kernels, such as
the Laplacian kernel, which calculates
dissimilarity between nodes, should
consider techniques that reverse the
kernel to meet the second condition.17
A representative graph kernel is an
exponential diffusion kernel that uses
an adjacency matrix of the graph.15,18
The exponential diffusion kernel is
defined as follows:
∞
K EXP =
∑
k=0
α kA k
= exp(α A),
k!
A k,
aijk
where the elements of
represent
the number of paths from node i to
node j with k transitions; this kernel
gives more weight to two nodes with
shorter paths. Note that the exponential function gives the kernel positive
values, indicating that the kernel is
clearly positive and semidefinite.
By changing the parameter of the
exponential diffusion kernel, we can
obtain the von Neumann diffusion
kernel.18,19 The von Neumann diffusion kernel has an exponential parameter a k, instead of a k /k!:
∞
K VN =
∑α
−1
A = (I − α A) .
If a has the range of value 0 < α <
-1
, the kernel is well-defined, posi2
tive, and definite.15
For the present study, we can use
another kernel—random walk with
restart kernel—introduced by Jia-Yu
Pan and his colleagues.20 The random
walk with restart kernel has an advantage: The kernel can capture the
graph’s global structure and the
multifaceted relationship between two
nodes. The random walk with restart kernel uses the random walk
from node i to adjacent node j
with probability pij = P(S(t + 1) =
j|S(t) = i) = aij/ai., where aij is an
46
n
ai⋅ =
∑ aij , and S(t) is the state of the
j =1
random walker at time t. In addition, we
assume that the random walker comes
back to node i with the probability 1 - g :
X(0) = ei, X(t + 1) = g PTX(t) = (1 - g)ei,
where ei is a starting vector where the
ith element is 1 and 0 for others. The
probability of finding the random
walker on each node j when it starts
at node i by steady state solution X(t
+ 1) = X(t) = X is given as follows:18,21
X = (1 - g )(I - g PT) -1ei.
From this probability, we can consider (I - g P) -1 to be a graph kernel
that indicates similarities between
nodes. To show that the ith row contains the similarity to node i in a similar way to other kernel functions, P T
was transposed to P.
Once these similarities are computed using a graph kernel, we can
exploit them to find clusters using the
kernel k-means clustering method.
function to transform the original input xi into a high-dimensional feature
space f(xi), and nc is the number of
nodes belonging to Cc. We can rewrite D as follows:
(
clustering 22
Kernel k-means
uses an
iterative algorithm that minimizes the
sum of within-cluster inertia in the
feature space as
D
({
}
k
Cc c =1
)=∑ ∑
k
2
φ ( x i ) − mc ,
c =1 x i ∈Cc
where mc is the center of cluster c in the
feature space and can be calculated as
∑ φ (xi )
mc =
x i ∈Cc
,
nc
where xi is the node vector corresponding to node i, f(.) is a mapping
www.computer.org/intelligent
k
( )
∑
T
φ (xi ) φ x j
2
x j ∈Cc
T
φ ( x i ) φ ( x i ) −
k
n
c
.
T
φ x j φ ( xl )
c =1 x i ∈Cc
C
∈
x
,
x
j
l
c
+
nc2
∑∑
( )
∑
Note that the inner product f(xi)Tf(xj)
can be replaced by the kernel function
Kij, which is called the kernel trick. The
value of kernel function K can be calculated using various kernel functions,
instead of mapping our data via f(.)
and then computing the inner product.
Therefore, the separate nonlinear mapping f(.) need never be explicitly computed. The kernel function performs a
role to map data from the input space to
a high-dimensional feature space. Then,
measure D can be rewritten as follows:
(
D {Cc }c =1
=
Kernel k-Means Clustering
)
D {Cc }c =1 =
k
)
∑ x ∈C Kij ∑ x ,x ∈C K jl .
2
K ii −
c =1 x i ∈Cc
k
k k
k=0
A
element of the adjacency matrix A,
∑∑
j
c
+
nc
j
l
c
nc2
The distance in feature space between
node f(xi) and the center of each cluster
is compared. Each node is then assigned
to its nearest cluster as follows:
bi = arg minc φ (x i ) − mc
2
∑ x ∈C Kij ∑ x ,x ∈C K jl
2
= arg minc K ii −
j
nc
c
+
j
l
nc2
c
,
where bi means the assigned cluster
label for node i. Then, the iteration for
kernel k-means clustering is repeated
until no more labels of each node are
changed. We summarize the procedures for kernel k-means clustering:
IEEE INTELLIGENT SYSTEMS
Input
K: kernel matrix
k: number of clusters
Output
C1, C2, …, Ck : clusters of
nodes
Procedures
1. Initialize each cluster:
(0)
(0)
C(0)
1 ,C2 ,…,C k .
2. Assign each node to its
nearest cluster:
2
bi = arg minc φ ( xi ) − mc
= arg minc
∑ x ∈C
2
× Kii −
(t)
c
j
nc
Kij
+
∑ x ,x ∈C
j
l
(t)
c
nc2
K jl
.
3.
Update the clusters:
C(1t +1),C(2t +1),…,C(kt +1).
4. Repeat Steps 2 and 3 until
no more nodes are assigned
to a cluster different from
the current one.
Patent groups are discovered as the result of kernel k-means clustering, where
each patent’s group is determined by
comparing distances in the feature space
from the patent to each group’s center. The patent is assigned to the group
whose center is closest to that patent.
Influential Measures for the
Detection of a Core Patent
Once patent clusters are detected using
kernel k-means clustering as described
previously, the core patent in each cluster is explored again, because the obtained center in each cluster isn’t always
consistent with one of the given nodes.
Usually, existing community detection
methods in a citation network don’t deal
with the problem of searching influential
nodes; however, in the present study, the
problem of detecting influential nodes,
along with the node-clustering problem,
is discussed. Using this parallel approach
gives us the advantage of being able to
recognize the distribution of similar
JULY/AUGUST 2014
nodes around influential nodes. The distances between the nodes and the center
of each cluster obtained from the kernel
k-means clustering are calculated for the
influential nodes. The closest node from
the center of each cluster is considered
to be an influential node of each cluster. Note that there’s one difference between the measure to build the clusters
in the previous “Kernel k-Means Clustering” subsection and the measure to
search the core patent of each cluster in
this section. The cluster building measure
aims to find a cluster to which node i belongs based on the sum of within-cluster
inertia, while the core patent searching
measure seeks to find a real node that’s
closest to the virtual core patent node in
each cluster. Our proposed measure for
an influential node—kernel-mean-based
centrality (KMBC)—is determined according to the selected kernel. If the
exponential diffusion kernel (EXP) is selected, an influential node wd for a specific cluster Cd ∈ {Cc }kc =1 is identified by
the following measure:
( wd )EXP
= arg min i φ (x i ) − md
2
∑
2
( K EXP )ij
x j ∈Cd
( K EXP )ii −
nd
= arg min i
( K EXP ) jl
x j , x l ∈Cd
+
2
n
d
∑
∑
2
(exp(α A))ij
x j ∈Cd
( exp(α A))ii −
nd
.
= arg min i
exp(α A)) jl
(
x j , x l ∈Cd
+
nd 2
∑
Using the von Neumann (VN) diffusion kernel, the measure is given as
follows:
The random walk with restart
(RWR) kernel defines the measure for
an influential node as follows:
( wd )RWR
(
∑
(I − γ P)−1
2
x j ∈Cd
−1
−
(I − γ P)
ii
nd
= arg min i
−1
(I − γ P)
x j , x l ∈Cd
jl
+
nd 2
(
)
∑
(
)
)
ij
.
From these measures, we find the
closest node from the center of each
cluster and consider the node as a
core patent.
Experimental Results
The proposed methods based on kernel k-means clustering, in combination with the previous three graph
kernels, are optimized in terms of
modularity, which quantifies the quality of a division of a network into
communities. Also, the core patents
obtained by the proposed KMBC are
compared to those of commonly used
centralities. Note that the kernel matrix based on the adjacency matrix A
from the directed network, such as a
citation network, is not symmetric,
and hence, we define a symmetric,
undirected version of an adjacency
matrix for the directed network as
A + AT ,
2
which leads to producing the symmetric kernel matrix.
Additionally, the Gaussian radialbasis-function (RBF) kernel, which is
not a graph kernel, is used for comparison. The Gaussian RBF kernel is
presented as follows:
(
)
K RBF = exp − u1 − u2 2τ 2 ,
( wd )VN
(
∑
(I − α A)−1
2
x j ∈Cd
−1
−
(I − α A)
ii
nd
= arg min i
−1
(I − α A)
x j , x l ∈Cd
jl
+
nd 2
(
)
∑
(
)
www.computer.org/intelligent
)
ij
.
where t is the width parameter that
controls the amplitude of the RBF, and
u1 and u2 are the columns of A. The
dataset used for the computational experiments includes US patents in the
47
CORE PATENT GROUPING
Table 1. Modularity value and optimal number of clusters and kernel parameters.
Exponential
von Neumann
Random walk
with restart
Radial basisfunction (RBF)
Kernel parameter
10 -1
10 -2
10 -1
101
Optimal number of
clusters
3
27
16
25
0.258
0.236
0.222
0.174
Kernel
Modularity
Table 2. Core patents obtained from kernel-mean-based centrality with exponential
diffusion kernel (KMBC_EXP) and centrality measures.
KMBC_EXP
Degree
Closeness
Weighted
reachability
US5862260
US5892900
US6314409
US5892900
US5982891
US5943422
US5920861
US5910987
US5915019
US5917912
US6185683
US5949876
US6112181
US5862260
US6226618
US6122403
US5745604
US6157721
US5745604
US5349655
US5892900
US5982891
US5949876
US5910987
US5915019
US5862260
US5917912
US5943422
US5920861
US5832119
US6112181
US6185683
US6157721
US5349655
US5745604
US5892900
US5862260
US5943422
US5920861
US5982891
US5949876
US5915019
US5910987
US5832119
US5822436
US5917912
US6112181
US6185683
area of information and security issued from 1976–2007. We used only
the top 1 percent of frequently cited
US patents from 2003–2007 for our
study. The dataset has 40,982 citations
from 19,677 patents. For computational efficiency, we selected all patents that cited the most-cited patent,
US5349655, directly and indirectly
among the extracted top 1 percent of
US patents, and then only used the
patents with two or more citations.
The patents and citations from the
corresponding patents constitute the
nodes and edges in a citation network,
respectively.
Performance Measure
Modularity is used to evaluate the
quality of a partitioning of a network
into communities. This measure is
frequently used in network analysis
research.23,24 Modularity has been successfully used to catch the reasonable
structure of communities.25,26 A good
partition—which has a high value of
modularity—is one in which dense
48
internal connections exist between
nodes within communities, but only
sparse connections exist between different communities.25 Modularity for
a directed network, such as a citation
network, is defined as follows:23,24
Q=
1
m
∑ Aij −
i, j
kiout kjin
δ c , c ,
m i j
(
)
where ci is the community to which
in
node i belongs; and kiout and kj are
the out-degree of node i and the indegree of node j, respectively. Also,
m is the total number of edges in
the network, and d(ci, cj) is the Kronecker delta symbol, which is defined
as 1 if ci is equal to cj and 0 otherwise. In the modularity measure,
kiout kjin / m is the probability of an
edge between two nodes i and j under randomization. Therefore, the
high modularity value tells us when
there are more edges within communities than we would expect on the
basis of chance.24
www.computer.org/intelligent
Results
The proposed method, combined with
the four kernel functions, is optimized
in terms of modularity. In the experiment, kernel parameters (a for von
Neumann and exponential diffusion,
g for random walk with restart, and t
for Gaussian RBF) are varied in such
a way that a, g, t = 10-3, 10-2, 10-1, 1,
101. In addition, the number of clusters is varied from two to 50.
For each kernel function, we selected
the optimal number of clusters and
kernel parameters by maximizing the
modularity value. Table 1 shows the optimized number of clusters and kernel
parameters for each kernel function and
their modularity values. Depending on
the kernel functions, the optimal numbers of clusters are different. The proposed KMBC_EXP produces the largest
modularity value and yields the best
community structure over other methods, although it has the smallest number of clusters. The RBF kernel, which
isn’t a graph kernel, didn’t achieve good
clustering performance in comparison
to graph kernels. This result indicates
that the graph kernel is effective for
clustering in a citation network.
Table 2 shows the core patents detected from KMBC using the exponential kernel function. It’s not easy to
measure the accuracies of the proposed
approaches for the core patents. Therefore, we compared the selected influential nodes using the proposed method
with 15 core patents by popular centrality measures. Using this comparison,
we can verify that the influential nodes
selected by the proposed method are
meaningful. The obtained patents are
compared to those from commonly used
centrality measures, such as the degree,
adjusted closeness, and weighted-reachability measures.27 The degree centrality
measure is the number of nodes connected to the target node, and the adjusted closeness measure emphasizes
the length of the shortest path from the
IEEE INTELLIGENT SYSTEMS
TabIe 3. The communities of patents obtained from KMBC_EXP.
Community
Name
Topic
k=3
(the optimal number of clusters)
k=4
k=5
Core patents
by the
proposed
method
Patents with
the highest
degree
centrality
Core patents
by the
proposed
method
Patents with
the highest
degree
centrality
Core patents
by the
proposed
method
Patents with
the highest
degree
centrality
Cluster 1
Information hiding:
digital watermarking or
steganography
US5862260
US5862260
US5613002
US5613002
US5349655
US5349655
Cluster 2
System for secure
transaction management
or rights protection
US5892900
US5892900
US5892900
US5745604
US5613002
US5613002
Cluster 3
US6314409
US6314409
US6122403
US5862260
US5822436
US5822436
Cluster 4
System for access control,
data management, or
communication network
_
_
_
_
_
US6226618
_
US5745604
_
US6226618
_
US5892900
Cluster 5
US6226618
US6226618
target node to all other nodes. Additionally, weighted reachability is the centrality measure that reflects the concept that
the directly cited patents are more influential than indirectly cited patents.27
Note that the core patents from commonly used centrality measures in a citation network are sorted according to
their importance, while the patents from
KMBC are sorted in the order of publication. This is because kernel k-means
clustering can detect only the core patents of each cluster but can’t quantify
their importance. Although ordering the
patents according to their importance
could be important in a global network,
it might not be helpful if someone intends to find a particular type of patent
community and its representative patents. In that case, the neighboring patent information is more useful than the
order of importance. The key contribution in this work is that even though the
order of importance of patents can’t be
determined, the neighbors of the specific patents—which deal with a similar
topic—can be found, as well as the representative patents among neighbors.
In Table 2, the bolded patents simultaneously belong to the lists of both
KMBC_EXP and commonly used
centralities. From Table 2, we can see
that two core patents detected from
KMBC_EXP are included in the top
15 patents of the degree for adjusted
JULY/AUGUST 2014
closeness and weighted reachability. Notably, the most cited patent,
US5349655, is directly or indirectly
not included in the list of KMBC_EXP.
The three listed patents in Table 2 are
influential in each cluster, but they
might not be influential in a global
network and vice versa—that is, the
influential patents in a global network
might not be influential in each cluster with similar topics. US6314409 is
an example of an influential patent
within a local network. US6314409 is
not in the top 15 patents of the degree
centrality in a global network, but it
has the largest number of in-degree citations in its corresponding cluster.
In addition, we investigated patents
in each cluster and observed that patents of each cluster deal with a similar
topic. Table 3 shows the main topic
of each cluster (see the results for k =
3 in Table 3). Clusters 1 and 2 have
patents about information hiding and
systems for secure transaction management, respectively. Cluster 3 simultaneously deals with systems for
access control and data management.
Figure 1 shows the citation network
in a circular graph, grouped using the
kernel k-means clustering algorithm
with an exponential kernel, which is
obtained using the NetMiner program
(see www.netminer.com). The three colors represent three clusters of patents,
www.computer.org/intelligent
subdivided into fields in the outer ring.
The patents with the same color and
shape belong to the same group, and
the patents with the larger shapes represent the core patents of each group. The
line represents the citation (that is, link)
between patents. For the sake of brevity,
only patents with 15 or more incoming
edges are plotted. This citation network
shows the edge concentration in each
cluster. From Figure 1, we can see that
the clustering process is well conducted
by observing that all clusters, except
cluster 3, have dense node-to-node connections within the clusters and sparse
connections with other clusters.
We can see that two core patents,
US5862260 and US5892900, have
more internal edges than other patents
within a group, while there’s little edge
intensity in the patent US6314409, selected from cluster 3. This is due to the
fact that many patents that cite patent
US6314409 had already been removed
when we chose the patents for the plot;
however, US6314409 is admittedly the
core patent, with the largest number of
in-degree citations in cluster 3. Table 3
shows that the core patent of each cluster, obtained from our approach with the
optimal number of clusters (k = 3), corresponds with the node with the highest
number of internal edges in each cluster. From the results, we can verify that
the proposed method has appropriately
49
CORE PATENT GROUPING
Cluster 1
recognized. The experimental results
revealed that this approach can detect
influential and meaningful patents and
patent groups efficiently. The methods
developed in this study enable a person to detect core patents and patent
groups in a citation network. Valuable
areas of future research could include
validating the findings using additional
data sets, and developing a method to
detect the change of core patents and
patent groups over a period of time.
Cluster 2
Clusterr 3
References
Figure 1. A citation network from kernel k-means clustering using exponential
kernel. The three colors represent three clusters of patents, subdivided into fields
in the outer ring. The patents with the same color and shape belong to the same
group, and the patents with the larger shapes represent the core patents of each
group. The line represents the citation (that is, link) between patents.
detected the core patents. The core patents from our approach and degree centrality when we use the different number
of clusters or different graph kernels,
can be different (see the results for k =
4 and 5 in Table 3). Note that the core
patents from the degree centrality were
obtained after we first found the technology groups using our proposed graph
kernel approach. The core patents obtained from the simple degree centrality
in the original space in Table 2 are quite
different from the results in Table 3.
The proposed algorithm could be
useful when we need to learn the leading patent within each patent group.
Using the database of US patents, we’ve
investigated the ratio of patents belonging to each cluster among the top
15 patents of each centrality in Table
2. In all centralities, the core patents of
cluster 2 have a portion more than 70
percent, while there are no core patents
within cluster 3 among the 15 patents.
In this way, the core patents obtained
50
from the global networks could be
concentrated in a specific cluster. Our
approach to simultaneously finding the
patent communities and their leading
patents can prevent this problem and
be helpful when exploring the core
patent in each specific area.
T
he objective of this study was to
develop algorithms to simultaneously detect core patents and patent groups in a complex network. To
this end, the algorithm for the simultaneous detection of core patents and
patent groups based on kernel k-means
clustering with a graph kernel was introduced. This method uses a graph
kernel because it can compute implicit similarities between patents in a
high-dimensional feature space. The
proposed method helps core patents
in each group to be discovered easily and the distribution of the similar
patents around the core patents to be
www.computer.org/intelligent
1. M.E.J. Newman, “Detecting Community
Structure in Networks,” The European
Physical J. B, vol. 38, 2004, pp. 321-330.
2. Y. Yang et al., “Personalized Email Prioritization Based on Content and Social
Network Analysis,” IEEE Intelligent
Systems, vol. 25, no. 4, 2010, pp. 12-18.
3. Z. Yang et al., “Social Community
Analysis via a Factor Graph Model,”
IEEE Intelligent Systems, vol. 26, no. 3,
2011, pp. 58-65.
4. L. Yen et al., “Graph Nodes Clustering
with the Sigmoid Commute-Time Kernel: A Comparative Study,” Data and
Knowledge Eng., vol. 68, no. 3, 2009,
pp. 338-361.
5. P. Chen and S. Redner, “Community
Structure of the Physical Review Citation Network,” J. Informetrics, vol. 4,
2010, pp. 278-290.
6. D. Kim et al., “Automated Detection
Of Influential Patents Using Singular Values,” IEEE Trans. Automation Science and
Eng., vol. 9, no. 4, 2012, pp. 723-733.
7. R. Rousseau, “The Gozinto Theorem: Using Citations to Determine
Influences on a Scientific Publication,”
Scientometrics, vol. 11, nos- 3-4, 1987,
pp. 217-229.
8. C.A. Yeung et al., “Measuring Expertise
in Online Communities,” IEEE Intelligent
Systems, vol. 26, no. 1, 2011, pp. 26-32.
9. M.M. Kessler, “Bibliographic Coupling
between Scientific Papers,” American
Documentation, vol. 14, no. 1, 1963,
pp. 10-25.
IEEE INTELLIGENT SYSTEMS
THE AUTHORS
Dohyun Kim is a senior researcher at the Korea Institute of Science and Technology In-
10. H. Small, “Co-Citation in the Scientific Literature: A New Measure of the
Relationship between Two Documents,”
J. Am. Soc. for Information Science, vol.
24, no. 4, 1973, pp. 265-269.
11. L. Egghe and R. Rousseau, “CoCitation, Bibliographic Coupling and
a Characterization of Lattice Citation
Networks,” Scientometrics, vol. 55,
no. 3, 2002, pp. 349-361.
12. K.W. Boyack and R. Klavans, “Co-Citation Analysis, Bibliographic Coupling,
and Direct Citation: Which Citation
Approach Represents the Research
Front Most Accurately?” J. Am. Soc. for
Information Science and Technology,
vol. 61, no. 12, 2010, pp. 2389-2404.
13. R. Klavans and K.W. Boyack, “Quantitative Evaluation of Large Maps of
Science,” Scientometrics, vol. 68, no. 3,
2006, pp. 475-499.
14. N. Doulamis et al., “Collaborative
Evaluation Using Multiple Clusters in
a Declarative Design Environment,”
Artificial Intelligence Techniques for
Computer Graphics, vol. 159, 2008,
pp. 141-157.
15. J. Kandola, N. Cristianini, and J.
Shawe-Taylor, “Learning Semantic
Similarity,” Proc. Advances in Neural
Information Processing Systems, 2002,
pp. 657-664.
16. C.M. Bishop, Pattern Recognition and
Machine Learning, Springer, 2006.
17. C. Gao et al., “Graph Ranking for
Exploratory Gene Data Analysis,” BMC
Bioinformatics, vol. 10, supplement 11,
2009; www.biomedcentral.com/14712105/10/S11/S19.
18. F. Fouss et al., “An Experimental Investigation of Kernels on Graphs for Collaborative Recommendation and Semisupervised Classification,” Neural Networks,
vol. 31, July 2012, pp. 53-72.
19. R.I. Kondor and J. Lafferty, “Diffusion
Kernels on Graphs and Other Discrete
Structures,” Proc. 19th Int’l Conf. Machine Learning, 2002, pp. 315-322.
20. J.-Y. Pan et al., “Automatic Multimedia
Cross-Modal Correlation Discovery,”
JULY/AUGUST 2014
formation. His research interests include statistical data mining, network data analysis,
and bibliometric analysis. Kim has a PhD in industrial engineering from the Korea Advanced Institute of Science and Technology. Contact him at ftgog@kisti.re.kr.
Bangrae Lee is a senior researcher in the Information Analysis Center of the Korea Institute of Science and Technology Information. His research interests include data mining, informetrics, scientometrics, and social network analysis. Lee has an MS in robotics
from the Korea Advanced Institute of Science and Technology. Contact him at brlee@
kisti.re.kr.
Hyuck Jai Lee is a principal researcher at the Korea Institute of Science and Technol-
ogy Information. His research interests include information visualization, social network
analysis, and research evaluation based on bibliometric analysis. Lee has a PhD in chemistry from Sogang University, Seoul, Korea. Contact him at hlee@kisti.re.kr.
Sang Pil Lee is a research fellow at the Information Analysis Center at the Korea Institute
of Science and Technology Information. His research interests include emerging technology analysis, knowledge science, bibliometric analysis, scientometrics, and data mining.
Lee has a PhD in applied biotechnology from Osaka University, Japan. Contact him at
splee@kisti.re.kr.
Yeongho Moon is a director at the Information Analysis Center at the Korea Institute of
Science and Technology Information. He is also a vice president of Korea Technology Innovation Associate. His research interests include emerging technology analysis, knowledge geometrics analysis, and data mining analysis and information service. Moon has a
PhD in construction and environment engineering from the Korea Advanced Institute of
Science and Technology. Contact him at yhmoon@kisti.re.kr.
Myong K. Jeong is an associate professor in the department of industrial and systems engineering and the Rutgers Center for Operations Research at Rutgers University. His research interests include statistical data mining, recommendation systems, machine health
monitoring, and sensor data analysis. Jeong has a PhD in industrial and systems engineering from the Georgia Institute of Technology. He received the Freund International
Scholarship and National Science Foundation Career Award and is an associate editor of
the IEEE Transactions on Automation Science and Engineering and International Journal of Quality, Statistics, and Reliability. He is a senior member of IEEE. Contact him at
mjeong@rci.rutgers.edu.
Proc. 10th ACM SIGKDD Int’l Conf.
Knowledge Discovery and Data Mining, 2004; http://dl.acm.org/citation.
cfm?id=1014135.
21. F. Fouss et al., “Random-Walk Computation of Similarities between Nodes of a
Graph with Application to Collaborative
Recommendation,” IEEE Trans. Knowledge and Data Eng., vol. 19, no. 3, 2007,
pp. 355-369.
22. B. Schölkopf, A. Smola, and K.-R.
Müller, “Nonlinear Component Analysis as a Kernel Eigenvalue Problem,”
Neural Computation, vol. 10, no. 5,
1998, pp. 1299-1319.
23. E.A. Leicht and M.E.J. Newman,
“Community Structure in Directed Networks,” Physical Rev. Letters, vol. 100,
no. 11, 2008; http://dx.doi.org/10.1103/
PhysRevLett.100.118703.
www.computer.org/intelligent
24. V. Nicosia et al., “Extending the Definition of Modularity to Directed Graphs
with Overlapping Communities,” J.
Statistical Mechanics: Theory and
Experiment, 2009, P03024.
25. M.E.J. Newman and M. Girvan, “Finding the Evaluating Community Structure
in Networks,” Physical Review E, vol.
69, 2004, 026113; doi:10.1103/PhysRevE.69.026113.
26. M.E.J. Newman, “Finding Community
Structure in Networks Using the Eigenvectors of Matrices,” Physical Review
E, vol. 74, 2006, 036104; http://dx.doi.
org/10.1103/PhysRevE.74.036104 .
27. O. Kwon et al., “A Method to Make the
Genealogical Graph of Core Documents
from the Directed Citation Network,”
Information, vol. 12, no. 4, 2009,
pp. 875–888.
51
A Graph Kernel
Approach for
Detecting Core
Patents and
Patent Groups
Dohyun Kim, Korea Institute of Science and Technology Information
and Myongji University
Bangrae Lee, Korea Institute of Science and Technology Information
and University of Seoul
An approach
Hyuck Jai Lee, Korea Institute of Science and Technology Information
to discovering
Sang Pil Lee and Yeongho Moon, Korea Institute of Science and Technology
Information and Korea University of Science and Technology
core patents and
clustering patents
uses a patent citation
network in which
A
long with the influential nodes detection problem, the graph nodes clustering or community detection problem is one of the main topics in network
core patents are
analysis. Researchers in diverse fields, including computer science and physics,
represented as an
have studied the problem.1–3 A cluster or community has dense node-to-node
influential node and
connections within the community but sparse
connections with nodes from other communities.4 For example, a community detection
method in a citation network finds similar
patent (or paper) subgroups. However, the
methods used to discover influential nodes
and detect communities have previously been
studied separately, especially in a citation network.3,5–8 Simultaneously exploring influential
nodes and communities allows for the
easy discovery of significant nodes in each
patent groups as a
cluster of nodes.
44
Myong K. Jeong, Rutgers University
community and the recognition of the distribution of similar nodes around the significant
nodes, which isn’t true for existing community
detection methods or centrality measures.
Moreover, community detection in a citation
network has generally been conducted using
document similarity measures such as bibliographic coupling9 and co-citation.10 The bibliographic coupling measure counts the number
of common references used by two documents,
while a co-citation measure is defined as the
1541-1672/14/$31.00 © 2014 IEEE
Published by the IEEE Computer Society
IEEE INTELLIGENT SYSTEMS
frequency with which two documents
are cited together.11 The bibliographic
coupling and co-citation are the most
popular citation-based similarity measures of documents. Although many
studies12-13 have shown that bibliographic coupling and co-citation can be
used to cluster patents or papers successfully, these two similarity measures
have the drawback that they inevitably overlook the case that two complementary documents are connected by
an edge. Hence, our proposed method
focuses on simultaneously discovering
influential nodes (that is, core patents)
and detecting communities (that is, patent groups) in a patent citation network over all nodes. Additionally, the
proposed method tries to alleviate the
complementary edge issue.
For the purpose of the present study,
we introduce a kernel k-means clustering algorithm with a graph kernel. A
graph kernel in network analysis helps
to compute implicit similarities between
patents in a high-dimensional feature
space.3 Then, the kernel k-means clustering algorithm performs patent clustering
based on a similarity matrix obtained
from the graph kernel. Namely, kernel
k-means clustering based on graph kernel searches clusters and their centers
in the feature space, not in the original
input space. The cluster center is obtained to minimize the sum of distances
from other nodes within a cluster in feature space. This means the center is the
most tightly connected to other patents
by citations. For this reason, we define
the center of each cluster as the cluster’s core patent. The concept is similar
to that of closeness centrality, which explores a node with the minimum length
sum of shortest path to other nodes
from the target node. The difference is
that the kernel k-means clustering uses
the distance in feature space, which is
calculated using a graph kernel, instead
of the length of the shortest path. A similar approach has been applied in the
JULY/AUGUST 2014
declarative modeling field for searching
representative designers.14
Counting the number of citations of
each patent can be one of the alternative approaches to finding the cluster
center in each cluster. However, this
approach counts only directly connected edges and thus can’t distinguish
between similar and complementary
documents. In general, the similar documents have one or more directly or indirectly connected common references,
while complementary documents have
only direct citations. Therefore, the
proposed graph kernel-based approach
prevents the complementary patents,
which aren’t similar, from being classified as similar by considering even indirectly connected edges. That is, the
proposed graph kernels, including the
exponential diffusion kernel and von
Neumann diffusion kernel, compute
the summation of rth power of adjacent
matrix A(r = 1, …, ∞) of which value
means the weighted sum over all paths
from one node to another.15 This approach leads to lower similarity scores
for complementary edges and higher
similarity scores for similar edges and
then selects the most tightly connected
patents to others as a cluster center. After all, the proposed approach alleviates
the complementary edge issue.
Note that, when using kernel k-means
clustering, the center of each cluster is
detected, and each cluster is built based
on each cluster’s center. This procedure
is performed iteratively until convergence is complete. We consider the finally obtained cluster centers as the
core patents and the determined clusters as the patent communities. From
this perspective, building clusters and
detecting influential nodes are considered a simultaneous work undertaken
during the clustering stage.
However, in kernel k-means clustering, the obtained center in each cluster
isn’t always consistent with one of the
given patents. Therefore, the center in
www.computer.org/intelligent
each cluster is regarded as a virtual,
core patent, and we might consider
the closest patent from the center as a
real, core patent.
Proposed Method
The proposed method consists of
• computation of a graph kernel function over all nodes in a network,
• clustering of nodes using kernel
k-means clustering,
• computation of the distance from
each node to its cluster center, and
• identification of an influential node
for each cluster based on the distance from each center.
In the following text, these steps are
explained in detail.
Graph Kernel
Similarity-based clustering approaches,
such as k-means clustering, generally use
the similarities or distances between observations as shown in input space. The
k-means clustering algorithm can be extended using a kernel function, which
helps to compute implicit similarities
between observations in a high-dimensional feature. The k-means clustering algorithm using a kernel function is
called kernel k-means clustering.
In kernel k-means clustering for a
network analysis, a similarity measure
between graph nodes integrating indirect paths should be calculated. Among
many kernels, graph kernel functions
aim to calculate the similarity between graph nodes. Therefore, kernel
k-means clustering in network analysis
uses graph kernels; however, the kernel function for kernel k-means clustering is required to meet the following
conditions:16
• kernel should be a positive, semidefinite matrix; and
• kernel should be a similarity measure, not a dissimilarity measure.
45
CORE PATENT GROUPING
Therefore, some kernels, such as
the Laplacian kernel, which calculates
dissimilarity between nodes, should
consider techniques that reverse the
kernel to meet the second condition.17
A representative graph kernel is an
exponential diffusion kernel that uses
an adjacency matrix of the graph.15,18
The exponential diffusion kernel is
defined as follows:
∞
K EXP =
∑
k=0
α kA k
= exp(α A),
k!
A k,
aijk
where the elements of
represent
the number of paths from node i to
node j with k transitions; this kernel
gives more weight to two nodes with
shorter paths. Note that the exponential function gives the kernel positive
values, indicating that the kernel is
clearly positive and semidefinite.
By changing the parameter of the
exponential diffusion kernel, we can
obtain the von Neumann diffusion
kernel.18,19 The von Neumann diffusion kernel has an exponential parameter a k, instead of a k /k!:
∞
K VN =
∑α
−1
A = (I − α A) .
If a has the range of value 0 < α <
-1
, the kernel is well-defined, posi2
tive, and definite.15
For the present study, we can use
another kernel—random walk with
restart kernel—introduced by Jia-Yu
Pan and his colleagues.20 The random
walk with restart kernel has an advantage: The kernel can capture the
graph’s global structure and the
multifaceted relationship between two
nodes. The random walk with restart kernel uses the random walk
from node i to adjacent node j
with probability pij = P(S(t + 1) =
j|S(t) = i) = aij/ai., where aij is an
46
n
ai⋅ =
∑ aij , and S(t) is the state of the
j =1
random walker at time t. In addition, we
assume that the random walker comes
back to node i with the probability 1 - g :
X(0) = ei, X(t + 1) = g PTX(t) = (1 - g)ei,
where ei is a starting vector where the
ith element is 1 and 0 for others. The
probability of finding the random
walker on each node j when it starts
at node i by steady state solution X(t
+ 1) = X(t) = X is given as follows:18,21
X = (1 - g )(I - g PT) -1ei.
From this probability, we can consider (I - g P) -1 to be a graph kernel
that indicates similarities between
nodes. To show that the ith row contains the similarity to node i in a similar way to other kernel functions, P T
was transposed to P.
Once these similarities are computed using a graph kernel, we can
exploit them to find clusters using the
kernel k-means clustering method.
function to transform the original input xi into a high-dimensional feature
space f(xi), and nc is the number of
nodes belonging to Cc. We can rewrite D as follows:
(
clustering 22
Kernel k-means
uses an
iterative algorithm that minimizes the
sum of within-cluster inertia in the
feature space as
D
({
}
k
Cc c =1
)=∑ ∑
k
2
φ ( x i ) − mc ,
c =1 x i ∈Cc
where mc is the center of cluster c in the
feature space and can be calculated as
∑ φ (xi )
mc =
x i ∈Cc
,
nc
where xi is the node vector corresponding to node i, f(.) is a mapping
www.computer.org/intelligent
k
( )
∑
T
φ (xi ) φ x j
2
x j ∈Cc
T
φ ( x i ) φ ( x i ) −
k
n
c
.
T
φ x j φ ( xl )
c =1 x i ∈Cc
C
∈
x
,
x
j
l
c
+
nc2
∑∑
( )
∑
Note that the inner product f(xi)Tf(xj)
can be replaced by the kernel function
Kij, which is called the kernel trick. The
value of kernel function K can be calculated using various kernel functions,
instead of mapping our data via f(.)
and then computing the inner product.
Therefore, the separate nonlinear mapping f(.) need never be explicitly computed. The kernel function performs a
role to map data from the input space to
a high-dimensional feature space. Then,
measure D can be rewritten as follows:
(
D {Cc }c =1
=
Kernel k-Means Clustering
)
D {Cc }c =1 =
k
)
∑ x ∈C Kij ∑ x ,x ∈C K jl .
2
K ii −
c =1 x i ∈Cc
k
k k
k=0
A
element of the adjacency matrix A,
∑∑
j
c
+
nc
j
l
c
nc2
The distance in feature space between
node f(xi) and the center of each cluster
is compared. Each node is then assigned
to its nearest cluster as follows:
bi = arg minc φ (x i ) − mc
2
∑ x ∈C Kij ∑ x ,x ∈C K jl
2
= arg minc K ii −
j
nc
c
+
j
l
nc2
c
,
where bi means the assigned cluster
label for node i. Then, the iteration for
kernel k-means clustering is repeated
until no more labels of each node are
changed. We summarize the procedures for kernel k-means clustering:
IEEE INTELLIGENT SYSTEMS
Input
K: kernel matrix
k: number of clusters
Output
C1, C2, …, Ck : clusters of
nodes
Procedures
1. Initialize each cluster:
(0)
(0)
C(0)
1 ,C2 ,…,C k .
2. Assign each node to its
nearest cluster:
2
bi = arg minc φ ( xi ) − mc
= arg minc
∑ x ∈C
2
× Kii −
(t)
c
j
nc
Kij
+
∑ x ,x ∈C
j
l
(t)
c
nc2
K jl
.
3.
Update the clusters:
C(1t +1),C(2t +1),…,C(kt +1).
4. Repeat Steps 2 and 3 until
no more nodes are assigned
to a cluster different from
the current one.
Patent groups are discovered as the result of kernel k-means clustering, where
each patent’s group is determined by
comparing distances in the feature space
from the patent to each group’s center. The patent is assigned to the group
whose center is closest to that patent.
Influential Measures for the
Detection of a Core Patent
Once patent clusters are detected using
kernel k-means clustering as described
previously, the core patent in each cluster is explored again, because the obtained center in each cluster isn’t always
consistent with one of the given nodes.
Usually, existing community detection
methods in a citation network don’t deal
with the problem of searching influential
nodes; however, in the present study, the
problem of detecting influential nodes,
along with the node-clustering problem,
is discussed. Using this parallel approach
gives us the advantage of being able to
recognize the distribution of similar
JULY/AUGUST 2014
nodes around influential nodes. The distances between the nodes and the center
of each cluster obtained from the kernel
k-means clustering are calculated for the
influential nodes. The closest node from
the center of each cluster is considered
to be an influential node of each cluster. Note that there’s one difference between the measure to build the clusters
in the previous “Kernel k-Means Clustering” subsection and the measure to
search the core patent of each cluster in
this section. The cluster building measure
aims to find a cluster to which node i belongs based on the sum of within-cluster
inertia, while the core patent searching
measure seeks to find a real node that’s
closest to the virtual core patent node in
each cluster. Our proposed measure for
an influential node—kernel-mean-based
centrality (KMBC)—is determined according to the selected kernel. If the
exponential diffusion kernel (EXP) is selected, an influential node wd for a specific cluster Cd ∈ {Cc }kc =1 is identified by
the following measure:
( wd )EXP
= arg min i φ (x i ) − md
2
∑
2
( K EXP )ij
x j ∈Cd
( K EXP )ii −
nd
= arg min i
( K EXP ) jl
x j , x l ∈Cd
+
2
n
d
∑
∑
2
(exp(α A))ij
x j ∈Cd
( exp(α A))ii −
nd
.
= arg min i
exp(α A)) jl
(
x j , x l ∈Cd
+
nd 2
∑
Using the von Neumann (VN) diffusion kernel, the measure is given as
follows:
The random walk with restart
(RWR) kernel defines the measure for
an influential node as follows:
( wd )RWR
(
∑
(I − γ P)−1
2
x j ∈Cd
−1
−
(I − γ P)
ii
nd
= arg min i
−1
(I − γ P)
x j , x l ∈Cd
jl
+
nd 2
(
)
∑
(
)
)
ij
.
From these measures, we find the
closest node from the center of each
cluster and consider the node as a
core patent.
Experimental Results
The proposed methods based on kernel k-means clustering, in combination with the previous three graph
kernels, are optimized in terms of
modularity, which quantifies the quality of a division of a network into
communities. Also, the core patents
obtained by the proposed KMBC are
compared to those of commonly used
centralities. Note that the kernel matrix based on the adjacency matrix A
from the directed network, such as a
citation network, is not symmetric,
and hence, we define a symmetric,
undirected version of an adjacency
matrix for the directed network as
A + AT ,
2
which leads to producing the symmetric kernel matrix.
Additionally, the Gaussian radialbasis-function (RBF) kernel, which is
not a graph kernel, is used for comparison. The Gaussian RBF kernel is
presented as follows:
(
)
K RBF = exp − u1 − u2 2τ 2 ,
( wd )VN
(
∑
(I − α A)−1
2
x j ∈Cd
−1
−
(I − α A)
ii
nd
= arg min i
−1
(I − α A)
x j , x l ∈Cd
jl
+
nd 2
(
)
∑
(
)
www.computer.org/intelligent
)
ij
.
where t is the width parameter that
controls the amplitude of the RBF, and
u1 and u2 are the columns of A. The
dataset used for the computational experiments includes US patents in the
47
CORE PATENT GROUPING
Table 1. Modularity value and optimal number of clusters and kernel parameters.
Exponential
von Neumann
Random walk
with restart
Radial basisfunction (RBF)
Kernel parameter
10 -1
10 -2
10 -1
101
Optimal number of
clusters
3
27
16
25
0.258
0.236
0.222
0.174
Kernel
Modularity
Table 2. Core patents obtained from kernel-mean-based centrality with exponential
diffusion kernel (KMBC_EXP) and centrality measures.
KMBC_EXP
Degree
Closeness
Weighted
reachability
US5862260
US5892900
US6314409
US5892900
US5982891
US5943422
US5920861
US5910987
US5915019
US5917912
US6185683
US5949876
US6112181
US5862260
US6226618
US6122403
US5745604
US6157721
US5745604
US5349655
US5892900
US5982891
US5949876
US5910987
US5915019
US5862260
US5917912
US5943422
US5920861
US5832119
US6112181
US6185683
US6157721
US5349655
US5745604
US5892900
US5862260
US5943422
US5920861
US5982891
US5949876
US5915019
US5910987
US5832119
US5822436
US5917912
US6112181
US6185683
area of information and security issued from 1976–2007. We used only
the top 1 percent of frequently cited
US patents from 2003–2007 for our
study. The dataset has 40,982 citations
from 19,677 patents. For computational efficiency, we selected all patents that cited the most-cited patent,
US5349655, directly and indirectly
among the extracted top 1 percent of
US patents, and then only used the
patents with two or more citations.
The patents and citations from the
corresponding patents constitute the
nodes and edges in a citation network,
respectively.
Performance Measure
Modularity is used to evaluate the
quality of a partitioning of a network
into communities. This measure is
frequently used in network analysis
research.23,24 Modularity has been successfully used to catch the reasonable
structure of communities.25,26 A good
partition—which has a high value of
modularity—is one in which dense
48
internal connections exist between
nodes within communities, but only
sparse connections exist between different communities.25 Modularity for
a directed network, such as a citation
network, is defined as follows:23,24
Q=
1
m
∑ Aij −
i, j
kiout kjin
δ c , c ,
m i j
(
)
where ci is the community to which
in
node i belongs; and kiout and kj are
the out-degree of node i and the indegree of node j, respectively. Also,
m is the total number of edges in
the network, and d(ci, cj) is the Kronecker delta symbol, which is defined
as 1 if ci is equal to cj and 0 otherwise. In the modularity measure,
kiout kjin / m is the probability of an
edge between two nodes i and j under randomization. Therefore, the
high modularity value tells us when
there are more edges within communities than we would expect on the
basis of chance.24
www.computer.org/intelligent
Results
The proposed method, combined with
the four kernel functions, is optimized
in terms of modularity. In the experiment, kernel parameters (a for von
Neumann and exponential diffusion,
g for random walk with restart, and t
for Gaussian RBF) are varied in such
a way that a, g, t = 10-3, 10-2, 10-1, 1,
101. In addition, the number of clusters is varied from two to 50.
For each kernel function, we selected
the optimal number of clusters and
kernel parameters by maximizing the
modularity value. Table 1 shows the optimized number of clusters and kernel
parameters for each kernel function and
their modularity values. Depending on
the kernel functions, the optimal numbers of clusters are different. The proposed KMBC_EXP produces the largest
modularity value and yields the best
community structure over other methods, although it has the smallest number of clusters. The RBF kernel, which
isn’t a graph kernel, didn’t achieve good
clustering performance in comparison
to graph kernels. This result indicates
that the graph kernel is effective for
clustering in a citation network.
Table 2 shows the core patents detected from KMBC using the exponential kernel function. It’s not easy to
measure the accuracies of the proposed
approaches for the core patents. Therefore, we compared the selected influential nodes using the proposed method
with 15 core patents by popular centrality measures. Using this comparison,
we can verify that the influential nodes
selected by the proposed method are
meaningful. The obtained patents are
compared to those from commonly used
centrality measures, such as the degree,
adjusted closeness, and weighted-reachability measures.27 The degree centrality
measure is the number of nodes connected to the target node, and the adjusted closeness measure emphasizes
the length of the shortest path from the
IEEE INTELLIGENT SYSTEMS
TabIe 3. The communities of patents obtained from KMBC_EXP.
Community
Name
Topic
k=3
(the optimal number of clusters)
k=4
k=5
Core patents
by the
proposed
method
Patents with
the highest
degree
centrality
Core patents
by the
proposed
method
Patents with
the highest
degree
centrality
Core patents
by the
proposed
method
Patents with
the highest
degree
centrality
Cluster 1
Information hiding:
digital watermarking or
steganography
US5862260
US5862260
US5613002
US5613002
US5349655
US5349655
Cluster 2
System for secure
transaction management
or rights protection
US5892900
US5892900
US5892900
US5745604
US5613002
US5613002
Cluster 3
US6314409
US6314409
US6122403
US5862260
US5822436
US5822436
Cluster 4
System for access control,
data management, or
communication network
_
_
_
_
_
US6226618
_
US5745604
_
US6226618
_
US5892900
Cluster 5
US6226618
US6226618
target node to all other nodes. Additionally, weighted reachability is the centrality measure that reflects the concept that
the directly cited patents are more influential than indirectly cited patents.27
Note that the core patents from commonly used centrality measures in a citation network are sorted according to
their importance, while the patents from
KMBC are sorted in the order of publication. This is because kernel k-means
clustering can detect only the core patents of each cluster but can’t quantify
their importance. Although ordering the
patents according to their importance
could be important in a global network,
it might not be helpful if someone intends to find a particular type of patent
community and its representative patents. In that case, the neighboring patent information is more useful than the
order of importance. The key contribution in this work is that even though the
order of importance of patents can’t be
determined, the neighbors of the specific patents—which deal with a similar
topic—can be found, as well as the representative patents among neighbors.
In Table 2, the bolded patents simultaneously belong to the lists of both
KMBC_EXP and commonly used
centralities. From Table 2, we can see
that two core patents detected from
KMBC_EXP are included in the top
15 patents of the degree for adjusted
JULY/AUGUST 2014
closeness and weighted reachability. Notably, the most cited patent,
US5349655, is directly or indirectly
not included in the list of KMBC_EXP.
The three listed patents in Table 2 are
influential in each cluster, but they
might not be influential in a global
network and vice versa—that is, the
influential patents in a global network
might not be influential in each cluster with similar topics. US6314409 is
an example of an influential patent
within a local network. US6314409 is
not in the top 15 patents of the degree
centrality in a global network, but it
has the largest number of in-degree citations in its corresponding cluster.
In addition, we investigated patents
in each cluster and observed that patents of each cluster deal with a similar
topic. Table 3 shows the main topic
of each cluster (see the results for k =
3 in Table 3). Clusters 1 and 2 have
patents about information hiding and
systems for secure transaction management, respectively. Cluster 3 simultaneously deals with systems for
access control and data management.
Figure 1 shows the citation network
in a circular graph, grouped using the
kernel k-means clustering algorithm
with an exponential kernel, which is
obtained using the NetMiner program
(see www.netminer.com). The three colors represent three clusters of patents,
www.computer.org/intelligent
subdivided into fields in the outer ring.
The patents with the same color and
shape belong to the same group, and
the patents with the larger shapes represent the core patents of each group. The
line represents the citation (that is, link)
between patents. For the sake of brevity,
only patents with 15 or more incoming
edges are plotted. This citation network
shows the edge concentration in each
cluster. From Figure 1, we can see that
the clustering process is well conducted
by observing that all clusters, except
cluster 3, have dense node-to-node connections within the clusters and sparse
connections with other clusters.
We can see that two core patents,
US5862260 and US5892900, have
more internal edges than other patents
within a group, while there’s little edge
intensity in the patent US6314409, selected from cluster 3. This is due to the
fact that many patents that cite patent
US6314409 had already been removed
when we chose the patents for the plot;
however, US6314409 is admittedly the
core patent, with the largest number of
in-degree citations in cluster 3. Table 3
shows that the core patent of each cluster, obtained from our approach with the
optimal number of clusters (k = 3), corresponds with the node with the highest
number of internal edges in each cluster. From the results, we can verify that
the proposed method has appropriately
49
CORE PATENT GROUPING
Cluster 1
recognized. The experimental results
revealed that this approach can detect
influential and meaningful patents and
patent groups efficiently. The methods
developed in this study enable a person to detect core patents and patent
groups in a citation network. Valuable
areas of future research could include
validating the findings using additional
data sets, and developing a method to
detect the change of core patents and
patent groups over a period of time.
Cluster 2
Clusterr 3
References
Figure 1. A citation network from kernel k-means clustering using exponential
kernel. The three colors represent three clusters of patents, subdivided into fields
in the outer ring. The patents with the same color and shape belong to the same
group, and the patents with the larger shapes represent the core patents of each
group. The line represents the citation (that is, link) between patents.
detected the core patents. The core patents from our approach and degree centrality when we use the different number
of clusters or different graph kernels,
can be different (see the results for k =
4 and 5 in Table 3). Note that the core
patents from the degree centrality were
obtained after we first found the technology groups using our proposed graph
kernel approach. The core patents obtained from the simple degree centrality
in the original space in Table 2 are quite
different from the results in Table 3.
The proposed algorithm could be
useful when we need to learn the leading patent within each patent group.
Using the database of US patents, we’ve
investigated the ratio of patents belonging to each cluster among the top
15 patents of each centrality in Table
2. In all centralities, the core patents of
cluster 2 have a portion more than 70
percent, while there are no core patents
within cluster 3 among the 15 patents.
In this way, the core patents obtained
50
from the global networks could be
concentrated in a specific cluster. Our
approach to simultaneously finding the
patent communities and their leading
patents can prevent this problem and
be helpful when exploring the core
patent in each specific area.
T
he objective of this study was to
develop algorithms to simultaneously detect core patents and patent groups in a complex network. To
this end, the algorithm for the simultaneous detection of core patents and
patent groups based on kernel k-means
clustering with a graph kernel was introduced. This method uses a graph
kernel because it can compute implicit similarities between patents in a
high-dimensional feature space. The
proposed method helps core patents
in each group to be discovered easily and the distribution of the similar
patents around the core patents to be
www.computer.org/intelligent
1. M.E.J. Newman, “Detecting Community
Structure in Networks,” The European
Physical J. B, vol. 38, 2004, pp. 321-330.
2. Y. Yang et al., “Personalized Email Prioritization Based on Content and Social
Network Analysis,” IEEE Intelligent
Systems, vol. 25, no. 4, 2010, pp. 12-18.
3. Z. Yang et al., “Social Community
Analysis via a Factor Graph Model,”
IEEE Intelligent Systems, vol. 26, no. 3,
2011, pp. 58-65.
4. L. Yen et al., “Graph Nodes Clustering
with the Sigmoid Commute-Time Kernel: A Comparative Study,” Data and
Knowledge Eng., vol. 68, no. 3, 2009,
pp. 338-361.
5. P. Chen and S. Redner, “Community
Structure of the Physical Review Citation Network,” J. Informetrics, vol. 4,
2010, pp. 278-290.
6. D. Kim et al., “Automated Detection
Of Influential Patents Using Singular Values,” IEEE Trans. Automation Science and
Eng., vol. 9, no. 4, 2012, pp. 723-733.
7. R. Rousseau, “The Gozinto Theorem: Using Citations to Determine
Influences on a Scientific Publication,”
Scientometrics, vol. 11, nos- 3-4, 1987,
pp. 217-229.
8. C.A. Yeung et al., “Measuring Expertise
in Online Communities,” IEEE Intelligent
Systems, vol. 26, no. 1, 2011, pp. 26-32.
9. M.M. Kessler, “Bibliographic Coupling
between Scientific Papers,” American
Documentation, vol. 14, no. 1, 1963,
pp. 10-25.
IEEE INTELLIGENT SYSTEMS
THE AUTHORS
Dohyun Kim is a senior researcher at the Korea Institute of Science and Technology In-
10. H. Small, “Co-Citation in the Scientific Literature: A New Measure of the
Relationship between Two Documents,”
J. Am. Soc. for Information Science, vol.
24, no. 4, 1973, pp. 265-269.
11. L. Egghe and R. Rousseau, “CoCitation, Bibliographic Coupling and
a Characterization of Lattice Citation
Networks,” Scientometrics, vol. 55,
no. 3, 2002, pp. 349-361.
12. K.W. Boyack and R. Klavans, “Co-Citation Analysis, Bibliographic Coupling,
and Direct Citation: Which Citation
Approach Represents the Research
Front Most Accurately?” J. Am. Soc. for
Information Science and Technology,
vol. 61, no. 12, 2010, pp. 2389-2404.
13. R. Klavans and K.W. Boyack, “Quantitative Evaluation of Large Maps of
Science,” Scientometrics, vol. 68, no. 3,
2006, pp. 475-499.
14. N. Doulamis et al., “Collaborative
Evaluation Using Multiple Clusters in
a Declarative Design Environment,”
Artificial Intelligence Techniques for
Computer Graphics, vol. 159, 2008,
pp. 141-157.
15. J. Kandola, N. Cristianini, and J.
Shawe-Taylor, “Learning Semantic
Similarity,” Proc. Advances in Neural
Information Processing Systems, 2002,
pp. 657-664.
16. C.M. Bishop, Pattern Recognition and
Machine Learning, Springer, 2006.
17. C. Gao et al., “Graph Ranking for
Exploratory Gene Data Analysis,” BMC
Bioinformatics, vol. 10, supplement 11,
2009; www.biomedcentral.com/14712105/10/S11/S19.
18. F. Fouss et al., “An Experimental Investigation of Kernels on Graphs for Collaborative Recommendation and Semisupervised Classification,” Neural Networks,
vol. 31, July 2012, pp. 53-72.
19. R.I. Kondor and J. Lafferty, “Diffusion
Kernels on Graphs and Other Discrete
Structures,” Proc. 19th Int’l Conf. Machine Learning, 2002, pp. 315-322.
20. J.-Y. Pan et al., “Automatic Multimedia
Cross-Modal Correlation Discovery,”
JULY/AUGUST 2014
formation. His research interests include statistical data mining, network data analysis,
and bibliometric analysis. Kim has a PhD in industrial engineering from the Korea Advanced Institute of Science and Technology. Contact him at ftgog@kisti.re.kr.
Bangrae Lee is a senior researcher in the Information Analysis Center of the Korea Institute of Science and Technology Information. His research interests include data mining, informetrics, scientometrics, and social network analysis. Lee has an MS in robotics
from the Korea Advanced Institute of Science and Technology. Contact him at brlee@
kisti.re.kr.
Hyuck Jai Lee is a principal researcher at the Korea Institute of Science and Technol-
ogy Information. His research interests include information visualization, social network
analysis, and research evaluation based on bibliometric analysis. Lee has a PhD in chemistry from Sogang University, Seoul, Korea. Contact him at hlee@kisti.re.kr.
Sang Pil Lee is a research fellow at the Information Analysis Center at the Korea Institute
of Science and Technology Information. His research interests include emerging technology analysis, knowledge science, bibliometric analysis, scientometrics, and data mining.
Lee has a PhD in applied biotechnology from Osaka University, Japan. Contact him at
splee@kisti.re.kr.
Yeongho Moon is a director at the Information Analysis Center at the Korea Institute of
Science and Technology Information. He is also a vice president of Korea Technology Innovation Associate. His research interests include emerging technology analysis, knowledge geometrics analysis, and data mining analysis and information service. Moon has a
PhD in construction and environment engineering from the Korea Advanced Institute of
Science and Technology. Contact him at yhmoon@kisti.re.kr.
Myong K. Jeong is an associate professor in the department of industrial and systems engineering and the Rutgers Center for Operations Research at Rutgers University. His research interests include statistical data mining, recommendation systems, machine health
monitoring, and sensor data analysis. Jeong has a PhD in industrial and systems engineering from the Georgia Institute of Technology. He received the Freund International
Scholarship and National Science Foundation Career Award and is an associate editor of
the IEEE Transactions on Automation Science and Engineering and International Journal of Quality, Statistics, and Reliability. He is a senior member of IEEE. Contact him at
mjeong@rci.rutgers.edu.
Proc. 10th ACM SIGKDD Int’l Conf.
Knowledge Discovery and Data Mining, 2004; http://dl.acm.org/citation.
cfm?id=1014135.
21. F. Fouss et al., “Random-Walk Computation of Similarities between Nodes of a
Graph with Application to Collaborative
Recommendation,” IEEE Trans. Knowledge and Data Eng., vol. 19, no. 3, 2007,
pp. 355-369.
22. B. Schölkopf, A. Smola, and K.-R.
Müller, “Nonlinear Component Analysis as a Kernel Eigenvalue Problem,”
Neural Computation, vol. 10, no. 5,
1998, pp. 1299-1319.
23. E.A. Leicht and M.E.J. Newman,
“Community Structure in Directed Networks,” Physical Rev. Letters, vol. 100,
no. 11, 2008; http://dx.doi.org/10.1103/
PhysRevLett.100.118703.
www.computer.org/intelligent
24. V. Nicosia et al., “Extending the Definition of Modularity to Directed Graphs
with Overlapping Communities,” J.
Statistical Mechanics: Theory and
Experiment, 2009, P03024.
25. M.E.J. Newman and M. Girvan, “Finding the Evaluating Community Structure
in Networks,” Physical Review E, vol.
69, 2004, 026113; doi:10.1103/PhysRevE.69.026113.
26. M.E.J. Newman, “Finding Community
Structure in Networks Using the Eigenvectors of Matrices,” Physical Review
E, vol. 74, 2006, 036104; http://dx.doi.
org/10.1103/PhysRevE.74.036104 .
27. O. Kwon et al., “A Method to Make the
Genealogical Graph of Core Documents
from the Directed Citation Network,”
Information, vol. 12, no. 4, 2009,
pp. 875–888.
51