IMAGE SEGMENTATION

9.13 IMAGE SEGMENTATION

Image segmentation refers to the decomposition of a scene into its components. It is

a key step in image analysis. For example, a document reader would first segment the various characters before proceeding to identify them. Figure 9.49 lists several image segmentation techniques which will be discussed now.

Amplitude Thresholding or Window Slicing Amplitude thresholding is useful whenever the amplitude features (see Section 9.2)

sufficiently characterize the object. The appropriate amplitude feature values are calibrated so that a given amplitude interval represents a unique object character­ istic. For example, the large amplitudes in the remotely sensed IR image of Fig.

7. lOb represent low temperatures or high altitudes. Thresholding the high-intensity values segments the cloud patterns (Fig. 7. lOd). Thresholding techniques are also useful in segmentation of binary images such as printed documents, line drawings

and graphics, multispectral and color images, X-ray images, and so on. Threshold

Image segmentation techniques

Amplitude Component

Template Texture thresholding

matching segmentation

and clustering

Figure 9.49

Sec. 9. 1 3 Image Segmentation 407 Sec. 9. 1 3 Image Segmentation 407

1. The histogram of the image is examined for locating peaks and valleys. If it is multimodal, then the valleys can be used for selecting thresholds.

2. Select the threshold (t) so that a predetermined fraction ( TJ) of the total number of samples is below t.

3. Adaptively threshold by examining local neighborhood histograms.

4. Selectively threshold by examining histograms only of those points that satisfy

a chosen criterion. For example, in low-contrast images, the histogram of those pixels whose Laplacian magnitude is above a prescribed value will exhibit clearer bimodal features than that of the original image.

5. If a probabilistic model of the different segmentation classes is known, determine the threshold to minimize the probability of error or some other quantity, for instance, Bayes' risk (see Section 9. 14).

Example 9.8 We want to segment the Washington Monument from the scene in Fig. 9.50. First, the

low intensities are thresholded to isolate the very dark areas (trees here). Then we detect a rectangle bounding the monument by thresholding the horizontal and vertical

projection signatures defined as h (n) � Lm u ( m, n)ILm 1, v ( m ) � Lm u ( m, n)ILn 1.

Contour-following the boundary of the object inside the rectangle gives the segmented object.

Segmented image

Input 255

Tree removed

Horizontal

image

and vertical

Contour image

n ---+-

Figure 9.50 Image segmentation using horizontal and vertical projections.

408 Image Analysis and Computer Vision Chap. 9

Figure

9.51 Neighborhood of pixel X in a pixel labeling algorithm.

Component Labeling

A simple and effective method of segmentation of binary images is by examining the connectivity of pixels with their neighbors and labeling the connected sets. Two practical algorithms are as follows.

Pixel labeling. Suppose a binary image is raster scanned left to right and top to bottom. The current pixel, X (Fig. 9.51), is labeled as belonging to either an object (ls) or a hole (Os) by examining its connectivity to the neighbors

A, B, C, and

D. For example, if X = 1 , then it is assigned to the object(s) to which it is connected. If there are two or more qualified objects, then those objects are declared to be equivalent and are merged. A new object label is assigned when a transition from Os to an isolated 1 is detected. Once the pixel is labeled, the features of that object are

updated. At the end of the scan, features such as centroid, area, and perimeter are saved for each region of connected ls.

Run-length connectivity analysis. An alternate method of segmenting binary images is to analyze the connectivity of run lengths from successive scan lines. To illustrate this idea, we consider Fig. 9.52, where the black or white runs are

denoted by a, b, c, . . . . A segmentation table is created, where the run a of the first scan line is entered into the first column. The object of the first run a is named A. The first run of the next scan line, b, is of the same color as a and overlaps a. Hence

Column

A a Level

e f g }· h

(a) I nput: binary image (b) Output: segmentation table

Figure

9.52 Run-length connectivity algorithm.

Sec. 9. 1 3 Image Segmentation 409 Sec. 9. 1 3 Image Segmentation 409

A is created, where dis placed. A divergence flag IDl is set in this column to indicate that object B has caused this divergence.

Also, the flag ID2 of B (column 2) is set to

A to indicate B has caused divergence in

A. Similarly, convergence occurs when two or more runs of Os or ls in a given line overlap with a run of same color in the previous line. Thus convergence occurs in

run u, which sets the convergence flags !Cl to C in column 4 and IC2 to B in column

A in column 2, and the column 5 is labeled as belonging to object A.

6. Similarly, w sets the convergence flag IC2 to

In this manner, all the objects with different closed boundaries are segmented in a single pass. The segmentation table gives the data relevant to each object. The convergence and divergence flags also give the hierarchy structure of the object. Since B causes divergence as well as convergence in

A and C has a similar relation­ ship with B, the objects

A, B, and C are assigned levels 1, 2, and 3, respectively.

Example 9.9 A vision system based on run-length connectivity analysis is outlined in Fig. 9.53a. The

input object is imaged and dig itized to give a binary image. Figure 9.53b shows the run-length representation of a key and its segmentation into the outer profile and the

Camera A to D

Feature converter

Pre process

Run-length

and binarize

segmentation

extraction Classification

(a) Vision system

(c) Display of Best-Fit Ellipse, Bounding (b) Run Length Data for a Key

Rectangle and Center of Mass for the Key.

Figure 9_53 Vision system based on run-length connecitvity analysis.

410 Image Analysis and Computer Vision Chap. 9 410 Image Analysis and Computer Vision Chap. 9

Boundary-Based Approaches Boundary extraction techniques segment objects on the basis of their profiles.

Thus, contour following, connectivity, edge linking and graph searching, curve fitting, Hough transform, and other techniques of Section 9.5 are applicable to image segmentation. Difficulties with boundary-based methods occur when objects are touching or overlapping or if a break occurs in the boundary due to noise or artifacts in the image.

Example 9.10 Boundary analysis-based vision system Figure 9.54a shows an example of an object-recognition system, which uses the bound­

ary information for image segmentation. The edges detected from the image of the input object are linked to determine the boundary. A spline fit (Section 9.6) is per­

formed to extract the control points (Fig. 9.54b), which are then used to determine the

object location (center of mass), orientation, and other shape parameters [71].

Edge

Segmentation

Spline control

detection

by boundary

point feature

and analysis

(a) System block diagram

Object boundary points

Cubic spline fit

(b)

Figure 9.54 Object recognition system based on boundary analysis.

Sec. 9. 1 3 Image Segmentation 41 1

Region-Based Approaches and Clustering The main idea in region-based segmentation techniques is to identify various

regions in an image that have similar features. Clustering techniques encountered in pattern-recognition literature have similar objectives and can be applied for image segmentation. Examples of clustering are given in Section 9. 14.

One class of region-based techniques involves region growing [72). The image is divided into atomic regions of constant gray levels. Similar adjacent regions are merged sequentially until the adjacent regions become sufficiently different (Fig. 9.55). The trick lies in selecting the criterion for merging. Some merging heuristics are as follows:

1. Merge two regions f?l; and fll1 if w!Pm > 0i, where Pm = min(P;, �), P; and � are the perimeters of Yl; and 9l1, and w is the number of weak boundary locations

(pixels on either side have their magnitude difference less than some threshold

a ) . The parameter 01 controls the size of the region to be merged. For example 01 =

1 implies two regions will be merged only if one of the regions almost

surrounds the other. Typically, 01 = 0.5.

2. Merge 9l; and fll1 if w/I > 02, where

I is the length of the common boundary between the two regions. Typically

0.75. So the two regions are merged if the boundary is sufficiently weak. Often this step is applied after the first heuristic has been used to reduce the number of regions.

3. Merge 9l; and £7(1 only if there are no strong edge points between them. Note that the run-length connectivity method for binary images can be interpreted

as an example of this heuristic.

4. Merge fll; and fll1 if their similarity distance [see Section 9.14) is less than a threshold.

Instead of merging regions, we can approach the segmentation problem by splitting a given region. For example the image could be split by the quad-tree approach and then similar regions could be merged (Fig. 9.56).

Region-based approaches are generally less sensitive to noise than the boundary-based methods. However, their implementation complexity can often be quite large.

Weak boundary

Merge

Figure 9.55 Region growing by merging.

412 Image Analysis and Computer Vision Chap. 9

3 (al Input

� l (A, B, Dl

2(A, B, Cl A 3(8, C, Dl

(bl Ouad tree split

(cl Segmented regions

Figure 9.56 Region growing by split and merge techniques.

Template Matching One direct method of segmenting an image is to match it against templates from a

given list. The detected objects can then be segmented out and the remaining image can be analyzed by other techniques (Fig. 9.57). This method can be used to segment busy images, such as journal pages containing text and graphics. The text can be segmented by template-matching techniques and graphics can be analyzed by boundary following algorithms.

Texture Segmentation Texture segmentation becomes important when objects in a scene have a textured

background. Since texture often contains a high density of edges, boundary-based techniques may become ineffective unless texture is filtered out. Clustering and

(al Template

(bl Input image

(cl Filtered image

Figure 9.57 Background segmentation (or filtering) via template matching.

Sec. 9. 1 3 Image Segmentation 413 Sec. 9. 1 3 Image Segmentation 413

problem. Use of a priori knowledge about the existence and kinds of textures that may be present in a scene can be of great utility in practical problems.