Petru Blaga. 107KB Jun 04 2011 12:09:21 AM

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials

REDUCING OF VARIANCE BY A COMBINED SCHEME
BASED ON BERNSTEIN POLYNOMIALS
by
Petru P. Blaga
Abstract. A combined scheme of the control variates and weighted uniform sampling methods
for reducing of variance is investigated by using of the multivariate Bernstein operator on the
unit hypercube. The new obtained estimators for the random numerical integration are
numerically compared with the crude Monte Carlo, control variates, and weighted uniform
sampling estimators.

1. INTRODUCTION
Definite integrals can be estimated by probabilistic considerations, and these
are rather when multiple integrals are concerned. The integral is interpreted as the
mean value of a certain random variable, which is an unknown parameter. To estimate
this parameter, i.e. the definite integral, one regards the sample mean of the sampling
from a suitable random variable. This sample mean is an unbiased estimator for the
definite integral and is refered as the crude Monte Carlo estimator.
Generally, this method is not fast-converging ratio to the volume of sampling,

and efficiency depends on the variance of the estimator, which is expressed by the
variance of the integrand. Consequently, for improving the efficiency of Monte Carlo
method, it must reduce as much as possible the variance of the integrated function.
There is a lot of procedures for reducing of the variance in the Monte Carlo method. In
the following we approach the reducing of variance by a combined scheme of the
control variates and weighted uniform sampling methods, using the multivariate
Bernstein operators on the unit hypercube.
Numerical experiments are considered comparatively with the crude Monte
Carlo, control variates, and weighted uniform sampling estimates.

2. CRUDE MONTE CARLO METHOD
Let X be an n-dimensional random variable having the probability density
function ρ :IRn→IR. In the random numerical integration the multidimensional
integral
(1)
I [ρ ; f ] = n ρ (x ) f (x )dx

∫

IR

47

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials

( ) , in other words ∫

f (X ) , where the function

is interpreted as the mean value of the random variable

f:IR →IR usually belongs to Lρ IR

ρ (x ) f 2 (x )dx exists, and

therefore the mean value I [ρ ; f ] exists.
Using a basic statistical technique, the mean value given by (1) can be
estimated by taking N independent samples (random numbers) x i , i = 1, N , with the
probability density function ρ . These random numbers are regarded as values of the

2

n

n

IR n

independent identically distributed random variables X i , i = 1, N , i.e. sample
variables with the common probability function ρ .

We use the same notation I N [ρ ; f ] for the sample mean of random variables

f (X i ), i = 1, N , and respectively for its value, i.e.

I N [ f ] = I N [ρ ; f ] =
I N [ f ] = I N [ρ ; f ] =

1
N

1
N

∑ f (X ) ,
N

i =1

i

∑ f (x ) .
N

The estimator I N [ρ ; f ] satisfies the following properties:

(

)

i =1

i

E I N [ρ ; f ] = I [ρ ; f ] , (unbiased estimator of I [ρ ; f ] ),

(

)

Var I N [ρ ; f ] → 0, N → ∞ ,

I N [ρ ; f ] → I [ρ ; f ] , N → ∞ , (with probability 1).
Taking into account these results, the crude Monte Carlo integration formula is
defined by

I [ρ ; f ] = ∫

IR n

ρ (x ) f (x )dx ≅

1
N

∑ f (x ) .
N

i =1

i

(2)

It must remark that in (1) the domain of integration is only apparently the
whole n-dimensional Euclidean space. Thus, it is possible that the density
ρ (x ) = 0, x ∉ D , where D is a region of the n-dimensional Euclidean space IRn,
therefore the integral (1) becomes

48

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials
I [ρ ; f ] = ∫ ρ (x ) f (x )dx ,
D

and the crude Monte Carlo method must be interpreted in an appropriate manner. For
n
example, if the integration region is the unit hypercube D n = [0,1] and

ρ (x ) = 1, x ∈ D n , then the crude Monte Carlo estimator is
1
I [f ]=
N
c
N

∑ f (X ) ,
N

i =1

i

(3)

where the sampling variables X i , i = 1, N , are independent uniformly distributed on

Dn .
3. CONTROL VARIATES METHOD

Control variates is a technique for reducing of variation, and it consists in the split of
the integral (1) into two parts,

I [ρ ; f ] = ∫

IR n

ρ (x )h(x )dx + ∫

IR n

ρ (x )[ f (x ) − h(x )]dx ,

(4)

which are integrated separately, the first by mathematical theory and the second by
Monte Carlo method. The function h must be simply enough to be integrate
theoretically, and mimics f to absorb most of its variation.
This method was applied in [7] considering the function h given by the
multivariate Bernstein polynomial, on the hypercube D n , having the degree m k in
the variable x k :
m
k
Bm ( f , x ) = ∑ p m,k (x ) f  
m
k =0

k
k
= ∑ L ∑ p m1 ,k1 ( x1 )L p mn ,kn ( x n ) f  1 , L , n

mn
k1 = 0
kn =0
 m1
m1

mn

49


 .


(5)

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials
We

have

used

x = ( x1 ,..., x n ) ,

the

following

notations

0 = (0,...,0 ) ∈ IR n ,

m = (m1 ,..., mn ) ,

k
k  k1
=  , L , n
m  m1
mn


 ,


k = (k1 ,..., k n ) ,

and

respectively

 m
m −k
p m,k ( x ) =   x k (1 − x ) , x ∈ [0,1] , m ∈ IN , k = 0, m , for the Bernstein basis.
k
In this manner the estimate of the integral I [ f ] =

I N [f ]=
v

∑

∏i=1 (mi + 1) k =0
1

n

m

k 1
f +
m N

∫

Dn

f (x )dx is

∑ [ f (x ) − B ( f ; x )]
N

i =1

i

m

i

(6)

where x i , i = 1, N , are independent uniformly distributed points in the hypercube

Dn .

4. WEIGHTED UNIFORM SAMPLING METHOD

This method was given in [4], reconsidered in [6], and recently in [2] it was compared
with other methods for reducing of the variance.
Let us consider the integral

I [ f ] = ∫ f (x )dx = V ∫
D

D

1
f (x )dx ,
V

where the bounded D ⊂ IR n region has the volume V.
The crude Monte Carlo estimator for I [ f ] is

I N [f ]=
c

V
N

∑ f (X ) ,
N

i =1

i

with the sampling variables X i , i = 1, N , independent uniformly in the region D , i.e.
these have the common density probability function

50

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials
1
 , if x ∈ D,

ρ (x ) = V

 0, if x ∉ D.

The method of weighted uniform sampling consists in the considering of function
g : D → IR such that

∫ g (x )dx = 1 ,
D

and the corresponding sampling function

w
w

  N
 N
I N [ f ] = I N [g ; f ] =  ∑ f (X i )  ∑ g (X i )

  i =1
 i =1

where X i , i = 1, N , are the same above sampling variables.
In [1] was considered the function g, from the weighted uniform sampling
method, given by the multivariate Bernstein polynomial corresponding to the
integrand f defined by (5), i.e.
g (x ) = K ⋅ Bm ( f ; x ) ,
where the constant K is such that

∫

Dn

g (x )dx = 1 .

From this condition we have that

∏ (m + 1) p (x) f  k 
g (x ) =
∑
k
m
f


∑ m
n

i =1

m

i

k =0

m

k =0

=

∏ (m

m,k

n

∑

i =1

m1

k1 = 0

L ∑k

mn

n =0

+ 1)

k
k
f  1 , L, n
mn
 m1
i





×

k
k
× ∑ L ∑ p m1 ,k1 ( x1 )L p mn ,k n ( x n ) f  1 ,L , n
mn
k1 = 0
kn =0
 m1
m1

mn

51


.


Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials

Finally, the random numerical integration formula is given by

I

w
N

[f ]= ∑
∑

N

i =1
N

i =1

∑

f (x i )

g (x i )

k
f 
m ⋅
=
n
∏i=1 (mi + 1)

∑i =1 f (x i )

m

k =0

(7)

N

k
∑i =1 ∑k =0 p m,k (x i ) f  m 
N

.

m

The random points x i , i = 1, N , are independent uniformly distributed in the
hypercube D n .

5. COMBINED SCHEME

The reducing technique of variance by proposed combined scheme consists in the split
of integral from the control variates method (4), with the integration domain D n , and

h = Bm ( f ) , i.e.

I[ f ] =

k
∑ f  m  + ∫ e(x )dx ,

∏i=1 (mi + 1) k =0
1

n

m

Dn

(8)

where e(x ) = f (x ) − Bm ( f ; x ) , then to apply the weighted uniform sampling method
for the integral from the right side of (8). In a such way, the function g, from
weighted uniform sampling method, is given by multivariate Bernstein polynomial,
namely
g (x ) = K ⋅ Bm (e; x ) ,
where the constant K is such that

∫

Dn

g (x )dx = 1 .

From this condition we have that

52

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials

∏ (m + 1) p (x)e k 
g (x ) =
∑
k
m
e
∑  m 
n

i =1

m

i

k =0

m

k =0

=

m,k

∏ (m
n

i =1

+ 1)

k
k
∑k1 =0 L ∑kn =0 e m1 ,L, mn
n
 1
m1

i

mn





×

mn
m1
k
k
× ∑ L ∑ p m1 ,k1 ( x1 )L p mn ,k n ( x n )e 1 ,L , n
mn
k1 = 0
kn =0
 m1


.


Finally, the random numerical integration formula is given by

I N [f ]=
vw

∏ (m
1

n

i =1

∑ f  m 
)

+1

m

k =0

k

[

]

 m  k 
N
(9)
∑k =0 e m  ∑i =1 e(x i )




.
+
n
 N
m
 k 
∏i=1 (mi + 1) ∑i =1 ∑k =0 p m,k (x i )e m 
i

[

]

The random points x i , i = 1, N , are independent uniformly distributed in the
hypercube D n .

6. NUMERICAL EXPERIMENTS

Numerical examples are considered in the unidimensional (n = 1) and bidimensional

(n = 2) cases for the estimator (9) with the integrand
f ( x, y ) =

f given by f ( x ) =

1
and
1+ x

1
respectively.
1+ x + y

The numerical results contained in the following two tables compare the
estimates obtained by the combined scheme (9), weighted uniform sampling technique
(7), control variates (6), and the crude Monte Carlo method (3).

53

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials
Each table contains: the sampling volume N, the degree m (or m1 , m 2 ) of the

Bernstein polynomials, the estimate given by the combined scheme I N [ f ] , the ratio
vw

values of the error estimates Err c , Err v , Err w and Err vw . We also remark that the
estimations from each row of tables represent the mean values in one hundred of
samplings.

I [ f ] = log 2 = 0.69314718...
N

m

I N [g ; f ]

50
100
300
500
50
100
300
500
50
100
300
500
50
100
300
500

2
2
2
2
3
3
3
3
5
5
5
5
7
7
7
7

0.6932623
0.6933181
0.6932220
0.6931902
0.6932174
0.6932408
0.6931841
0.6931683
0.6931797
0.6931856
0.6931608
0.6931549
0.6931656
0.6931676
0.6931541
0.6931511

vw

I [ f ] = log

N

m1

m2

50
100
300
50

2
2
2
2

2
2
2
3

Err c Err v

Err c Err w

Err c Err vw

11
7
5
6
15
9
7
9
22
14
11
15
29
19
16
21

25
10
7
9
27
13
9
13
36
19
14
20
46
25
20
28

18
12
13
20
29
23
27
41
62
55
73
113
110
104
144
223

Err c Err v

Err c Err w

Err c Err vw

5
6
5
6

8
9
7
8

18
17
37
21

27
= 0.5232481...
16

I N [f ]
vw

0.5233825
0.5233434
0.5232675
0.5233648

54

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials
100
300
50
100
300
50
100
300
50
100
300
50
100
300
50
100
300

2
2
2
2
2
3
3
3
3
3
3
4
4
4
5
5
5

3
3
5
5
5
3
3
3
5
5
5
5
5
5
5
5
5

0.5233352
0.5232573
0.5233474
0.5233276
0.5232523
0.5233172
0.5232994
0.5232556
0.5233049
0.5232942
0.5232508
0.5232850
0.5232777
0.5232502
0.5232748
0.5232690
0.5232500

7
7
7
9
8
8
9
8
9
11
11
11
13
12
13
15
14

11
9
9
12
12
11
13
11
12
15
14
15
18
17
17
20
18

18
79
25
20
175
35
31
97
43
34
269
66
53
355
92
76
389

References

1.Blaga, P.P., Bernstein operators to reduce of variance in random numerical
integration, in Proceedings of the International Symposium on Numerical Analysis
and Approximation Theory, Cluj-Napoca, May 9-11, 2002, Cluj University Press,
pp.76-91.
2.Gorbacheva, N.B., Sobol, I.M., Trikuzov, A.I., Variance-reducing multipliers in the
computation of integrals by the Monte Carlo method (Russian), Zh. Vychisl. Mat.
Mat. Fiz, Vol.41 (2001), 1310-1314.
3.Hammersley, J. M., Handscomb, D.C., Monte Carlo Methods, Methuen-John Wiley
and Sons, London-New York, 1964.
4.Handscomb, D.C., Remarks on a Monte Carlo integration method, Numer. Math.,
Vol.6 (1964), 261-268.
5.Lorentz, G.G., Bernstein Polynomials, University of Toronto Press, Toronto, 1953.
6.Powell, M.J.D., Swann, J., Weighted uniform sampling-a Monte Carlo technique
for reducing variance, J.Inst. Math. Appl., Vol.2 (1966), 228-236.
7.Rosenberg, L., Bernstein polynomials and Monte Carlo integration, SIAM J.
Numerical Analysis, Vol.4 (1967), 566-574.
8.Stancu, D.D., Coman, Gh., Blaga, P., Numerical Analysis and Approximation
Theory, Vol.II, Cluj University Press, 2002.

55

Petru P. Blaga-Reducing of variance by a combined scheme based on Bernstein
polynomials
Author:
Petru P. Blaga
“Babeş-Bolyai” University of Cluj-Napoca, Romania
pblaga@math.ubbcluj.ro

56