The Pyramid of Quality of Evidence

Diagram 9. The Pyramid of Quality of Evidence

Systematic Reviews

Critically-Appraised Topics

videnc &

Articles

y of e qualit

Randomized Controlled Trials

(RCTs) Cohort Studies

Case-Controlled Studies Case Series / Reports

Background Information / Expert Opinion

Source: Dartmouth College and Yale University 2006

Is Development Planning and Budgeting in

planning. In general, RPJMN 2015-2019 is

Indonesia Evidence-Based?

categorised into three books: Book I on the To answer the question above, RPJMN National Development Agenda; Book II on the 2015-2019 could be used to assess how far Development Agenda by Sector; and Book III on development planning in Indonesia has used the Development Agenda by Region. These three evidence as the basis for national development books, along with the Development Sector Matrix

Tabel 10. Use of Evidence in Planning and Budgeting Document Document

How is evidence being used?

2015–2019 • Vision and mission of president and vice president (government agenda) RPJMN

translated with RPJMN by using research results data and statistical data • Data to support strategic issues mapping, such as IPM, IPG, IKG, the National Socio-Economic Survey, etc. (time series analysis) • Data from government and non-government institution review results (for

example: BPS, KPU, LAPAN, Kementerian, KIP, IDI, etc.) • Developing programme achievement indicators based on available data

2016 RKP

• RKP is an annual breakdown of RPJMN • BPS data from past years in various sectors (time series analysis) • Data from government institution review results, such as the Indonesia

Democracy Index • Data from international organisations from past years in various sectors

(time series analysis) • Results of opinion polls or media surveys, or non-government organisations

• Budget availability (inance and resource implications)

Even though evidence has been used in the annual work plan has used evidence to determine

planning document, it cannot be used completely programmes, activities and temporary budget until the programme or activity is approved. This caps.

is due to the above discussed disconnections Based on 2015-2019 RPJMN checking, between the planning document (RKP) and the several facts found that these planning budgeting document which impact directly on documents–in a limited way–used evidence to the delivery of the funded / approved work plan develop the plan (see Table 10).

(RKA-K/L and APBN).

Various types of evidence used in the Based on interviews with several sources, a development of RPJMN and RKP are set as pre-planned programme by Bappenas could be baselines to determine targets, programmes “sabotaged”, as evidence does not necessarily or activities. The 2016 RKP’s innovation become the basis of determining budget ceiling. and technology area implicitly states that One example is the programme “15 Tourist the government will advance the technology Destinations”, proposed by Bappenas through and innovation sector through research a comprehensive review. However it became institutionalisation to improve Indonesian “10 Tourist Destinations” when implanted by the

Table 11. Factors Limiting Evidence-Based Policy Application

Planning and Budgeting Context in No

1 Speed Decision makers work in a situation In the context of planning and budgeting in where time is limited and there is political

Indonesia, the tight timeline of Musrenbang pressure. This causes available information from village to national level, starting from or evidence to be collected wherever

January every year, and political pressure possible and to be used as soon as

such as the tight schedule of budgeting possible.

discussions in DPR inluence the low use of evidence in the development planning

This issue leads to improvisation and

process.

compromise due to limited time and political pressure, as well as wrongful decision making.

2 Supericiality As decision makers have to deal

Government, especially regional

with various themes without in-depth government, lacks experts in all sectors. competency on all issues, they often

This produces low quality policy.

depend on the knowledge of people who provide them with information.

A similar issue also occurs in DPR. Assignment of an individual in DPR is This leads to the question: Who should

not necessarily based on expertise or give advice to decision makers and how

scholarship of a board member, and can should they assess the given advice or

be based merely on political consideration. information?

In addition, each commission in DPR manages many sectors.

Critical Study on

Development Planning and Budgeting in Indonesia

Planning and Budgeting Context in No

In the political world, perception is

Currently in Indonesia, the role of public

important. Public perception on certain

opinion has become more inluential in

issues–even if the reality is opposite to the

determining public policy. Developing

evidence–often becomes the reference in

programmes in accordance with public

decision making.

opinion will be directly proportional to the popularity of related oficials (president, minister or head of region).

4 Secrecy

Some evidence is conidential. Confusion

Procurement of a presidential airplane often occurs when explaining a public policy processed by former President Susilo based on evidence that has to be treated as Bambang Yudhoyono and completed by

a state secret.

President Jokowi was criticised as budget waste. Meanwhile, the presidential press team explained that the purchase actually saved the government rental costs. In addition, a presidential plane is crucial as protection for the president. The details of

the plane’s speciications and technology are state secrets.

5 Scientiic

It is scientiically acknowledged that ignorance

The growing phenomenon in public of

apathy or disbelief towards scientiic proof

cigarettes cause cancer and death, and are

has inluenced efforts to improving public

the second biggest expenditure for the poor

policy based on evidence.

(after rice) which causes increased APBN spending on public health. Nevertheless, the Ministry of Trade still steadfastly increases cigarette production in Indonesia,

while the Ministry of Health ights to conduct education on the dangers of smoking.

Ministry of Tourism. This “10 Tourist Destinations”

evaluation.

was never proposed in the RKP; instead it was

the result of direct lobbying by a private party to Other factors limiting the evidence-based policy application in Indonesia

the Ministry of Tourism. As explained, even though evidence has been

The government has attempted to strengthen used in the development planning document,

evidence as a development evaluation tool there are still many challenges to it dominating

through several evaluation initiatives, such as the

development in Indonesia.

Government Agency Performance Accountability Sutcliffe and Court (2005) in a publication

Report, a BPK audit, a BPKP audit, and a released by the Overseas Development Institute

performance review of K/L by the Ministry of State (ODI) explain that ‘5 S’ restrict evidence-based

Apparatus and Bureaucracy Reform on public policy from being applied in public policy

service delivery by the Ombudsman. However, development (see Table 11).

evaluation results from these state institutions Davies (2004) discusses several factors that

are yet to become reference points (for example in giving rewards or restrictions) for the next can inluence policy making in government.

They are experience and expertise, judgment, year’s planning and budgeting. For example, a

resources, values, habits and traditions, lobbyists ministry/institution can receive an average 10

and pressure groups, and pragmatism and percent budget increase without considering

contingencies.

the audit result from BPK or the Ombudsman’s

In the context of planning and budgeting anti-corruption commission (KPK) investigation in Indonesia, these factors have been proven revealed that an entrepreneur played a central valid. Lobbyists and pressure groups push role in ensuring the winning company and even the government to act within the framework could lobby to replace oficials in a certain ministry of evidence-based policy or vice versa. For (Kontan 2017). This project deviated from Law example, due to insistence from several NGOs, No. 23 2006 on population administration. The the Joko Widodo administration extended a forest

wrongdoing began in the planning process, moratorium established by former President as the electronic ID cards had been launched Susilo Bambang Yudhoyono (Seskab 2015). This

before the government actually had an integrated moratorium was extended based on a review by population grand design (ICW no year). This NGOs (pressure groups) which concluded that project was launched without any comprehensive the Presidential Instruction on the moratorium review and is still not fully implemented. released in 2011 by him had not been effective in decreasing the deforestation of primary forest

3.2. Bottlenecks in Planning and Budgeting

and peat lands (Syarif 2015). Based on the description of the main problems Another example involves the role of lobbyists

in the planning and budgeting stage above, in a negative way. When the Electronic National bottlenecks can be identiied based on the type ID card procurement project began in 2009, an of each problem (see Table 12).

Table 12. Bottlenecks in Planning and Budgeting

Areas

Problems

Political Process Disconnect in

Regulations

Institutional Actors

• Planning and budgeting

• Final decision maker of

• Still room for

development budget

regulated in two different

budget ceiling in executive

negotiation in budget

planning

laws (Law No. 17/2003 and

level before submission to

discussion

Law No. 25/2004)

legislature is not clear

• Different interpretation

• Bappenas as the

of the implementation of

development planner is not

Law No. 17/2003 and Law

involved in the budgeting

No. 25/2004 related to

process

institutional authority

• Sectoral ego of K/L in

• The review process of the

scoring its own programme/

planning and budgeting

activity

document still limited on formality

Development

• Planning and budgeting

• The role of regional

• DPRD still discusses

planning of central

regulated in two different

planning and budgeting in

budget details of

region is not in sync

laws (SPPN Law and

central government is

APBD

Regional Government Law)

handled by three different

• Political fragmentation

• Low compliance towards

institutions (Bappenas, MoF

between central and

regulation on authority

and MoHA.

regional government

distribution between

• MoHA is not involved in the

causing heads of

central, provincial and

formulation of the budget

region more loyal to

regional, causing overlap in

transfer process to regions

the supporting political

programmes/activities

• Low capacity of regional

party than central

• Location not described

government resources in

government (who often

in RKP and Renja K/L

regional development

come from different

documents causing

• Technical coordination

political parties)

regional government to not

meeting (Rakortek)

possess information on

between K/L and regional

programmes and activities

government is facilitated by

in their regions

MoHA and is yet to be the reference in determining DAK.

Critical Study on

Development Planning and Budgeting in Indonesia

Central-regional

• Law No. 17/2003 (UU KN)

• High number of institutions

• General and regional

planning and

regulates planning and

required to be involved

elections yet to be

budgeting timeline

budgeting between central

in this process: MoHA

run simultaneously

unaligned

and regional governments

Bappenas, the Ministry of

and yet to consider

conducted according to

Finance, K/L

the relationship with

different schedules

development budget planning

Development

• Yet to be a clear regulation • Planners lack capacity

• The legislature has

planning is not

mandating the use of

especially at province and

limited data related to

based on evidence

evidence in national

regency/city level

development planning,

development planning

• MoF and DPR have political

causing loss of control

• Budgets are not based on

imperatives

of the data submitted

plans

by the executive

Solutions and Strategic 4

Partners

4.1. Civil Society

Efforts to minimise bottlenecks and encourage systemic improvements in planning and budgeting through formal legal actions started in 2014, when a coalition of NGOs submitted a judicial review to the Constitutional Court (MK) on Law No. 27/2009 on the Legislative Institution (UU MD3) and Law No. 17 of 2003 on State Finances. The judicial review was submitted to minimise on- going transactional practices (read, corruption).

As a result, in May 2014, the Constitutional Court granted the judicial review. It decided to annul the authority of the budget committee of DPR to discuss budget paragraphs technically with government. The MK assembly reasoned that discussion conducted by the DPR committee on activities and type of spending of each ministry or government institution was beyond its authority, as technically it is a government function. However, the committee will limit its tasks discuss and approval draft APBN plans and APBN Laws, along with the government (Hukum Online 2014). In its decision, Constitutional Court repealed the phrases, “activity, and type of spending” in Article 15 paragraph (5) of the State Finances Law; “and activity” in Law No. 107 paragraph (1) letter c MD3 Law; “interactivity and intertype of

Critical Study on

Development Planning and Budgeting in Indonesia Development Planning and Budgeting in Indonesia

national priorities and RKP. (1) letter c MD3 Law; and “activity, and type of

3. Eliminate two different working committees spending” in Article 159 paragraph (5) MD3 Law.

(planning and budgeting) in shared The court provided a constitutional

discussion with DPR.

interpretation of Article 71 letter g MD3 Law,

4. Assist the president and vice president which stated that it is contradicting the 1945

to monitor and check the performance of Constitution when being interpreted as, “there

ministries/institutions. This has not been will be further discussion process after the APBN

able to be done due to the “tug of war” Bill is promulgated to APBN Law”.

between Bappenas and the Ministry of The court also shared an opinion on the practice

Finance.

of applying “asterisks” (bintang) to certain budget lines. These asterisks are interpreted by DPR

The development of the PP was as do not disburse the budget until the problem not easy. Bappenas has worked on causing the application of asterisks is resolved. the early draft of RPP since 2014. According to the court, this practice of blocking According to the PP, the Ministry of Finance or applying asterisks on certain ministry or agrees to monitor the national priority programme institution budget lines causes legal uncertainty, and activity from planning and budgeting to and it is already part of APBN implementation. In

implementation, so that no priority programme reality, DPR often requested kick-backs in return is cut off midway. In addition, the government for “deleting the asterisk” ( mencabut bintang). regulation can also make the roles of Bappenas This is not part of DPR’s monitoring function, as and the Ministry of Finance more eficient, as DPR’s authority is limited to approving RAPBN the development planning process is no longer and monitoring budget.

overlapping.

4.2. The Government

4.3. Possible Solutions and Interventions

At the end of January 2017, President Jokowi Even though MK has annulled DPR’s instructed Bappenas and MoF to joint develop authority to discuss the “project list” 1 and award

a Government Regulation (PP) on integrating asterisks, it is not enough to eradicate planning the planning and budgeting. This regulations and budgeting practices that do not comply with was approved in May 2017 as PP 17/2017 good governance and planning, as the root of on Synchronization of National Development the problem is still left untouched. Therefore, Planning and Budgeting. This regulation is targeted solutions to each of the bottlenecks are expected to solve several problems attached required. These solutions can be mapped (see to planning and budgeting in Indonesia. Points Table 13, below). expected to be achieved through this integration process are (Setkab 2017):

1. Close down space for negotiation and intervention towards budget planning, as the planning process is conducted by Bappenas, while budgeting is conducted by the Ministry of Finance. The government will regulate this process into one government regulation.

1 Satuan tiga is a unit in APBN that details information

2. Avoid ineficiency and budget leaks due to

of activity type, targets and budget amount.

Institutions Regulation

Planning and budgeting

• Bappenas regulated in two different laws

• In the long term, revise the SPPN

Planning Law and KN Finance Law

• MoF

• Vice President’s No. 25/2004)

(Law No.17/2003 and Law

into a single Law on Planning and

Budgeting

Ofice

• In the long term, government must

• DPR

develop a single institution to manage planning and budgeting by integrating existing institutions, such as the US Ofice of Management and Budget

• Vice President’s on the derivative of Law

Different interpretations

• Monitoring of PP 17/2017 on

Synchronization of National

Ofice

No. 17/2003 and Law No.

• Coordinating 25/ 2004 related to the

Development Planning and

Ministry for the institutional authority

Budgeting

Economy • Executive Ofice of the President • Bappenas • MoF

Architecture and Performance • Improving ADIK: • Executive Ofice Information (Arsitektur dan

of the President Informasi Kinerja/ ADIK) yet

o Ministry of Finance, MoHA

Bappenas and other ministries/

• MoF

to be uniformed

institutions must have the same

• Bappenas

terminology for programs and

o Declassiication programme cost

should become “service cost” and “non-service cost”

Planning and budgeting

• Bappenas in central and regions

• In the long term, regulations must

be uniied into a single Law on

• MoF

regulated in two different laws

Planning and Budgeting

• MoHA

(SPPN Law and Regional Government Law)

• Presidential Decree or Government

Regulation is required to regulate the synergy of central and regional planning, especially for programmes with national priority

Low compliance towards

• Executive Ofice regulation on authority

• Government rigor is required in

of the President distribution between central,

complying with regulations and

giving sanction to digressing K/L.

• BPK

provincial and regional,

• BPK audit result must become the

causing overlapping

reference in evaluating government

programme/activity

performance, especially related to the indings on overlapped

programmes/activities

Location not being described • Improving RKP and Renja K/L by • Bappenas in RKP and Renja K/L

adding information on programme / • K/L

documents causing regional

activity location

government to not have information on programmes/ activities in their regions

Critical Study on

Development Planning and Budgeting in Indonesia

Law No. 17/2003 regulates

• Harmonisation of planning and

• Bappenas

planning and budgeting

budgeting in one cycle

• MoF

between central and regional • Differentiating the budget year

• MoHA

conducted in accordance with

between central and regional

electoral cycles

governments

Lack of clear regulation

• Evidence use needs to be

• Executive Ofice of

mandating the use of

conirmed in a regulation, at least

the President

evidence in national

a Presidential Decree for central

• Bappenas

development planning

government and Regulation

• MoHA

of Minister of Home Affairs for regional government with inclusion of guidance on evidence-based planning development

Institutional Final decision maker of

• Single budget cap that cannot

• President

actors

budget ceiling at executive

be revised without approval from

level before submission to

Bappenas

legislature is not clear Bappenas as the

• Bappenas together with the Ministry • Bappenas

development planner is not

of Finance involved since the

• MoF

involved in the budgeting

beginning of the planning process

process

until APBN is passed

Sectoral ego of K/L in scoring • Alignment of out-of-sync working

• K/L

its own programme/ activity

process across K/L

The role of regional planning • Better synergy between Ministry of • MoF

and budgeting in central

Finance and MoHA

• MoHa

government is handled by two different institutions

MoHA is not involved in

• MoHA needs to be involved in

• MoF

formulation process of budget

fund transfer formulation process

• MoHA

transfer to regions

so that the role of supervision and evaluation can be maximised

Low capacity of regional

• Supervision on regional

• MoHA

government resources

development planning by

• Pemda

in regional development

maximising the role of MoHA

planning areas causing

• Increasing the capacity of Bappeda

regional planning to be out of

as the planner at regional level

synergy with central level Technical coordination

• Maximising the function of Rakortek • MoHA

between K/L and regional

to be the forum for synergic central

government, which is

and regional planning, including as

facilitated by MoHA, is yet

one of the references in determining

to be the reference point in

DAK

determining DAK High number of institutions

• The roadmap of central and regional • Bappenas

required to be involved in this

synergic planning needs to be

• MoF

process, MoHA

developed together by Bappenas,

• MoHA

Bappenas, Ministry of

MoF, MoHA, and KPU.

• Election

Finance, Line Ministries

Commission (KPU)

Planners lack capacity,

• Increasing the capacity of regional

• Bappenas

especially at province and

planners in order to understand the

regency/city level causing

importance of evidence-based policy

regional planning not to be

as we`ll as having technical skills in

based on evidence

developing evidence-based planning

Institutions Political

Room for negotiation on

• Close down room for negotiation

• MoF

process

budget discussion is still wide

• Bappenas open

on budgeting that may trigger

ineficiency and corruption

• K/L • DPR

DPRD still discusses the

• President APBD project list

• Regulation is required to prohibit

DPRD from discussing budget

• MoHA

details in RAPBD discussions. This regulation can be released as a Presidential Decree or Regulation of the Minister of Home Affairs.

Political fragmentation

• Simultaneous general election

• KPU

between central and regional

and regional election will minimise

• MoHA

government causing heads

political fragmentation

of regions to be more loyal to the supporting political party than central government (who often come from different political parties).

General election and regional • General election and regional

• KPU

election yet to be run

election conducted simultaneously

• MoHA

simultaneously

in order to consider the synergy of central and regional planning

• Executive Ofice of data related to development

The legislature has limited

• Data used in planning document

the President planning causing loss

need to be open to the public

• Improvement in data integrity and

of control over the data

reliability

submitted by the executive

The stakeholders mapped above only come support can be given for its implementation. from the government element. Non-government

3. Improving the Architecture and

elements, such as NGOs, think tanks and the Performance Information (ADIK). media continue to be important stakeholders

4. Improving RKP and Renja K/L by adding who need to be involved. They can take on the

information on the location of programmes/ role of building public opinion, pressure and

activities.

policy advocacy as well as providing assistance

5. Supervising regional development planning to increase the capacity of regional government.

by maximising the role of MoHA. Among the solutions above, several are

6. Maximising the function of Rakortek categorised as quick wins and can be delivered

between MoHA with K/L to synergise within the next one to two years. These quick wins

national and regional priority programmes. can also be inputs for KSI to conduct programme

7. Developing regulations and guidance interventions, such as:

in evidence-based planning for regional

1. Monitoring RPP National Development

government.

Planning and Budgeting. This RPP is also

8. Increasing the capacity of Bappeda as a the basis to close down room for budget

planner at regional level.

negotiation in DPR and regulate a single

9. Increasing the capacity of planners budget cap.

in regions in order to understand the

2. If a government regulation is issued, importance of evidence-based policy,

Critical Study on

Development Planning and Budgeting in Indonesia Development Planning and Budgeting in Indonesia

2. Government needs to build one develop evidence-based planning.

separate institution to manage planning and budgeting by integrating existing

The intermediate term solutions that can be

institutions.

delivered in the next three to ive years are:

3. Simultaneous general election and regional

1. Involvement of MoHA in the fund transfer elections by considering the synergy of formulation process in order to maximise

central and regional planning. the role of supervision and evaluation.

2. Joint development of a roadmap of

4.4. Strategic Partners

central and regional synergic planning by The tool places identiied stakeholders into

Bappenas, MoF, MoHA, and KPU.

four categories:

1. Key players: Stakeholders in this category and availability of data used in planning

3. Improvement of the integrity, reliability

are considered key stakeholders, as they documents that need to be open to the

have high interest and inluence or power. public.

These stakeholders are the main focus and should be involved routinely in every

The long-term solutions

(ive years and applied intervention.

above) that can be done:

2. Meet their needs: Stakeholders in this

1. Revision of the Planning Law and the category are those with relatively low Finance Law into single Law on Planning

interest and inluence or power. This group and Budgeting. This law should also

still needs to be involved in the process by regulate the synergy of planning and

increasing their interest towards advocated budgeting between central and regional

issues. In the end, this group is expected to governments.

transform into key players.