The Pyramid of Quality of Evidence
Diagram 9. The Pyramid of Quality of Evidence
Systematic Reviews
Critically-Appraised Topics
videnc &
Articles
y of e qualit
Randomized Controlled Trials
(RCTs) Cohort Studies
Case-Controlled Studies Case Series / Reports
Background Information / Expert Opinion
Source: Dartmouth College and Yale University 2006
Is Development Planning and Budgeting in
planning. In general, RPJMN 2015-2019 is
Indonesia Evidence-Based?
categorised into three books: Book I on the To answer the question above, RPJMN National Development Agenda; Book II on the 2015-2019 could be used to assess how far Development Agenda by Sector; and Book III on development planning in Indonesia has used the Development Agenda by Region. These three evidence as the basis for national development books, along with the Development Sector Matrix
Tabel 10. Use of Evidence in Planning and Budgeting Document Document
How is evidence being used?
2015–2019 • Vision and mission of president and vice president (government agenda) RPJMN
translated with RPJMN by using research results data and statistical data • Data to support strategic issues mapping, such as IPM, IPG, IKG, the National Socio-Economic Survey, etc. (time series analysis) • Data from government and non-government institution review results (for
example: BPS, KPU, LAPAN, Kementerian, KIP, IDI, etc.) • Developing programme achievement indicators based on available data
2016 RKP
• RKP is an annual breakdown of RPJMN • BPS data from past years in various sectors (time series analysis) • Data from government institution review results, such as the Indonesia
Democracy Index • Data from international organisations from past years in various sectors
(time series analysis) • Results of opinion polls or media surveys, or non-government organisations
• Budget availability (inance and resource implications)
Even though evidence has been used in the annual work plan has used evidence to determine
planning document, it cannot be used completely programmes, activities and temporary budget until the programme or activity is approved. This caps.
is due to the above discussed disconnections Based on 2015-2019 RPJMN checking, between the planning document (RKP) and the several facts found that these planning budgeting document which impact directly on documents–in a limited way–used evidence to the delivery of the funded / approved work plan develop the plan (see Table 10).
(RKA-K/L and APBN).
Various types of evidence used in the Based on interviews with several sources, a development of RPJMN and RKP are set as pre-planned programme by Bappenas could be baselines to determine targets, programmes “sabotaged”, as evidence does not necessarily or activities. The 2016 RKP’s innovation become the basis of determining budget ceiling. and technology area implicitly states that One example is the programme “15 Tourist the government will advance the technology Destinations”, proposed by Bappenas through and innovation sector through research a comprehensive review. However it became institutionalisation to improve Indonesian “10 Tourist Destinations” when implanted by the
Table 11. Factors Limiting Evidence-Based Policy Application
Planning and Budgeting Context in No
1 Speed Decision makers work in a situation In the context of planning and budgeting in where time is limited and there is political
Indonesia, the tight timeline of Musrenbang pressure. This causes available information from village to national level, starting from or evidence to be collected wherever
January every year, and political pressure possible and to be used as soon as
such as the tight schedule of budgeting possible.
discussions in DPR inluence the low use of evidence in the development planning
This issue leads to improvisation and
process.
compromise due to limited time and political pressure, as well as wrongful decision making.
2 Supericiality As decision makers have to deal
Government, especially regional
with various themes without in-depth government, lacks experts in all sectors. competency on all issues, they often
This produces low quality policy.
depend on the knowledge of people who provide them with information.
A similar issue also occurs in DPR. Assignment of an individual in DPR is This leads to the question: Who should
not necessarily based on expertise or give advice to decision makers and how
scholarship of a board member, and can should they assess the given advice or
be based merely on political consideration. information?
In addition, each commission in DPR manages many sectors.
Critical Study on
Development Planning and Budgeting in Indonesia
Planning and Budgeting Context in No
In the political world, perception is
Currently in Indonesia, the role of public
important. Public perception on certain
opinion has become more inluential in
issues–even if the reality is opposite to the
determining public policy. Developing
evidence–often becomes the reference in
programmes in accordance with public
decision making.
opinion will be directly proportional to the popularity of related oficials (president, minister or head of region).
4 Secrecy
Some evidence is conidential. Confusion
Procurement of a presidential airplane often occurs when explaining a public policy processed by former President Susilo based on evidence that has to be treated as Bambang Yudhoyono and completed by
a state secret.
President Jokowi was criticised as budget waste. Meanwhile, the presidential press team explained that the purchase actually saved the government rental costs. In addition, a presidential plane is crucial as protection for the president. The details of
the plane’s speciications and technology are state secrets.
5 Scientiic
It is scientiically acknowledged that ignorance
The growing phenomenon in public of
apathy or disbelief towards scientiic proof
cigarettes cause cancer and death, and are
has inluenced efforts to improving public
the second biggest expenditure for the poor
policy based on evidence.
(after rice) which causes increased APBN spending on public health. Nevertheless, the Ministry of Trade still steadfastly increases cigarette production in Indonesia,
while the Ministry of Health ights to conduct education on the dangers of smoking.
Ministry of Tourism. This “10 Tourist Destinations”
evaluation.
was never proposed in the RKP; instead it was
the result of direct lobbying by a private party to Other factors limiting the evidence-based policy application in Indonesia
the Ministry of Tourism. As explained, even though evidence has been
The government has attempted to strengthen used in the development planning document,
evidence as a development evaluation tool there are still many challenges to it dominating
through several evaluation initiatives, such as the
development in Indonesia.
Government Agency Performance Accountability Sutcliffe and Court (2005) in a publication
Report, a BPK audit, a BPKP audit, and a released by the Overseas Development Institute
performance review of K/L by the Ministry of State (ODI) explain that ‘5 S’ restrict evidence-based
Apparatus and Bureaucracy Reform on public policy from being applied in public policy
service delivery by the Ombudsman. However, development (see Table 11).
evaluation results from these state institutions Davies (2004) discusses several factors that
are yet to become reference points (for example in giving rewards or restrictions) for the next can inluence policy making in government.
They are experience and expertise, judgment, year’s planning and budgeting. For example, a
resources, values, habits and traditions, lobbyists ministry/institution can receive an average 10
and pressure groups, and pragmatism and percent budget increase without considering
contingencies.
the audit result from BPK or the Ombudsman’s
In the context of planning and budgeting anti-corruption commission (KPK) investigation in Indonesia, these factors have been proven revealed that an entrepreneur played a central valid. Lobbyists and pressure groups push role in ensuring the winning company and even the government to act within the framework could lobby to replace oficials in a certain ministry of evidence-based policy or vice versa. For (Kontan 2017). This project deviated from Law example, due to insistence from several NGOs, No. 23 2006 on population administration. The the Joko Widodo administration extended a forest
wrongdoing began in the planning process, moratorium established by former President as the electronic ID cards had been launched Susilo Bambang Yudhoyono (Seskab 2015). This
before the government actually had an integrated moratorium was extended based on a review by population grand design (ICW no year). This NGOs (pressure groups) which concluded that project was launched without any comprehensive the Presidential Instruction on the moratorium review and is still not fully implemented. released in 2011 by him had not been effective in decreasing the deforestation of primary forest
3.2. Bottlenecks in Planning and Budgeting
and peat lands (Syarif 2015). Based on the description of the main problems Another example involves the role of lobbyists
in the planning and budgeting stage above, in a negative way. When the Electronic National bottlenecks can be identiied based on the type ID card procurement project began in 2009, an of each problem (see Table 12).
Table 12. Bottlenecks in Planning and Budgeting
Areas
Problems
Political Process Disconnect in
Regulations
Institutional Actors
• Planning and budgeting
• Final decision maker of
• Still room for
development budget
regulated in two different
budget ceiling in executive
negotiation in budget
planning
laws (Law No. 17/2003 and
level before submission to
discussion
Law No. 25/2004)
legislature is not clear
• Different interpretation
• Bappenas as the
of the implementation of
development planner is not
Law No. 17/2003 and Law
involved in the budgeting
No. 25/2004 related to
process
institutional authority
• Sectoral ego of K/L in
• The review process of the
scoring its own programme/
planning and budgeting
activity
document still limited on formality
Development
• Planning and budgeting
• The role of regional
• DPRD still discusses
planning of central
regulated in two different
planning and budgeting in
budget details of
region is not in sync
laws (SPPN Law and
central government is
APBD
Regional Government Law)
handled by three different
• Political fragmentation
• Low compliance towards
institutions (Bappenas, MoF
between central and
regulation on authority
and MoHA.
regional government
distribution between
• MoHA is not involved in the
causing heads of
central, provincial and
formulation of the budget
region more loyal to
regional, causing overlap in
transfer process to regions
the supporting political
programmes/activities
• Low capacity of regional
party than central
• Location not described
government resources in
government (who often
in RKP and Renja K/L
regional development
come from different
documents causing
• Technical coordination
political parties)
regional government to not
meeting (Rakortek)
possess information on
between K/L and regional
programmes and activities
government is facilitated by
in their regions
MoHA and is yet to be the reference in determining DAK.
Critical Study on
Development Planning and Budgeting in Indonesia
Central-regional
• Law No. 17/2003 (UU KN)
• High number of institutions
• General and regional
planning and
regulates planning and
required to be involved
elections yet to be
budgeting timeline
budgeting between central
in this process: MoHA
run simultaneously
unaligned
and regional governments
Bappenas, the Ministry of
and yet to consider
conducted according to
Finance, K/L
the relationship with
different schedules
development budget planning
Development
• Yet to be a clear regulation • Planners lack capacity
• The legislature has
planning is not
mandating the use of
especially at province and
limited data related to
based on evidence
evidence in national
regency/city level
development planning,
development planning
• MoF and DPR have political
causing loss of control
• Budgets are not based on
imperatives
of the data submitted
plans
by the executive
Solutions and Strategic 4
Partners
4.1. Civil Society
Efforts to minimise bottlenecks and encourage systemic improvements in planning and budgeting through formal legal actions started in 2014, when a coalition of NGOs submitted a judicial review to the Constitutional Court (MK) on Law No. 27/2009 on the Legislative Institution (UU MD3) and Law No. 17 of 2003 on State Finances. The judicial review was submitted to minimise on- going transactional practices (read, corruption).
As a result, in May 2014, the Constitutional Court granted the judicial review. It decided to annul the authority of the budget committee of DPR to discuss budget paragraphs technically with government. The MK assembly reasoned that discussion conducted by the DPR committee on activities and type of spending of each ministry or government institution was beyond its authority, as technically it is a government function. However, the committee will limit its tasks discuss and approval draft APBN plans and APBN Laws, along with the government (Hukum Online 2014). In its decision, Constitutional Court repealed the phrases, “activity, and type of spending” in Article 15 paragraph (5) of the State Finances Law; “and activity” in Law No. 107 paragraph (1) letter c MD3 Law; “interactivity and intertype of
Critical Study on
Development Planning and Budgeting in Indonesia Development Planning and Budgeting in Indonesia
national priorities and RKP. (1) letter c MD3 Law; and “activity, and type of
3. Eliminate two different working committees spending” in Article 159 paragraph (5) MD3 Law.
(planning and budgeting) in shared The court provided a constitutional
discussion with DPR.
interpretation of Article 71 letter g MD3 Law,
4. Assist the president and vice president which stated that it is contradicting the 1945
to monitor and check the performance of Constitution when being interpreted as, “there
ministries/institutions. This has not been will be further discussion process after the APBN
able to be done due to the “tug of war” Bill is promulgated to APBN Law”.
between Bappenas and the Ministry of The court also shared an opinion on the practice
Finance.
of applying “asterisks” (bintang) to certain budget lines. These asterisks are interpreted by DPR
The development of the PP was as do not disburse the budget until the problem not easy. Bappenas has worked on causing the application of asterisks is resolved. the early draft of RPP since 2014. According to the court, this practice of blocking According to the PP, the Ministry of Finance or applying asterisks on certain ministry or agrees to monitor the national priority programme institution budget lines causes legal uncertainty, and activity from planning and budgeting to and it is already part of APBN implementation. In
implementation, so that no priority programme reality, DPR often requested kick-backs in return is cut off midway. In addition, the government for “deleting the asterisk” ( mencabut bintang). regulation can also make the roles of Bappenas This is not part of DPR’s monitoring function, as and the Ministry of Finance more eficient, as DPR’s authority is limited to approving RAPBN the development planning process is no longer and monitoring budget.
overlapping.
4.2. The Government
4.3. Possible Solutions and Interventions
At the end of January 2017, President Jokowi Even though MK has annulled DPR’s instructed Bappenas and MoF to joint develop authority to discuss the “project list” 1 and award
a Government Regulation (PP) on integrating asterisks, it is not enough to eradicate planning the planning and budgeting. This regulations and budgeting practices that do not comply with was approved in May 2017 as PP 17/2017 good governance and planning, as the root of on Synchronization of National Development the problem is still left untouched. Therefore, Planning and Budgeting. This regulation is targeted solutions to each of the bottlenecks are expected to solve several problems attached required. These solutions can be mapped (see to planning and budgeting in Indonesia. Points Table 13, below). expected to be achieved through this integration process are (Setkab 2017):
1. Close down space for negotiation and intervention towards budget planning, as the planning process is conducted by Bappenas, while budgeting is conducted by the Ministry of Finance. The government will regulate this process into one government regulation.
1 Satuan tiga is a unit in APBN that details information
2. Avoid ineficiency and budget leaks due to
of activity type, targets and budget amount.
Institutions Regulation
Planning and budgeting
• Bappenas regulated in two different laws
• In the long term, revise the SPPN
Planning Law and KN Finance Law
• MoF
• Vice President’s No. 25/2004)
(Law No.17/2003 and Law
into a single Law on Planning and
Budgeting
Ofice
• In the long term, government must
• DPR
develop a single institution to manage planning and budgeting by integrating existing institutions, such as the US Ofice of Management and Budget
• Vice President’s on the derivative of Law
Different interpretations
• Monitoring of PP 17/2017 on
Synchronization of National
Ofice
No. 17/2003 and Law No.
• Coordinating 25/ 2004 related to the
Development Planning and
Ministry for the institutional authority
Budgeting
Economy • Executive Ofice of the President • Bappenas • MoF
Architecture and Performance • Improving ADIK: • Executive Ofice Information (Arsitektur dan
of the President Informasi Kinerja/ ADIK) yet
o Ministry of Finance, MoHA
Bappenas and other ministries/
• MoF
to be uniformed
institutions must have the same
• Bappenas
terminology for programs and
o Declassiication programme cost
should become “service cost” and “non-service cost”
Planning and budgeting
• Bappenas in central and regions
• In the long term, regulations must
be uniied into a single Law on
• MoF
regulated in two different laws
Planning and Budgeting
• MoHA
(SPPN Law and Regional Government Law)
• Presidential Decree or Government
Regulation is required to regulate the synergy of central and regional planning, especially for programmes with national priority
Low compliance towards
• Executive Ofice regulation on authority
• Government rigor is required in
of the President distribution between central,
complying with regulations and
giving sanction to digressing K/L.
• BPK
provincial and regional,
• BPK audit result must become the
causing overlapping
reference in evaluating government
programme/activity
performance, especially related to the indings on overlapped
programmes/activities
Location not being described • Improving RKP and Renja K/L by • Bappenas in RKP and Renja K/L
adding information on programme / • K/L
documents causing regional
activity location
government to not have information on programmes/ activities in their regions
Critical Study on
Development Planning and Budgeting in Indonesia
Law No. 17/2003 regulates
• Harmonisation of planning and
• Bappenas
planning and budgeting
budgeting in one cycle
• MoF
between central and regional • Differentiating the budget year
• MoHA
conducted in accordance with
between central and regional
electoral cycles
governments
Lack of clear regulation
• Evidence use needs to be
• Executive Ofice of
mandating the use of
conirmed in a regulation, at least
the President
evidence in national
a Presidential Decree for central
• Bappenas
development planning
government and Regulation
• MoHA
of Minister of Home Affairs for regional government with inclusion of guidance on evidence-based planning development
Institutional Final decision maker of
• Single budget cap that cannot
• President
actors
budget ceiling at executive
be revised without approval from
level before submission to
Bappenas
legislature is not clear Bappenas as the
• Bappenas together with the Ministry • Bappenas
development planner is not
of Finance involved since the
• MoF
involved in the budgeting
beginning of the planning process
process
until APBN is passed
Sectoral ego of K/L in scoring • Alignment of out-of-sync working
• K/L
its own programme/ activity
process across K/L
The role of regional planning • Better synergy between Ministry of • MoF
and budgeting in central
Finance and MoHA
• MoHa
government is handled by two different institutions
MoHA is not involved in
• MoHA needs to be involved in
• MoF
formulation process of budget
fund transfer formulation process
• MoHA
transfer to regions
so that the role of supervision and evaluation can be maximised
Low capacity of regional
• Supervision on regional
• MoHA
government resources
development planning by
• Pemda
in regional development
maximising the role of MoHA
planning areas causing
• Increasing the capacity of Bappeda
regional planning to be out of
as the planner at regional level
synergy with central level Technical coordination
• Maximising the function of Rakortek • MoHA
between K/L and regional
to be the forum for synergic central
government, which is
and regional planning, including as
facilitated by MoHA, is yet
one of the references in determining
to be the reference point in
DAK
determining DAK High number of institutions
• The roadmap of central and regional • Bappenas
required to be involved in this
synergic planning needs to be
• MoF
process, MoHA
developed together by Bappenas,
• MoHA
Bappenas, Ministry of
MoF, MoHA, and KPU.
• Election
Finance, Line Ministries
Commission (KPU)
Planners lack capacity,
• Increasing the capacity of regional
• Bappenas
especially at province and
planners in order to understand the
regency/city level causing
importance of evidence-based policy
regional planning not to be
as we`ll as having technical skills in
based on evidence
developing evidence-based planning
Institutions Political
Room for negotiation on
• Close down room for negotiation
• MoF
process
budget discussion is still wide
• Bappenas open
on budgeting that may trigger
ineficiency and corruption
• K/L • DPR
DPRD still discusses the
• President APBD project list
• Regulation is required to prohibit
DPRD from discussing budget
• MoHA
details in RAPBD discussions. This regulation can be released as a Presidential Decree or Regulation of the Minister of Home Affairs.
Political fragmentation
• Simultaneous general election
• KPU
between central and regional
and regional election will minimise
• MoHA
government causing heads
political fragmentation
of regions to be more loyal to the supporting political party than central government (who often come from different political parties).
General election and regional • General election and regional
• KPU
election yet to be run
election conducted simultaneously
• MoHA
simultaneously
in order to consider the synergy of central and regional planning
• Executive Ofice of data related to development
The legislature has limited
• Data used in planning document
the President planning causing loss
need to be open to the public
• Improvement in data integrity and
of control over the data
reliability
submitted by the executive
The stakeholders mapped above only come support can be given for its implementation. from the government element. Non-government
3. Improving the Architecture and
elements, such as NGOs, think tanks and the Performance Information (ADIK). media continue to be important stakeholders
4. Improving RKP and Renja K/L by adding who need to be involved. They can take on the
information on the location of programmes/ role of building public opinion, pressure and
activities.
policy advocacy as well as providing assistance
5. Supervising regional development planning to increase the capacity of regional government.
by maximising the role of MoHA. Among the solutions above, several are
6. Maximising the function of Rakortek categorised as quick wins and can be delivered
between MoHA with K/L to synergise within the next one to two years. These quick wins
national and regional priority programmes. can also be inputs for KSI to conduct programme
7. Developing regulations and guidance interventions, such as:
in evidence-based planning for regional
1. Monitoring RPP National Development
government.
Planning and Budgeting. This RPP is also
8. Increasing the capacity of Bappeda as a the basis to close down room for budget
planner at regional level.
negotiation in DPR and regulate a single
9. Increasing the capacity of planners budget cap.
in regions in order to understand the
2. If a government regulation is issued, importance of evidence-based policy,
Critical Study on
Development Planning and Budgeting in Indonesia Development Planning and Budgeting in Indonesia
2. Government needs to build one develop evidence-based planning.
separate institution to manage planning and budgeting by integrating existing
The intermediate term solutions that can be
institutions.
delivered in the next three to ive years are:
3. Simultaneous general election and regional
1. Involvement of MoHA in the fund transfer elections by considering the synergy of formulation process in order to maximise
central and regional planning. the role of supervision and evaluation.
2. Joint development of a roadmap of
4.4. Strategic Partners
central and regional synergic planning by The tool places identiied stakeholders into
Bappenas, MoF, MoHA, and KPU.
four categories:
1. Key players: Stakeholders in this category and availability of data used in planning
3. Improvement of the integrity, reliability
are considered key stakeholders, as they documents that need to be open to the
have high interest and inluence or power. public.
These stakeholders are the main focus and should be involved routinely in every
The long-term solutions
(ive years and applied intervention.
above) that can be done:
2. Meet their needs: Stakeholders in this
1. Revision of the Planning Law and the category are those with relatively low Finance Law into single Law on Planning
interest and inluence or power. This group and Budgeting. This law should also
still needs to be involved in the process by regulate the synergy of planning and
increasing their interest towards advocated budgeting between central and regional
issues. In the end, this group is expected to governments.
transform into key players.