Monitoring Service Delivery Outcomes
On this page you will find a variety of tools that can be used to design your own activities to monitor service delivery in the public sector. The diagrams offer you a step by step guide to the activities.
Participatory Budgeting is a process through which citizens participate directly in the different phases of budget formulation, decision making, and monitoring of budget execution. Public budgeting increases public expenditure transparency and budget targeting. It also promotes civic engagement and social learning. CIVICUS identifies the following steps outlined in the diagram below.
Examples of Zambian applications of participatory budgeting
Participatory Budgeting in Kabwe
The Municipal Development Partnership for Eastern and Southern Africa, in conjunction with Kabwe Municipal Council, decided to strengthen civic participation in municipal governance by including citizens in the Kabwe Strategic Planning Process. This led to citizens participation in the budgeting process, which is now undertaken with participation from Residents Development Committees (RDCs) that are based in both planned and unplanned settlements, as well as with the participation of groups such as the Chamber of Commerce, Rotary club, informal traders groups, churches and transport operators. To establish a link between the Council and the RDCs and other stakeholders, Kabwe Municipal Council created a Settlement Improvement Unit in the Public Health and Social Service Department, who is tasked with the promotion of civic participation and identifying strategies for improving the living conditions of local residents.
RDCs in their respective wards develop proposals for local development and pass these on to the ward councillor and the Council’s Departmental Heads who then present them to the full Council. The Council then provides feedback through the same channels. The participation of the RDCs in the budgetary process has led to the prioritisation of a large number of community projects and to the RDCs also acting as revenue collectors on behalf of the council, e.g. the fees from water charges. As an incentive the RDCs retain a certain percentage of the collected revenue. The retained amount is ploughed back into community projects. This helped to improve the revenue collection capacity of the municipal council particularly in the case of water provision charges. However the process has also faced many challenges, such as: the lack of participation in communal works by certain parts of the community, the low percentage of female chairpersons of the RDCs, and also limited interaction between elected officials and their constituents and the fact that the elected councillor is not the chairperson of the RDCs.
From MUMVUMA, T: MDP Policy Research Sub-National Experiences of Civic Participation in Policy Making and Budgetary Processes, A Case Study of Kabwe Municipal Council, Zambia. Municipal Development Partnership, Eastern & Southern Africa
Read More About Participatory Budgeting
A Guide to Participatory Budgeting,WAMPLER, B. 2010.
Participatory Budgeting. Washington: The World Bank SHAH, A. 2007.
Independent budget analysis demystifies the often highly technical language of official budgets and opens up to public scrutiny the budgetary process. budget analysis is closely linked with the process of budget formulation, as it is aimed at generating debate on the national budget and at influencing the budget that is ultimately app roved. Typically, this work focuses on one or more of the following issues:
- improving information sharing and public understanding of the budget
- increasing pro-poor allocations
- initiating debates on sector specific implications of budget allocations
- influencing revenue policies and
- tracking revenues and expenditures
According to the World Bank, IBA involves the following steps:
An Example of IBA in Zambia
In Zambia, the Non-Governmental Organisation Coordinating Council (NGOCC) has undertaken an analysis of the national budget from a gender perspective. Copies of these can be found on the NGOCC website, for example the NGOCC 2012 National Budget Analysis from a gender perspective.
Read more about it
RAMKUMAR, V. 2011. A Citizen’s Guide to Monitoring Government Expenditures. International Budget Partnership.
SHAPIRO, I. 2001. A Guide to Budget Work for NGOs. International Budget Partnership.
The social contract or local agenda is an agreement that is put into writing between the people (this may be at local or national level) and people who are in or who hope to be in government positions in the future. The social contract states clearly what the public official has agreed to in terms of service delivery, mandated responsibilities and public obligations. The social contract is a means of securing a public, signed agreement so that the public can hold duty bearers to account. Often it is a tool used before an election process whereby citizens ask candidates for election to commit in writing publically to the social contract. Those who do not sign the contract are therefore less likely to secure votes locally; and those who do sign can then be held to account for what they have signed.
The social contract process also encourages active citizenship, since it requires citizens to identify the key issues they wish to hold their public officials to. The process encourages voters to choose leaders for their clear commitments and performance, not for the small gifts or clever speeches they make. In the Philippines, CSOs have used social contracts in conjunction with innovative modes of citizen engagement, emphasising the participation of a critical mass that can create social pressure. In Peru, child stunting fell by a third between 2006 and 2011 following a push for presidential candidates before the 2006 election to sign a social contract knows as ‘5 by 5 by 5’ in which they committed to reducing stunting in children under 5 by 5 per cent in 5 years and to lessen inequities between urban and rural areas. The approach can also be used beyond elections since the social contract can be monitored on an ongoing basis.
Examples of Zambian applications of Social Contracts
At the local level, in 2006 Micah Challenge Zambia and the Jubilee Centre of Ndola took advantage of the general elections to engage local politicians. Mapalo pastors mobilized the community to draft a memorandum of understanding (MoU) or social contract with the candidates during the 2006 elections. They used this social contract to advocate for the commitment of candidates to meeting the needs of the community. The pastors started by identifying a range of needs in Mapalo Community (such as road repairs, establishing a secondary school, formalizing the settlement, upgrading local clinic to a hospital , bringing in piped water, improving and so on). After drafting the MoU that sought commitment and support from the politicians, the pastors organized a meeting for the community leaders and then with the wider community to gain their views, and members of the community signed the memorandum of understanding to show their commitment. The community even agreed to contribute 25% of the labor and resources necessary for the needs to be met. Community leaders were given copies of the social contract and went out to get support from the whole community and 3500 residents signed it to say they agreed that it outlined their needs. Two leaders then represented Mapalo at an open press conference for all the candidates standing for elections. The leaders challenged the candidates to sign the MoU and the candidates were invited to a community meeting to publicly show their support. The community leaders followed this up by sending a copy of the MoU and an invitation letter to each candidate.
In September 2006 the electoral candidates, pastors, community leaders and over 1000 community members attended a community meeting. All the candidates standing as councilors and three of the candidates standing for parliament attended the meeting at which they signed the MoU and agreed to meet most of the stated needs in their first three years of their term if elected. Signed copies of the memorandum were sent to the district court for public documentation. News of the meeting spread quickly and was heard on radios and television across Zambia. Those candidates who had previously been unable or unwilling to sign the MoU then quickly visited Mapalo to sign the document. A community youth initiative in Mapalo which had tried to get government funding for the previous three years, received government approval.
After the elections the community leaders wrote a letter congratulating the newly elected councilors and the Members of Parliament to remind them to meet the needs of Mapalo during their terms of office. Since this time there have been many improvements in Mapalo: setting up water kiosks to enable residents to buy safe water; rebuilding a bridge that links the community to the town centre; promises to upgrade the community from an illegal settlement to a site and service community; and establishing the Ndola Development Trust which has brought together politicians and the business community to develop various communities in the City. Whilst the Electoral Commission of Zambia identified the Mapalo Social Contract as a model to be adopted by other communities, some of the key pastors were harassed by politicians who accused them of being too political. However other communities in Ndola developed their own MoU for use in advocacy work after the election.
At the national level, prior to the 2011 elections the Citizens Forum asked the four presidential candidates to sign a social contract regarding the national constitution and limits to salary increments for the constitutional office holders. Whilst the President at the time, Rupiah Banda, refused to sign it, the PF government, who won the election, put in its Manifesto under page 42 that “the PF government shall: review all previous Constitutional Review Commissions submissions through a Committee of Experts for submission to a referendum and enactment only, by the National Assembly.” However as of August 2014 the PF government had not yet shared the draft constitution with the public, and the President was saying that the country already has a functional constitution and the state will not be pushed into “fast and reckless conclusions by individuals with dubious agendas”. On the 4th January hundreds of Zambians thronged the Cathedral of the Holy Cross to press for the immediate release of the final draft constitution.
Links to further information on social contracts:
A summary of the approach used to secure the successful Peruvian social contract known as the ‘5 by 5 by 5’ Child Malnutrition Initiative can be seen in box 2, page 12, Addressing Malnutrition Multi-sectorally, 2013, United Nations.
A Citizens’ Charter (sometimes called a Clients Charter or a Patient’s Charter) is a document that states the commitment of an organisation or entity towards specific standards, quality and time frames of service delivery. The Citizen’s Charter enables the service recipients to know their rights to a service with clear conditions – and makes the service providers aware of their duties to attend to the problems of the citizens within a reasonable time-frame. Thus, the dissemination of information about the Charter’s contents is key to its success since this is what raises awareness on rights and deepens the service deliver’s sense of responsibility & accountability.
A Citizen’s Charter usually consists of the following elements:
- Details of the specific services offered by the department, the quality standards and conditions of the service
- Standard charges for the services, and how those payments should be made
- Location and hours of operation of offices connected with the delivery of services
- The names and contact details of the officers responsible for delivery of the services
- The time required for the delivery of each service
- The authority to whom complaints can be made if the services offered are delayed or denied
- A ‘compensation clause’ where the standard of service is not met
Some challenges and shortcomings to avoid when developing Citizens’ Charters:
- Lack of awareness and knowledge and adequate publicity are particular problems for Citizens’ Charters which may lead to a loss of trust among service recipients. The service recipients need to have clear access to the details of their rights which should be clearly and displayed in the establishment offering the services. For example, a Citizens’ Charter on how the police should interact with victims of gender-based violence should be on a poster next to the front desk of the police station.
- Service delivery staff may not be prepared to shoulder the responsibility due to lack of motivation and accountability. This can be overcome by providing training to the staff and line managers, sensitizing them to the new approach. Ideally, involving the entire team in the development of the Citizens’ Charter will help them feel a sense of ownership of the document and commitment to deliver to it.
- The service delivery supervisory staff need to be fully aware of any existing barriers to front-line staff being able to deliver upon the agreements in the Citizen’s Charter. Where problems of implementation are experienced, the supervisory staff need to be both supportive of their own staff in overcoming the challenges, as well as to make every effort to ensure the service recipient is provided with adequate service. It is important that the service provider sees it as a team responsibility to deliver to the Charter.
- Citizens’ Charters need to be reviewed periodically to make sure they are still appropriate. Not revision, overly complicated and rules and procedures which are not possible to comply with are particularly destructive to a Citizens’ Charter.
Links to further information on Citizens’ Charters:
The Citizen Report Card (CRC) is a survey instrument that is filled in by citizens using numbers to show how satisfied they are with public services such as health centres, schools, or utilities. Using the information they receive from a citizen report card, CSOs, governments and service providers can better understand how citizens experience the services they are providing. The feedback is then analyzed and the data is used to start a dialogue between communities, CSOs and local government with service providers on how to improve services. The process is more powerful if it is accompanied by media coverage and civil society advocacy.
How to develop and use citizen report cards:
The World Bank identifies 6 stages of the CRC process:
Ask yourself: What is it that we want to achieve with this study? What is the PURPOSE?
- What service(s) do you want to assess or evaluate?
- Is there a government policy or programme you wish to assess?
- Who can give you the information you need? Are there sub-groups in the population that are of particular interest to your study (poor households, children, women, elderly people, people with disabilities etc.)?
- Do you want to focus on a single service provider or multiple services?
- Across what geographical area?
- How will the information be used? (Be very specific with your answer to this question since you do not want to collect any information you are not going to use – only collect information that is going to enable you to answer the questions you have)
- Will it improve current services or programmes? Will it be used to consider developing new services or programmes?
- Who will see the results of the study? Who will collaborate with you in doing the study?
It is a good idea to include the service providers themselves in the design and undertaking of the study. This will help them to be more open to receiving the information you create through the research and acting on it. CRCs can sometimes lead to the service providers being publically embarrassed if the results show areas of weakness. However careful design of the process and development of good relationships with the service provider before you start the surveys will help to reduce tensions once the results come out. It will also help the service provider to work with citizens into the longer-term to bring about any changes that are identified as needing to take place.
The second essential step is to find out how services are delivered to the population you are interested in. This will show you who the different actors are in the process of service delivery – and therefore the various stages and places where things can go wrong. It will also help you identify who to consult and who to share your results with at the end.
- Is the central, regional or local government the main service provider?
- Are services provided by a combination of providers from various levels of the government?
- Has some part of service delivery been contracted out to a private company or a CSO or CBO?
- Through which parts of government does the funding for this service delivery pass before it reaches the actual service deliverer?
- Are there existing standards for service delivery set by government? What are they? And who monitors them?
Getting the right questions for the questionnaire is essential. One way to do this is to hold two focus group discussions: one with a cross-section of the affected target population of service users; and one with service providers. Then have a close look at government documents related to the standard of service delivery to find out if there are indicators, targets or performance standards already set for this service and find out how does government normally monitor and evaluate it.
In the focus group discussion with service users, ask them:
- What are the problem areas related to this service?
- Are there any recent areas of improvement?
- What aspects of the service delivery are important to you and why are they important? (For example: availability, access for different groups, quality of service, incidence and resolution of problems, interaction with staff, corruption?)
In asking these questions you are not looking for solutions at this stage – but instead to map the overall issue so you can decide which questions to ask the wider public and which data and facts to gather.
It is also important to understand from the service provider’s point of view which issues are of interest to them and which issues they feel could realistically be acted on. It is quite likely they will know of underlying causes and challenges that need to be addressed and they can add valuable insights. Different levels of management and different types of staff should participate in a focus group discussion. For example, for the education service, you might need to ask (local, provincial and/or national) educational authorities, management staff, parents teacher associations, teachers, cleaners and caretakers, etc.
For the service providers, always start with positive questions so that the service providers do not feel you are only there to blame them for problems.
- What aspects of this service work well?
- What are the problem areas related to this service?
- Are there any recent areas of improvement?
- What aspects of service delivery do you think still need further improvement?
- What aspects of service delivery do you think could be realistically improved?
In the government document review, look to see if there are any existing government commitments related to the service you are studying? Have a look at the strategic plan and budgets, any public statements made by government and so on. What are the norms for service delivery (standard indicators, standard equipment, levels of training required of staff, etc.)? Where there are existing service standards one area of questioning for your survey could be to check the extent to which those service standards are being met currently. Government may even be gathering data on this already so it is worth finding that out from the service deliverers before you start your survey.
Are there any particular areas the government is trying to improve upon (e.g. numbers of teachers per school, or classroom size; access to medicines, % of births at a health facility, etc.)? Even if the government documents prioritise issues that citizens have not prioritised, you may still decide to include some of these services in the CRC because the government may then be more likely to respond to the findings of the CRC if it includes information about services which are on the government’s agenda.
On the basis of the two focus group discussions and the examination of the government documents you will be able to identify the key issues around which to build the CRC questionnaire. Then pre-test the questionnaire and improve on the questionnaire as you try it out.
Stage 3: Sampling
Who and how many do you want to ask? Determine the sample size for your CRC. Can you do it with the resources you have at hand? Ensure that the respondents vary in terms of age, gender, vulnerability and income group. Be sure to disaggregate your data by category so you understand the different perspectives of each group. Have a look at the ZGF toolkit on Building the Evidence to Inform Policy: A Guide to Policy-relevant Research and Additional Resources for more details on sampling.
Stage 4: Execution of Survey
Who will conduct the survey? The people doing the interviewing/surveying need to be polite and patient. They should introduce the work in an open and neutral way, for example, saying that they are there to understand better the quality of service provision and to gather ideas on how to improve it further. Do not say the purpose is to find out about the bad services they have been getting! They should fill out the survey form in the room – gathering the details on the exact type of person who is answering the questions, and capturing their answers as thoroughly as they can. Do not leave the write-up until later on. Be very careful about following appropriate cultural practices and protocol when interviewing people – if a community leader or the head of a household must provide their permission for someone to be interviewed then please get that permission in advance – or there could be serious consequences for the person you are interviewing. You will also need to politely explain that the leader or head of household cannot be present when the interview is taking place since the feedback has to be provided confidentially and anonymously.
Stage 5: Data Analysis
Aggregate the collected data and analyze the user satisfaction. ZGF‘s toolkit on Building the Evidence to Inform Policy: A Guide to Policy-relevant Research and Additional Resources can provide more details on data analysis.
Stage 6: Dissemination
The results of the survey should be first shared with the service provider and it should be written in a way that shows what is working well and what can be improved upon – rather than a long list of problems and criticisms. The results should be made available to the public and media – ideally in the presence of and with the participation of the service provider. The service provider and the users should be brought together to engage in a dialogue. Users should have the chance to voice their satisfaction/dissatisfaction with the service – and the service provider should have the opportunity to respond. Actions ideally should be agreed between all present to work towards improving service delivery standards into the future – providing a role both for service providers and service users in the process so that a continual cycle of improvement can be set up.
ZGF‘s toolkit on Building the Evidence to Inform Policy: A Guide to Policy-relevant Research and Additional Resources can provide more details on communicating results of your research.
As the Asian Development Bank and Asian Development Bank Institute put it in their Citizen Report Card Learning Tool Kit:
“The CRC methodology should not be seen as a social science survey that ends with a written report; findings need to be publicly distributed and followed up!”
Links to further information on citizen report cards:
Asian Development Bank and Asian Development Bank Institute: Citizen Report Card Learning Tool Kit, 2007, http://www.citizenreportcard.com/crc/pdf/manual.pdf
World Bank: Citizen Report Card Surveys-A Note on the Concept and Methodology, Note No. 91 February 2004: http://siteresources.worldbank.org/INTPCENG/1143380-1116506267488/20511066/reportcardnote.pdf
UNICEF, Citizen Report Card Manual, A social audit tool to monitor the progress of Viet Nam’s Socio-Economic Development Plan ttp://www.unicef.org/vietnam/citizen_TA.pdf
The Community Score Card is a participatory monitoring tool at community level. It allows community members to assess the performance of service providers, and to provide this feedback directly to the service providers to demand improved service delivery. Compared to the Citizen Report Card, the Community Score Card gives immediate feedback from the community rather than from individual citizens, in this way it is a good way to prepare for joint decision making. In the implementation process it can also be used to inform community members about available services and their entitlements.
How to develop and use Community Score Cards (CSCs):
The CSC process consists of the following phases:
Phase I: Planning and Preparation
- Identify which facility e.g. a health post, road, ablution block ) or service (e.g. quality of education, police service, public transport) is to be assessed
- Identify people who can convene and help to facilitate the CSC process (e.g. local leaders, teachers, even the local government officials)
- Identify the local leaders that you should inform of your plans
- Carry out preliminary research regarding current inputs, entitlements, degree of usage etc.
Phase II: Creating and Filling in the Score Card with the Community
- Inform community members about the CSC process
- Support the community to develop indicators to help them assess the facility/service (e.g. for an educational service this might include: attendance of staff, number of pupils per class, quality of teaching, average distance travelled to the school and so on)
- Support the community to identify a range of scores for them for the evaluation with a sub-group of the community themselves
- Bring the community together in small focus groups and work through each indicator with the participants allowing them to allocate scores, providing reasons why they have scored an indicator at each level
- Generate suggestions for improvement
Phase III: Creating and Filling in a self-assessment Score Card with Service Providers
- Convene a group of the service providers and support them in identifying indicators and a range of scores for each indicator to self-assess their performance. If they already have their own indicators then you can use those.
- Facilitate the service providers to allocate scores against each indicator, providing an explanation as to the score given.
- Identify priority issues for working on and generate suggestions for improvement
Phase IV: Interface Meeting and Action Planning
- Bring the community and service providers together so that they can share each of their findings from the CSC process. The particular focus should be on the priority issues identified and their ideas for moving forward.
- Re-prioritize the issues together and agree an action plan together
Phase V: Action Plan Implementation and Monitoring and Evaluation (M&E)
- Execute action plan
- Monitor and evaluate actions
- Repeat cycles to ensure institutionalization
Examples of Zambian application of Community Score Cards:
The Zambian Network of Youth Organisations (ZANEYO) formerly known as FYOZ used Community Score Cards (CSCs) and Citizen’s Report Cards (CRCs), together with the support of 420 community volunteers, to evaluate the use of Constituency Development Funds in 6 wards within two districts. The funds has been spent on projects such as: the construction of a Police Post, clinic, ablution blocks, a market shelter and maternity ward; the gravelling and installation of drainage for roads; and the renovation of a clinic. They used the results to profile the key shortcomings of projects in the media and with the local MPs and immediately many of the identified issues were corrected. http://zaneyoplatform.wordpress.com/projects-and-programmes/archives/social-accountability-project/
The Zambian Ministry of Health in 2012 participated in a process of social auditing of its health services. In its findings it set itself a broad array of follow up recommendations, including making maternal deaths an issue that must be reported on, improving use of ICTs to encompass eHealth, and the need to institutionalise Service Readiness Availability Mapping. The full report can be downloaded from http://www.internationalhealthpartnership.net/fileadmin/uploads/ihp/Documents/Country_Pages/Zambia/Zambia_Scorecard_and_Roadmap%5B1%5D.pdf
Links to further information on Community Score Cards
Care: The Community Score Card (CSC): A generic guide for implementing CARE’s CSC process to improve quality of services. Toolkit, 2013, http://www.care.org/sites/default/files/documents/FP-2013-CARE_CommunityScoreCardToolkit.pdf
UNICEF and Ministry of Planning and Investment: Community Scorecard Manual. A Social Audit Tool to monitor the progress of Vietnam’s socio-economic development plan http://www.unicef.org/vietnam/Community_TA.pdf
Public Expenditure Tracking Surveys (PETS) is a survey to get information on the flow of public money for the provision of public goods, to identify if the money actually does reach the service providers (clinics, schools etc.) and if funds are used as intended. The objective of PETS is to improve the efficiency of public expenditures and quality of services. Expenditure tracking can be undertaken at the local, district or sub-national level. In contrast to Audits, PETS do not try to find missing resources or identify the persons responsible – the focus is instead on identifying whether or not there is an overall problem with the flow of money. This information can then be used to persuade government to investigate further the issue if a problem is identified.
How to develop and use PETS?
Public Expenditure Tracking Surveys (PETS) are increasingly used to understand the flows of money from the national through to the provincial and district level. At district level it shows how money passes from local government to service delivery agents such as the police or health posts. It is often implemented alongside a Citizens Report Card or a Community Score Card which will assess the quality of the service provided. The PETS traces the flow of resources through the different bureaucratic layers, measuring how much of the originally allocated resources reach each level and how long they take to get there. The surveys therefore assess the leakage of public funds and can help to assess the efficiency of public spending and the quality and quantity of services. It also helps to analyse the causes underlying problems, so that informed policies can be developed.
The results of PETS are particularly useful if they are undertaken in parallel to a Citizens Report Card or Community Scorecard. This is what Civil Society for Poverty Reduction (CSPR) undertakes in its application bi-annually of budget tracking and service delivery tools to develop its Barometer (http://www.csprzambia.org/index.php/programs/research-and-policy-analysis/barometer). Link to case study below.
PETS results are said to be useful in campaigning to improve transparency and accountability by supporting information access. However, a particular challenge reported in Tanzania with PETS is that it is difficult to get hold of the data from local government, as the SNV report said “Even information that local governments are legally obliged to make public is kept hidden, usually in an apologizing manner (‘not in my mandate’, ‘talk to my superior’ and ‘the District Executive Director has to decide’). From the side of the NGOs there is a number of factors that prevent them from accessing data. They generally lack awareness of their right to access information”. Whilst this is to some extent also the case within Zambia, CSPR has at least to some extent been able to overcome these challenges in its application of PETS. Link here to the case study.
The PETS process consists of the following phases:
Phase 1: Determine the scope and purpose of the tracking exercise
- Define the purpose and the objectives of the survey: What are you trying to find out with the help of PETS?
- Determine the scope of the PETS: Do you want to assess a specific service provider, an entire sector, a programme etc.
Phase 2: Identify partners and key stakeholders
- Identify partners to support you with the tracking of public funds from the national to the local level. This will need to include good contacts within the different departments where the data is housed. It is highly unlikely that the information you will want to access will be made available to you without you having strong political contacts inside the relevant departure, or up the hierarchy. When CSPR set up their budget tracking activities they secured permission from the Ministry of Finance, via a letter from the Secretary to the Cabinet, so that lower levels of government felt confident in releasing data to them.
- Involve your target audience e.g. different layers of government, citizens, service providers etc. in the survey – the more you engage them the more likely they are to feel ownership of the results and take action upon them.
According to SNV, in most surveys the elected leaders easily became allies in a Public Expenditure Tracking Survey and welcomed the results and process – it made them better informed on what was happening in their local area and gave them the ability to bring about positive changes, sometimes very quickly.
Phase 3: Design the research/survey
- Review the availability of data – it is highly likely that many layers of the data will be difficult to access and will take many attempts to get hold of it, even using all the different political contacts you have! Be sure to know how you will encourage duty bearers to provide the data to you before you embark on the process.
- Outline hypotheses – where in the public expenditure process (at which layer) do you think different parts of the allocated funding might be spent differently from how it was originally intended? What might be the reasons for this?
- Create the survey tools – and be sure to warm up your contacts at each point. Ask others who have undertaken such processes in Zambia (such as SNV and CSPR) what steps they took to be able to get access to different types of data.
Phase 4: Gather and analyze the data
- Implement the survey and gather data regarding different aspects with the support of partners and stakeholders. Make sure you have allocated sufficient time to this process since it will take a lot of time and effort to get hold of each piece of data. The local level service deliverers will certainly need permission from higher up their organisational hierarchy to be allowed to release budget data to you.
- Analyse the collected data. What story is it telling you? Does it make sense to you? Does it raise more questions – if so, be sure to follow those through to understand them also.
Phase 5: Public dissemination of findings and advocacy
- Share and discuss the results with citizens and public authorities
- Engage the media
According to DFID, the more comprehensive the assessment of the flow of resources, the greater its impact will be. This is particularly the case where the PET is strongly linked with an assessment of service delivery. However, DIFD believe they will not have any effect without “a clear commitment from governments to:
- Disseminate the results widely;
- Engage all levels of government in changing the way in which sector policies are developed and resources are managed;
- Remain committed to transparency over the allocation and use of resources”.
Phase 6: Explore possibilities for institutionalization
Examples of Zambian application of PETS:
SNV (Zambia) trained Kasama Christian Community Care (KCCC), a local capacity builder, on Resource Tracking and Community Scorecards so that they in turn can train communities on their use and can then generate evidence on the resources reaching an educational institution and the quality of education being delivered. Such approaches do work – for instance, in Nakonde District, after the PTA was equipped with these skills, the District Education Board responded quickly to community demands by posting 5 teachers to two different schools that had been without teachers for more than 6 months.
Civil Society for Poverty Reduction (CSPR)
Links to further information on Public Expenditure Tracking Survey
The World Bank: Public Expenditure Tracking Surveys (PETS) and Quantitative Service Delivery Survey (QSDS) Guidebook, 2012, http://pets.prognoz.com/prod/Guidelines.aspx
SNV, Public Expenditure Tracking in Tanzania at district level: Effects on Local Accountability,
PG Exchange: Public Expenditure Reporting and Tracking, http://www.pgexchange.org/index.php?option=com_content&view=article&id=161:public-expenditure-reporting-and-tracking&catid=58:public-budgets-a-expenditures&Itemid=156
Zambia – Public Expenditure Tracking Surveys in Health, 2007, World Bank, Ministry of Health and Ministry of Finance, http://catalog.ihsn.org/index.php/catalog/1049
Public Expenditure Tracking Surveys in Education, UNESCO, 2004, Ritva Reinikka and Nathanael Smith, http://www.unesco.org/iiep/PDF/pubs/Reinikka.pdf
UNICEF, Public Expenditure Tracking Survey Manual, A Social Audit Tool to monitor the progress of Vietnam’s socio-economic development plan, http://www.unicef.org/vietnam/PET_MANUAL_TA.pdf
Social audits are a process that collects information on the resources of an organization. The information is analyzed and shared publicly in a participatory fashion. Although the term “Audit” is used, social auditing does not only look at costs and finance – the central concern of a social audit is how well resources are used to achieve social objectives. Social audits can be used to assess the work of many government departments over a number of years in several districts; or they can be used to manage a particular project in one village at a given time. Most social audits will involve the following: producing information that is evidence-based, accurate and impartial that can be valuable to identify corruption or incompetence; creating awareness among beneficiaries and providers of local services; improving citizens’ access to information on government; permitting stakeholders to influence the behavior of the government; and monitoring progress and help to prevent fraud by deterrence.
A social audit checks how programmes and services are being carried out in order to improve them and focus them more to social, environmental, and community priorities. It involves an evaluation of public records and user feedback (using tools such as Citizen Report Cards and Community Score Cards) so that users can understand and assess the programme or service and propose improvements. Social audit practices depend on the program or service under review and so can use various techniques and methodologies. The following are broadly the key phases involved:
Phase 1: Preparations
- Decide what you want to achieve through the social audit – and through this what you need to investigate (e.g. which department, a programme, a specific service or project, a sub-component or activity). See Gender Social Audit Tool and Child Rights Social Audit Tool.
- Bring together a group of people who will implement and oversee the social audit.
- Identify stakeholders (users of the service, community members, local CSOs, service providers, duty bearers, employees etc.).
- Develop a good understanding of the relevant administrative structures and responsible agencies/actors, and the vision and objectives of the service/project in question;
- Create performance indicators through stakeholder consultation
- Raise public awareness about the aims and benefits of the social audit
Phase 2. Information gathering and analysis
- Review relevant public documents (such as accounting records, cash books, technical project reports etc). You will need to see the original documents rather than second-hand reports which may not be accurate.
- Gather data from relevant stakeholders about their perceptions and experiences of the service/project in question
- Analyse collected data.
Phase 3. Public disclosure and dialogue
- Communicate the results widely.
- Meet with community members to discuss findings and develop proposals for improvements.
- Hold public meeting(s) to allow community members to discuss the evidence with authorities and service providers, and to plan future improvements.
Phase 4. Next phase
- Work to have social audits institutionalised within governance processes or at least for them to be repeated regularly; and train up future volunteers.
An Example of a Zambian Application of a Social Audit:
Nyimba District Farmers Association (DFA) focuses on creating awareness on rights and entitlements of citizens (particularly peasant farmers) to enable them to hold local government accountable for its obligations. The DFA has also facilitated the formation and/or strengthening of structures for popular participation and influencing of duty bearers to deliver quality services. In the recent part, much of Nyimba DFA’s work has revolved around enabling farmer groups in communities to demand accountability on Constituency Development Funds (CDF) in the district. In working towards its goal, Nyimba DFA has established relationships with the District Council, District Commissioner, chiefs and headmen, and the district CSO forum. All these stakeholders meet in the DDCC to discuss pertinent issues that affect the district…additionally, the CSO forum meets on a quarterly basis as a platform for sharing experiences and lessons among CSOs working in the district.
In terms of specific tools for demanding accountability, Nyimba DFA uses the social audits methodology. Social audits start with training of social auditors on how to go about the process. Their social audit process is based on seven principles: Multi-perspective, Comprehensive, Participatory, Multi-directional, Regular, Verification and Disclosure. The social audit is done by community members, who are volunteers, with the leadership of the Area Development Committees. The social audit process includes defining audit boundaries, social accounting and book keeping, preparing and using social accounts, undertaking social audit and dissemination/feedback. Information obtained from the audits is used to lobby for improved implementation of CDF projects in the district, and is shared with the local authority and heads of government departments in Nyimba.
Nyimba DFA has seen some positive results: in Mtilizi Ward, the ADC is now involved in the implementation of a nutrition programme under the Ministry of Health, demonstrating that duty bearers now view ADCs as structures with whom they can actively collaborate; in Mchenga Ward, the ADC was requested by the Ministry of Agriculture to be actively involved in the distribution of relief maize to communities in the area; and a higher-level ADC platform has been created for ADCs to bring their views together and engage with duty bearers at district level.
Links to further information on Social Audits:
The CDF Social Audit Guide, a Handbook for Communities, 2008, Open Society Initiative for East Africa, http://www.opensocietyfoundations.org/publications/cdf-social-audit-guide-handbook-communities
Social Audit: A Toolkit, A Guide for Performance Improvement and Outcome Measurement, 2005, Centre for Good Governance, http://unpan1.un.org/intradoc/groups/public/documents/cgg/unpan023752.pdf
Social Audits as a Budget Monitoring Tool, 2012, International Budget Partnership, http://internationalbudget.org/wp-content/uploads/Social-Audits-as-a-Budget-Monitoring-Tool.pdf
Social Audits, Participatory Governance Exchange, Civicus, http://www.pgexchange.org/index.php?option=com_content&view=article&id=142&Itemid=136
This tool takes the overall approach to undertaking social audits and focusing it in on the issue of child rights so that you can examine government policy and practice to understand the extent to which children’s rights have been taken into consideration across all areas of policy and practice. Areas it looks at include: Policy Templates for Child Poverty and Disparities; Integrating a Child Focus into Poverty and Social Impact Analysis (PSIA); Mainstreaming Child Rights in Poverty Reduction Strategies & National Development Plans; Diagnosing Child Friendly Social Protection; Child Friendly Schools Quality Assessment Criteria: Manual; Child Protection System Mapping and Assessment Toolkit; and so on.
Links to information on Child Rights Social Audit
UNICEF, Child Rights-Based Social Audit Manual: Detailed Methodological Description, including Report, Guidelines and Feedback Mechanisms; A social audit tool to monitor the progress of Viet Nam’s Social and Economic Development Plan, http://www.unicef.org/vietnam/child_Rights_TA.pdf this document also provides extensive links to other tools, toolkits and manuals on the area of children’s rights and government policy and practice.
This tool takes the overall approach to undertaking social audits and enables you to analyse a particular programme or project to understand its impact on gender. For example, it looks at: the extent to which a plan or a policy has incorporated gender issues as a focus; the degree to which these policies and plans then translate into measurable action on gender issues (including the scope and quality of indicators that measure progress in specific gender priorities); and an assessment of the gender impact of plans and policies, including through participatory means.
Examples of Zambian applications of Gender Social Audits:
In Zambia, the main organisation that has been undertaking a gender audit of the national budget each year has been the Non-Governmental Organisation Coordinating Council (NGOCC). Copies of these can be found on the NGOCC website, for example the NGOCC 2012 National Budget Analysis from a gender perspective, http://www.ngocc.org.zm/index.php/publications/ngocc-publications
Links to further information on Gender Social Audit:
UNICEF, Gender audit manual: A social audit tool to monitor the progress of Viet Nam’s Socio-Economic Development Plan, http://www.unicef.org/vietnam/GENDER_TA.pdf