Impact evaluation in small and medium-sized organizations. Is it possible? – this was the topic of the round table organized by the Jerzy Regulski Foundation in Support of Local Democracy as part of the Youth Impact project.
The webinar took place on July 5, 2022, and allowed to present a number of examples of the use of counterfactual methods to measure the actual impact of the project on its recipients. Dr. Tomasz Gajderowicz showed, among others, how the randomization of the selection of participants in vocational activation projects implemented by Labour Offices or by external entities in combination with data on employment status before and after the intervention proved much greater effectiveness and durability of activation of the long-term unemployed carried out by external entities. Interestingly, the relevant ministry, on the basis of another, poorly performed evaluation study giving opposite conclusions, decided to stop outsourcing vocational activation.
Equally interesting conclusions can be drawn from activation projects, the effectiveness of which turned out to be low. The results of the evaluation of the impact of such projects were presented by Rafał Trzciński from the IDEA of Development Foundation and the Polish Evaluation Society, using data on the employment of project recipients and the control group selected by the PSM method.
Regarding small projects, the panellists proposed, inter alia, comparing the effects achieved by participants from different editions of the same project or different projects from the same program, ensuring that the composition of the compared groups is similar by random selection(it is worth using stratified drawing) or forming the compared groups using PSM, possibly by using the DiD or RDD models. It was also recommended to verify with facts to what extent the effects described in the project theory occur in the prescribed manner and to use qualitative methods for the exploration of processes that are to lead to the achievement of the project’s goal.
The moderator of the meeting – Dr. Sławomir Nałęcz from FSLD – expressed the hope that the subject of impact evaluation in small projects will be continued, e.g. at the webinars of the Polish Evaluation Society, which was a partner of this round table.
We invite you to watch the recording of the event on the FRDL channel on the YouTube platform at https://youtu.be/-yzPrC34zgo
– by FRDL
