<p>Abstract<br />This paper employs “Cultural orientation framework” to explain the cultural difference across the school and home of four girls with foreign culture. Mistrust emerged between the school and the parents when outdoor school activity had been discriminated from the indoor and the girls used to be absent on the excursion days. The teachers’ immediate solution was to question the validity of the illness the girls submitted after the excursion. Additionally, the teachers became frustrated with not being communicated openly if there was a standpoint. Neither the schedule had not been adhered nor the teachers were informed in a good time the girls’ absence from the excursion.</p>
Read it<p>people who design evaluations is expected to select an approach that allows the organization to learn from the interventions’ success and failure. In other words, a good evaluation system should start at early stage when a new intervention is being designed, keeping in mind that the evaluation system and the program design are closely connected.</p> <p>Author Ibrahim H. Saed</p>
Read it<p>It has become visible that one disaster may completely damage the fruits of decades of development activities. The impacts of natural disaster attracted the attention of development practitioners. Because it is not only that it victimizes severely vulnerable groups in a society such as women, children and elder and poor people, but it disrupts development efforts and investments</p> <p>Author: Ibrahim. H Saed</p>
Read itpeople who design evaluations is expected to select an approach that allows the organization to learn from the interventions’ success and failure. In other words, a good evaluation system should start at early stage when a new intervention is being designed, keeping in mind that the evaluation system and the program design are closely connected.
Author Ibrahim H. Saed
A theory-based evaluation focuses both on learning and accountability while it promotes the evaluation culture within the organizations. The continuous learning may help organizations to close the gap in the internal processes for managing the evaluations. Generally, the approach is less expensive in which organizations may go beyond a donor driven assessment that emphasizes on the accountability as learning receives less attention. The best practice in the evaluations entail embracing the culture of evaluation with or without donors’ funds, in which the cornerstone is a continuous learning, adequately explaining the intervention’s success and failure.
It is, therefore, critical for people who design evaluations to select an approach that allows the organization to learn from the interventions’ success and failure. In other words, a good evaluation system should start at early stage when a new intervention is being designed, keeping in mind that the evaluation system and the program design are closely connected. This entails designing a good system that captures adequate data from key elements that help the organizations to learn from the interventions. It is true that organizations need adequate information that can be used to describe and prescribe the actions, context and changes that address social problems as well activities that maximize the success of the intervention.
The traditional approaches generate information about the success or failure of a program using the dimension of DAC factors (the relevance, effectiveness, efficiency, impact and sustainability). This paper aims to shed a light on how theory of change model can be a good choice when organizations intend generating information that goes beyond explaining the relation between the result chain (input, output, outcome and impact). This means that the organizations can collect more valuable information with the theory of change to explain mechanism that make an intervention work, underlying assumptions and factors that contribute to the changes.
Most organizations struggle to strike a balance between adequately explaining causes and effects of an intervention and limited resources. If the objective of the evaluation is to explicate the relations between the intervention and the observed results, the organizations need to collate comprehensive data and credible information. It became clear that the traditional methods that concentrate its attentions on the intervention’s program logic which produces insufficient information to explain causes of observable changes in a target group. The evaluations systems that are built on the experimental design can be a good choice to generate credible and transparent estimates of program’s impact. This approach assigns meaningfully and randomly observational units of a study to different treatment and control groups. However, the resource prerequisite for carrying out the evaluations that employ the experimental method makes it less practical methodological choice that fit for organizations’ circumstance.
The theory of change model can alternatively be a good choice to assess the intervention’s causal relations between the intervention and the observed results. Unlike other traditional approaches, the theory of change methods is founded on beliefs that only program logic (input-output-outcome) do not make an intervention work, but underlying mechanism that are either explicitly or implicitly articulated, and concepts and theories underpinned with these mechanisms. This involves that the evaluations’ design must be incorporated with the theories to examine implicitly or explicitly program’s theories and test them.
It is, however, a good idea to agree first what a “theory” means. A “theory” represents stories that people tell regarding how problems arise and how can those be solved (Weiss, 1995). In addition, “theory-based evaluation” is defined as “an approach in which an attention is paid to theories of policy makers, program manager and other stakeholders” (Leeuw, 2003). “The aim is to examine the extent to which program theories hold. The theory-based evaluations should show which of the underlying assumptions of the program broke down, where they broke down, and which of underlying theories of the program the evidence best supports” (Weiss,1995). The theory of change models are made of two vital components; a conceptual framework which articulates a policy or program theory and an empirical part which seeks to test this theory, to investigate whether, why or how policies or programs cause intended or observed outcomes (Leeuw. Undated).
The question worth asking is, why organizations need to employ a theory of change in the evaluations? The literature reviewed indicates experimental designs require sufficient resources to develop evidences and to articulate causal relations of an intervention. The fact is resource is often limited, so the choice of experimental design is out of scope. Secondly, other conventional approaches fail to provide detailed information concerning the relation between the intervention and the observed results. Literature reviewed show that a traditional single point or fixed interval longitudinal assessments of outcome has not effectively captured the change in totality. Therefore, these limitations give the organization a good reason to apply a theory-based approach in the evaluations.
The essence is a theory-based approach challenges conventional evaluation methods that attribute success of intervention to result chain (input-output-outcome). Consequently, the evaluations assess only what is easy to measure rather than what is significant. This simplicity discounts the relation between the context and intervention, i.e. was it about the intervention or the context that caused the results? (Brandon et al, 2014,). A theory-based approach investigates not only intended outcome, but it confirms or disconfirms hypotheses on which an initiative was constructed. It questions other factors influencing the observed results.
Furthermore, organizations, which choose applying a theory-based approach in evaluations, can logically gain the advantages which Weis (1995) underlined. First, resources and efforts can effectively be deployed by narrowing the focus of this evaluation on key aspects of program. “No evaluation, however well-funded, can address every question that might be of interest to someone” Due to limited resources, it was suggested that central hypotheses about the program appear to represent potential issues that evaluation should address (Weiss, 1995). Particularly, when there exists various hypotheses and a true diversity of stakeholder’s theory, a development of priorities is also needed for inclusion or discrimination of hypotheses and how it is included. Brandon (2014) and his associates underlined limiting their focus on institutional and sector level impact as a benefit gained from applying theory of change in evaluating career development.
Secondly, organizations are able to generate knowledge about key theories of change. “It would be very useful to direct new evaluations toward studying these theoretical hypotheses, so that knowledge accrues more directly on these key matters. This is the reason that the focus of traditional evaluation was reversed by employing the theory of change in evaluations. Our effort is directed at examining underlying assumptions on which different parts of the intervention are based. The traditional approaches, which concentrated much of the energy at investigating result chain and process, are often criticized in shortcomings of producing sufficient knowledge.
Thirdly, the advantage of the theory of change is organizations make assumptions explicit, defining methods, and clarifying goals which help a theory-based evaluations to “elicit formulations of program theory from those engaged in the initiatives; they may begin to see some of the leaps of faith embedded in it”. When assumptions and theories are made explicit, it will increase a likelihood of colleagues to reach consensus about what they are trying to do, and why they are doing it. The process of explicating theories and hypotheses in controversial issues proves to be difficult to determine which one’s theory is included or discriminated. Some of these challenges can be observed in situation where some stakeholders are unable to theorize properly assumptions and changes, they expect from the intervention.
Final advantage is organizations are able to influence policy making. It is generally claimed that evaluations that address the theoretical assumptions embedded in programs may have more influence on elite’s and popular opinion. “As evaluative evidence piles up confirming or disconfirming such theories, it can influence the way people think about issues, what they see as problematic, and where they choose to place their bets. The climate of opinion can veer and wiser policies and programs become possible” (Weiss, 1995).
The line of reasoning pursued in this paper is that the conventional evaluation methods have been criticized for concentrating their efforts on explaining the program logic or the result chain, which is easy to assess, but not necessarily sufficient. Our argument is the organizations need to reconsider when choosing the approach that is used to exclusively evaluate what is commonly referred as “program logic”. This “program logic “approach is a good choice when the intervention’s result occurs in a linear manner, a sequence of events and results (outputs, immediate outcomes, intermediate outcomes and ultimate outcomes). However, many agree that the development intervention is getting complex and many other factors including the context and other factors influence the intervention’s success or failure. Therefore, it is much needed an approach that goes beyond the result chain, which explains mechanisms of change as well as the assumptions, risks and context that support or hinder the intervention’s success or failure. The use of “theory of change “approaches help organizations to draw a conclusion about whether the intervention had and how it had contributed to observed results.
We are not arguing that the theory of change not having a limitation. The paper acknowledges the approach’s limitation such as difficulties involved in determining which one’s theory is included or discriminated when program’s theory is being evaluated. As Weiss (1995) stated, “If we cannot spell out fine-grained theories of change that would apply generally, we can attempt to identify certain implicit basic assumptions and hypotheses that underlie the larger endeavor.”
Our conclusion is the organizations need to select carefully the evaluation’s design to be able to use efficiently resources and prioritize the important theories and specific aspects of the program to be assessed. Our conclusion calls when the “theory of change” approach is being chosen, its limitation should be taken into account, particularly when the learning and knowledge is to be generalized. However, the good news is that theory-based approaches can be used mostly in a combination with other approaches and data collection techniques. Such methods for data collection and analysis include qualitative and quantitative methods to test and refine program’s theory.
Reference
Weis. H. Carol 1995. Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families
Chen, H. T 2012. Theory-driven evaluation: Conceptual framework, application and advancement
Brandon et al, 2014. Monitoring and Evaluation of African Women in Agricultural Research and Development (AWARD): An Exemplar of Managing for Impact in Development Evaluation
Leeuw, Frans L. (2003). Reconstructing program theories: methods available and problems
to be solved. American Journal of Evaluation, 24 (1): 5‐20.
Posted: 2022-04-20 13:49:28 Back home
IHS offers three value propositions: Innovation, Performance, Get specific jobs done. IHS will place a high priority on innovation which sophisticated research will guide mapping clients' innovations to fit for the emerging technologies, innovating products/services and processes. IHS will help the clients to demonstrate a strong performance that produce tangible results for end users. IHS will help the clients to get certain jobs done.