The Eurasian Century, by Hal Brands

 

This is a fact-filled book written in a highly readable and lively style. I found it very satisfying on several fronts, although I can sympathize with some of the reviewers who disagree with Hal Brands’ point of view. He makes no effort to hide where he is coming from, seeing the democratic West as something worth defending, and the USA as the necessary guarantor to the free world.

 Brands draws upon the geopolitical theories and traditions of Halford Mackinder, Alfred Thayer Mahan, Nicholas Spykman, and others. This is one of the attractions of the book. As a trained geographer myself, I’ve always thought that geopolitics was unfairly criticized for decades. Sure, it was originally associated with the Swede Rudolf Kjellén and the German Friedich Ratzel both of whom were something of social Darwinists and whose ideas were embraced by the Nazis. Brand’s view is nuanced and his application of the perspective is intelligent. He sums it up: “Geopolitics tempered by democracy was stark but rarely evil. Geopolitics with an autocratic bent was poison, pure and simple” (p. 31). Bad application of an approach should not be a reason to ban the entire discipline (for example, genetics has been misused for bad purposes but it nevertheless is a vital science).

 The first two-thirds of the book is dedicated to a backward look at the 20^th century: the two World Wars, the Cold War, and the quarter century of peace and prosperity following the fall of the Soviet Union. The final part looks at the “second Eurasian century” which we are currently living and draws lessons from the past.

 It is easy to see why readers of a certain persuasion would object to Brand’s interpretation. He readily acknowledges the American mistakes and overreactions in building and defending a safe and free world. Often, in defending democracy, America ended up crushing it and supporting oppressive regimes that it saw to be on its side. Scores of people died in wars from Vietnam to Iraq and many smaller skirmishes. However, Brand is unabashedly of the view that the US-dominated world after World War II has been, in the balance, hugely positive to everyone, and that US involvement in the world—especially the central Eurasian continent and the surrounding seas—is a necessity for the world and its people to prosper.

 He does see the threat posed by rising isolationism. Although the book was published in 2025, the manuscript must have been completed prior to the November 2024 elections. Brand writes: “If America makes a habit of electing aspiring strongmen, it may not remain a democratic superpower. And don’t think for a second that troubled autocracies can’t bring a world of woe” (p. 219). Unfortunately, it seems this is where we stand for the moment.

 Although a multipolar world would seem desirable at the face of it, the contenders to fill the other pole posts—notably China and Russia—do not provide an attractive alternative. In fact, these revisionist powers represent illiberal authoritarian regimes that would limit free speech and civil liberties in their spheres of influence. Western Europe without the US will not be able to resist. Brand makes a strong case for a broad coalition of democracies to cooperate, in terms of politics and policy, innovation and technology, and military. Citing Harry Truman in 1948: “America must pay the price for peace.” Brand’s final warning: “There is no law of nature that expansion must fail and tyranny must be vanguished. There is no guarantee that history takes the path of progress” (p. 245).  As the book shows, the 20^th century history is full of examples where things could have gone badly wrong.

How population characteristics shape our world, and how we can influence them

Jennifer D. Sciubba, currently President and CEO of Population Reference Bureau, is a political demographer. She is a leading expert in how population trends and composition affect politics and social relations -- and vice versa. This is what makes her analysis more interesting than your run-of-the-mill books on demography. My own background was originally in population geography, so the fundamentals that Sciubba explains in the book are very familiar to me.

Beyond those fundamentals, there is a lot of interesting and insightful in the book. For example, Sciubba points out that neither rapid population growth nor aging are per se not inherently problematic. Instead, their impact depends on the governance, political and economic circumstances. While fertility has dropped to below replacement levles, resulting in population shrinkage without immigration in most parts of the world (including Europe, North America, Japan, China, even Brazil), it remains very high in most of Africa where the societies' ability to accommodate the large cohorts of new entrants into labor force are generally weak. Sub-Saharan Africa contines to be the place with the largest growth in population. Almost 90% of world's population growth until mid-century will take place in lower-middle and low-income countries. This will pose challenges in terms of political stability and pressures to migrate. Sciubba also points out that it is not to poorest people from the poorest countries who migrate, because migration requires a certain amount of financial resources as well as skills to navigate the complex challenges of international migration. This fact casts doubt on the oft-heard justification for foreign aid to help people where they are, so that they will be discouraged to move.

A feature that makes Sciubba's book a lively read is her ample use of historical and current examples to demonstrate the more technical or theoretical points. They range from the impacts of China's one-child policy to the Rwandan genocide; from the relationship between aging societies and peace, to implications of Nigeria's youthful population. (I have to point out an odd slip: At one point, Sciubba places the southern African country of Botswana in the Sahel.)

Although Sciubba mentions climate change in several places, as acting as a stressor and as a driver of migration, it features fairly little in the book. That of course could be the focus of a separate book in itself.

A worthwhile and entertaining read, I'd recommend this book to anyone interesting in how population trends, fertility, mortality and migration shape society all over the world.

Blue Marble Evaluation: Premises and Principles, by Michael Quinn Patton

Blue Marble Evaluation is a call to action for all evaluators (and others) concerned with the state of planetary affairs in the new geological era of Anthropocene in which we live. What defines Anthropocene is that human impact on the Earth is so pervasive as to be the dominant force in modifying biosphere and atmosphere. Patton brings a palpable sense of urgency to the task of transforming evaluation to deal with transformative change. As an evaluator, Patton weighs the global challenges we are facing—including climate change and related phenomena, growing concentration of wealth and inequality, virulent infectious diseases and evolving super-viruses (yes, the book was written before the emergence of COVID-19), pollution, terrorism, refugee crises, etc.—against the positive trends (such as reductions in poverty and illiteracy, hunger and violence).  Striving toward a balanced assessment, he concludes that the “evidence points overwhelmingly … to a severe and growing crisis” (p. 19). 

Blue Marble Evaluation takes its name from the iconic photograph taken by the Apollo 17 astronauts on 7 December 1972, the first photograph of our home planet taken from space. This signifies the view of the whole Earth, which is fundamental to the Blue Marble Evaluation vision. Patton, a veteran evaluator and one of the most respected thinkers in the field, elaborates on the premises and principles of his vision for a renewed evaluation profession that is better able to respond to the global challenges. In the process he provides a biting critique of the narrow project mindset that dominates the evaluation practice today.

The book is organized in three parts. The first, The Blue Marble Perspective, presents four overarching blue marble principles: (1) Global thinking principle, (2) Anthropocene as context principle, (3) Transformative engagement principle, and (4) Integration principle. “The Blue Marble worldview constitutes a paradigm,” Patton writes (p. 6). Taking a holistic perspective and understanding global patterns and their implications is essential, as is thinking about all aspects of systems change at all levels from local to global. The integration principle emphasizes that transformation requires multiple interventions and actions on many fronts by multiple actors. Blue Marble principles should not only be applied to evaluation but also to design and implementation. Bringing the four initial principles together, Patton takes apart traditional project and program evaluation, as well as performance measurement and monitoring, as insufficient to addressing systems change at the global scale. He writes (p. 30): 

“Static and rigid randomized control designs—emphasis on control—are irrelevant to the uncontrollable dynamics of complex systems. Indeed, these traditional approaches to evaluation can create barriers to systems change by forcing transformational visions into narrow project boxes amenable to methods evaluators are comfortable with (e.g., logic models and specific, measurable, achievable, relevant, time-bound [SMART] goals). Innovations in evaluation include eclectic approaches created by tapping into a vast array of many-splendored, diverse, and innovative knowledge-generating and learning-oriented processes.”

Part II adds eight Operating Principles: (5) Transboundary engagement principle, (6) GLOCAL principle, (7) Cross-silos principle, (8) Time being of essence principle, (9) Yin-yang principle, (10) Bricolage methods principle, (11) World savvy principle, and (12) Skin in the game principle. Ranging from page 39 to 148, this section contains the bulk of the book. I won’t attempt to summarize it but will highlight a few points that I found essential. Patton discusses the need to think and act at the global scale and what this means. He provides a thoughtful critique of the Sustainable Development Goals (SDGs) adopted by over 190 countries all over the world. He rightly points out that they are actually not global, given that they are based on country-level targets. He wisely notes that “simply aggregating nation-state data doesn’t generate a global picture” (p. 44). Another important insight that flies in the face of conventional evaluation (and development) thinking—and with which I couldn’t agree more—is that there are no “best practices” because the context in which any intervention takes place is so important. “Global context informs local actions. Local contexts make global understandings meaningful and actionable” (p. 50). This up and down engagement between levels is the essence of the GLOCAL principle. Patton discusses scaling up as the most common way of having global impact, i.e. that successful initiatives are scaled out, scaled up or scaled deep so that they can reach more people and places. Scaling up is also at the heart of the operational model of the Global Environment Facility (GEF) where I work and just last year my office completed an evaluation of scaling up at the GEF. 

Apart from integrating across scales and geographies, Blue Marble principles call for integrating across silos. It is worth noting, like Patton does, that the SDGs, while intended to be integrated, also easily lend themselves to new silos. Yet in real life, the SDGs are indeed all interconnected and “evaluating the interactions between the SDG targets and indicators, both actual and aspirational, positive and negative, and short term and long term, offers significant opportunities for Blue Marble thinkers, designers and evaluators to contribute to the 2030 Agenda” (p. 65). (In full transparency, in the discussion on the cross-silos principle, I was very flattered to find Patton citing my work.) 

With the time being of essence, Patton addresses another key notion in evaluation and development professions: sustainability. According to the established principles promoted by OECD’s Development Assistance Committee, sustainability refers primarily to the continuation of project benefits after the completion of the intervention. Although, OECD/DAC has recently updated the definition to incorporate more of the environmental sustainability, there has been a lively debate about a stronger articulation of the environmental dimension. Patton’s discussion is most welcome as he elaborates on adaptive sustainability and resilience, making a clear distinction between engineering resilience (a performance-related notion that focuses on stability, efficiency, control and predictability) and ecosystem resilience that focuses on persistence, adaptiveness, variability and unpredictability. Aiming for long-term resilience and sustainability forces us to concentrate on the latter, in which case evaluators, too, must focus on adaptability of the system, rather than a static endpoint after an intervention ends and the benefits remain (pp. 78-82). Patton illustrates this with concrete examples, including some in which short-term focus on accountability undermined long-term sustainability. 

The chapter on the bricolage methods principle describing the evaluator as a bricoleur, a traditional French traveling “jack-of-all-trades” who would use whatever tools were at hand to fix a problem, hit a note with me. Similarly, evaluators have to use an eclectic variety of methods to fit the evaluation situation that they need to address. The Canadian evaluator Andy Rowe has similarly made the call for All Hands on Deck for evaluators to put their methods wars behind them and join hands in combating the greatest challenges facing humankind. Claims of “gold standards” (often made by those promoting experimental methods) sound ludicrous. To be clear, Patton is no Luddite: he fully acknowledges the value of new methods that involve, i.a., geospatial tools, big data and AI, but emphasizes the importance of choosing the most appropriate methods from an eclectic toolkit. Here he elaborates on six “bricolaged Blue Marble evaluation methods lessons” (pp. 114-116).  

The chapters on the world savvy principle and skin in the game principle are highly personal. The former emphasizes the global competencies that all evaluators should possess. He notes that the “profession of evaluation is fairly obsessed with competency” (p. 122) but that the concept of competence is problematic and can lead to “exclusion, reinforcing the status quo and power of the status quo” (p. 124). These again are concerns I can easily relate to, having been involved in evaluation groups that sometimes feel like aiming to be guilds with their exclusive approaches and membership. Without getting into detail about what being world savvy means, some of the central concepts include reflexivity and ongoing learning. 

The skin in the game principle will be hard for some evaluators to swallow (and I’ve already witnessed discussions to this end), given that evaluators (most of whom are social scientists by training) have been indoctrinated to think that we need to be neutral observers of the object of evaluation, looking from the outside in, without taking sides or showing emotion. Patton challenges this by making it clear that we should not hide our values, especially in the current world situation where our common future is at stake. Independence is usually seen as one of the most important characteristics of an evaluator, but in some cases total independence can even undermine the credibility of the evaluator. This might be the case with indigenous communities where a total outsider would not carry any credence. The evaluator would still use her or his best judgment weighing the evidence (for example not assuming that the local communities would necessarily always know best). He makes a clear and useful distinction between caring and bias (pp. 139-141). His argumentation stands on the broad shoulders of two evaluation thought leaders, Robert Stake and Michael Scriven, as well as others. He also makes a strong case for evaluation as transdisciplinary science, and for science as intervention aimed at generating knowledge and solving an urgent problem. 

The final part of the book outlines three Global Systems Transformation principles: (13) Theory of transformation principle, (14) Transformation fidelity principle: evaluating transformation, and (15) Transformational alignment principle: transforming evaluation to evaluate transformation. The first of these chapters is the most theoretical in the book. In it Patton attempts to move from a theory of change to a theory of transformation, doing so by integrating multiple theories to explain transformation. The theoretical frameworks he most relies on are network theory and innovation theory. He also expounds on a hypothesis developed by Jerald Hage, director of the Center of Innovation at the University of Maryland’s Department of Sociology who also was Patton’s dissertation advisor at the University of Wisconsin decades earlier. Altogether, the theory of transformation that Patton comes up with is a plausible one. It claims that transformation flows from an “understanding that the status quo is not a viable path forward and that networked action on multiple fronts using diverse change strategies across multiple landscapes will be needed to overcome the resistance from those who benefit from the status quo” (p. 168). This will lead to critical mass tipping points and consequently transformations. Earlier in the chapter Patton reminds us of the old truth found in innovation theory that it is futile to try to convince (and despair over) the often powerful interests that benefit from the status quo and resist change; instead, focus on supporting the early adopters and spreading their message until a tipping point is reached. This is a very useful message to keep in mind when hopelessness sets in ahead of the Sisyphean task of moving towards a more environmentally benign future.

In the chapter on transformation fidelity principle, Patton starts by reminding us that transformation has become a new buzzword but not everything touted as such is in fact transformational. Of true transformations in relatively recent history he names three examples: the end of apartheid in South Africa, the fall of the Berlin Wall, and the significant fall in new cases of HIV/AIDS. Transformation is difficult to define, but easy to recognize when it happens by (borrowing the term from the statistician Fred Mosteller) interocular significance (i.e., it hits you in between the eyes). Transformation can be defined as a sensitizing concept, “a way of talking about something that is not yet well understood, precisely defined, or operationally measured” (p. 174). Evaluating transformational change, however, is more challenging. Patton sets out to define an evaluation framework for evaluating transformation with the theory of transformation that he developed in the previous chapter. He reviews an influential evaluation of transformational engagement by the World Bank’s Independent Evaluation Group (using an approach that we adapted to evaluate transformational change at the GEF Independent Evaluation Office a few years back). While this approach was fine, it was, according to Patton, a definition-based, rather than theory-based evaluation. Interestingly, Patton concludes that “evaluating transformation must involve capturing the story, communicating the process and results, interpreting meanings, making values-based judgments about what occurred, extracting lessons, and facilitating visioning the way forward both short term ad long term. Telling the transformation story will involve mixed methods” (p. 185).

In the final chapter, Patton lays out the need to transform evaluation itself to evaluate transformation. This chapter also acts as a summary of the book and his thinking around the Blue Marble principles. He concludes that global sustainability should become a universal criterion in evaluation. The book ends with a vision of a global network of Blue Marble evaluators, a vision that is in the making to become reality.

Blue Marble Evaluation is a powerful book that transmits the sense of urgency, the very caring for the planet and the state of affairs that Michael Quinn Patton writes about. He emphasizes that while he has pulled the text together, it has been a collaborative effort with contributions from many people, including fellow Blue Marble evaluators, like Pablo Vidueira and Glenn Page (the acknowledgements run over 3.5 pages). The book is also amply illustrated, including with cartoons by Mark Rogers, Chris Lysy, Simon Kneebone and Claudius Ceccon. All of this collaboration is further proof that Michael Patton practices what he preaches. The book is also very erudite, drawing upon literature and theories of science, philosophy, sociology, astronomy and more in addition to evaluation. There are many practical examples from interventions and evaluations of different kinds, some more profound than others. All of this makes for interesting, if occasionally rambling reading as the reader is left guessing how it all comes back to the theme (spoiler alert: it usually does). And as Patton states, this is not a book about methods, but evaluators tend to be methods people so he writes quite much about methods. All of this may make the book quite a ride for a traditional evaluator versed in project performance evaluation. Those are some of the people who most should read this book. Having said that, I would recommend the book to anyone interested in evaluation and applied social science in the age of the Anthropocene.

What Counts as Credible Evidence in Applied Research and Evaluation Practice? by Stewart I. Donaldson
My rating: 5 of 5 stars

The evaluation field continues to engage in paradigm wars that involve heated debates over which approaches and methodologies produce the most reliable results to support evidence-based policy-making. Somewhat regretfully, the commendable goal of enhanced rigour in evaluation research has been hijacked by a focus on a narrow set of experimental methods—randomized controlled trials or RCTs—which have been proclaimed as the ‘gold standard’ by their proponents. This trend has been boosted by calls for unambiguous measurements of results and impacts, and cost-efficiency, by policy-makers and bureaucrats struggling with making policy choices and undertaking programs under increasing resource constraints. On the other side, the reaction from the proponents of more qualitative methodologies and participatory approaches to evaluation has been strong, even emotional at times. As a professional evaluator, I’ve witnessed these brawls first hand.

This book makes an excellent contribution to the debate through a balanced presentation of the issues and by letting the different sides to make their respective cases. The authors in the book include a number of leading scholars and practitioners in the field. The perspective is North American (all authors work in the US or Canada) and draws heavily on experiences from education and social services. Although my own work pertains to evaluating international development programs, I found the discussion in the book on what constitutes credible evidence very valuable.

In the two introductory chapters the editors frame the debate in the context of a search for evidence-based society and how this has played out in the use of experimental and non-experimental designs for collecting evidence. Quite didactically, they place the debate within the broader scientific paradigms of social inquiry, including logical positivism, post-positivism, constructivism and related thinking, and pragmatism. They also tentatively place the chapter authors along these paradigmatic axes. The following eight chapters are divided into two sections arguing, respectively, for experimental and non-experimental routes to credible evidence. The trajectory of the chapters moves from the most hard-core case for experimental designs to an argument for the credibility of image-based research in evaluation. The chapters in between tend to take a conciliatory approach to the extent that the last of the chapters in the experimental section could have been moved to the non-experimental section.

Part II entitled ‘Experimental Approaches as a Route to Credible Evidence’ contains four chapters. In the first, Gary Henry argues for high-quality policy and program impact evaluations as a necessity for providing solid evidence for policy choices, linking the matter to democratic theory and the need to detect and debunk bad policies. His view assumes that there actually are ways to objectively define what produces desirable outcomes—and what such outcomes are. He acknowledges types of bias in evaluation research, but sees them in technical, rather than political terms. In the chapter that follows, Leonard Bickman and Stephanie Reich assess the credibility, reliability and validity of RCTs, concluding that while there are threats to the validity (especially external validity) of RCTs, they still can be seen to be amongst the most credible designs available to evaluators. Despite this overall conclusion, Bickman and Reich acknowledge that there are other, non-experimental approaches to establish causality (many natural science disciplines—geology, astronomy, engineering, and subfields of medicine—base their research on non-experimental designs). In social sciences, they particular highlight the program theory, theory-driven or pattern-matching method, recognizing that such other approaches are needed to supplement RCTs that can only answer a very limited number of questions.

The last chapter in this part of the book, by George Julnes and Debra Rog, introduces the concept of actionable evidence. The authors assert that for evidence to be useful, it should not only be credible but also actionable, defined as adequate and appropriate for guiding actions in targeted real-world contexts. The lengthy chapter takes as its starting point the question of relating the choice of methods to the questions that stakeholders want addressed. They proceed to outline a multitude of evaluation tasks (borrowing from Carol      Weiss) and then consider the implications of the tasks for methodology. Another way of framing the evaluation questions relates to the level of conclusion and the different levels of causal questions in impact evaluation, including whether the evaluation seeks to provide an aggregate description, disaggregation for causal analysis, or an inferential analysis of the underlying constructs and causal mechanisms. They state that, “experimental methods are argued as appropriate for strengthening impact-evaluation conclusions, but the value of these methods is dependent on the level of conclusions being addressed” (p. 104). Summarizing the discussion on the relationships between questions and methods, Julnes and Rog express their view that, while particular questions call for quantitative designs, there is substantial territory open to other designs. This summary leads the authors to consider the contextual factors that affect the adequateness and appropriateness of alternative methods, including policy context and the nature of the phenomena studied. They then discuss how to judge the adequacy of methods for providing the evidence that is needed to address the stakeholder questions identified, and when is it appropriate to use particular methods for causal analysis, taking into account constraints posed by factors internal to the program, evaluation capacity and political constraints, as well as ethical considerations. The discussion is nuanced and fair. Julnes and Rog conclude by affirming the primacy of the evaluation stakeholder questions in influencing the types of evidence needed. They caution against “simple frameworks that drive method choice in a somewhat automatic fashion” and instead wish to support “more informed judgments on method choice in political public policy environments” (p. 128). To me, this is one of the strongest, most thoughtful and balanced chapters in the book. Therefore, it also deserves its place in the middle, bridging the quantitative and qualitative parts.

Another chapter in this section of the book, preceding the chapter discussed above, by Russell Gersten and John Hitchcock focuses on the role of the What Works Clearinghouse, established in 2002 by the U.S. Department of Education. The chapter is descriptive and possibly useful to education researchers and evaluators, but did not raise my own somewhat biased interest.

Part III on ‘Nonexperimental Approaches for Building Credible Evidence’ consists of another five chapters, starting with one of the grand old men of evaluation, Michael Scriven. His chapter, ‘Demythologizing Causation and Evidence,’ is written with flair in lively language. Like Julnes and Rog before him, Scriven enlists other sciences—from mathematical physics and geology to anthropology, ethnography and epidemiology—to demonstrate how experimental methods are but one of the many approaches to analyse causation. He writes, “much of the world of science, suffused with causal claims, runs along very well with the usual high standards of evidence, but without RCTs” (p. 136). Breezing through the origins of causal concepts, the cognitive process of causal inference vs. observation, and the level of evidential certainty required for scientific, legal and practical purposes, he then addresses the alleged supremacy of RCTs as well as “other contenders.”  His myth-busting position is that “(i) the attempted takeover of the terms evidence and cause is partly inspired by the false dichotomy between experiment and quasi-experiment, and (ii) the whole effort is closely analogous to the attempted annexation of the concept of significance by statistically significant” (p. 151).

The following chapter by Jennifer Greene discusses evidence as ‘proof’ vs. evidence as ‘inkling.’ Her premise is that evaluation is both influenced by the political, organizational and sociocultural contexts, as well as it serves to shape that context. Consequently, “evaluation is not a bystander or neutral player in the debates that often surround it, but rather an active contributor to those debates and to the institutions that house them” (p. 153). Her chapter attempts to demonstrate how the present discourse assumes that ‘evidence’ can make social systems ‘efficient and effective’ and how these assumptions convey a particular view of human phenomena and responsibilities of government in democratic societies. The argument is a useful antidote to the positivistic view presented by Henry earlier in the book. Her vision of evidence is not providing the truth, or neat and tidy small answers to small questions. Rather, in Greene’s view evidence must provide a “window into the messy complexity of human experience; evidence that accounts for history, culture, and context; evidence that respects difference in perspective and values; evidence about experiences in addition to consequences; evidence about the responsibilities of government, not just the responsibilities of its citizens; evidence with the potential for democratic inclusion and legitimization of multiple voices—evidence not as proof but as inkling” (p. 166).

Sharon Rallis starts her chapter by telling the story of how she first encountered evaluation when she was teaching a federally funded summer program that was subject to an evaluation. The evaluators insisted on holding on to their plan to assess the program against a single outcome, with no regard to the important associated benefits that the program had bolstering the self-esteem of the participating students. Furthermore, the evaluation based on a quasi-experimental design deprived half of the students from participating in an important part of the program, which Rallis and other program colleagues felt was unfair. While the evaluators claimed that their work was scientific and rigorous, Rallis pondered about the missing piece and came to the conclusion that it was ‘probity’—goodness and moral soundness. Consequently, she began to study evaluation with a commitment to make evaluations useful for the program personnel and participants. This chapter elaborates on her vision of evaluation with probity and moral reasoning, grounded in nonconsequentialist theories. She explains: “The evidence we collect looks quite different from that of our colleagues who measure outcomes. Our aim is not to cast judgment … but to discover what happened and what the experience meant to the program participants. We hope that our discoveries can lead to improving the program and thus the well-being of the participants” (pp. 174-175). Rather than RCTs, evaluation done with these principles borrows tools from fields such as ethnography, phenomenology and sociolinguistics/semiotics. She presents a case from an evaluation and needs assessment of an HIV/AIDS education and prevention program that provides some unexpected insights into the participants’ experiences. She asserts that this work is rigorous because “it is grounded in theory and previous research and in moral principles of justice and caring” (p. 178).

Sandra Mathison in her chapter ‘Seeing Is Believing’ explores the credibility of image-based research and evaluation, as one form of evidence to establish and represent truth and value. Like the part III authors before her, she emphasizes how the credibility of evidence and the knowledge thus created is contingent on experiences, perception and social conventions. Image-based research uses images in three ways: (i) as data or evidence; (ii) as an elicitation device to collect other data, and (iii) as a representation of knowledge (p. 184). Mathison posits four considerations for establishing the credibility of image-based research: (1) quality of the research design, (2) attention to context, (3) adequacy of the image from multiple perspectives, and (4) the contribution images make to new knowledge (p. 188).

The last chapter in part III, by Thomas Schwandt, is entitled ‘Toward a Practical Theory of Evidence for Evaluation’ and it functions as a kind of recap of what has come before; as such, it could have equally well been placed in part IV on conclusions. This is another rich chapter that goes to the heart of the debate of what we mean by evidence: “…information helpful in forming a conclusion or judgment. Framed in a more rigorous epistemological perspective, evidence means information bearing on whether a belief or proposition is true or false, valid or invalid, warranted or unsupported. At present, we face some difficulty and confusion with understanding the term evidence in evaluation because it is often taken to be synonymous with the term evidence-based” (p. 199). He then proceeds to problematize the term evidence-based, as being narrowly interpreted to mean only a specific kind of finding regarding causal efficacy. Secondly, Schwandt argues why evidence cannot serve as a secure and infallible base or foundation for action. Furthermore, he emphasizes that, as an aspect of policy making, evaluation must consider ethics. Schwandt concludes that “deciding the question of what constitutes credible evidence is not the same as deciding the question of what constitutes credible evaluation … However necessary, developing credible evidence in evaluation is not sufficient for establishing the credibility of an evaluation” (p. 209). He further asserts that method choice alone does not determine what is credible and convincing evidence. He calls for framing evidence in a practical-theoretical way that that is concerned with the character and ethics of evidence and the contexts in which evidence is used.

In the final part of the book, Melvin Mark summarizes the different perspectives of the book with an aim of changing the terms of the debate. He concludes: “Extensive and continued discussion of the relative merits and credibility of RCTs versus other methods would have limited capacity to move forward our understanding and our practice. … by changing the terms of the debate, we may be able to improve understandings of deeply entrenched disagreements; move toward a common ground where such can be found; better understand the disagreements that remain; allow, at least in select places, credible evidence as part of the conversation; and enhance the capacity of stakeholders to make sensible decisions rather than be bewildered by our disagreement or draw allegiances based on superficial considerations” (pp. 237-8). Certainly a deserving goal.  The book ends with an epilogue by Stewart Donaldson that attempts to provide a practitioners guide for gathering credible evidence in the evidence-based global society.

I thoroughly enjoyed reading this book and although much of the debate around epistemology, approaches and methods is familiar to someone educated and working in evaluation and applied social science research, the way it is framed in this book is truly enlightening. For a thoughtful reader, it becomes evident that the truth—as almost always in such debates—is somewhere in the middle. All of the approaches and methodologies have merit when used appropriately and in appropriate contexts. Both experimental and non-experimental methods can be rigorous, but both can also have serious flaws with regard to internal and external validity, relevance and appropriateness. The old saw about everything looking like a nail when you have only a hammer in your tool box is true here as well. The take-home lesson is that, instead of allowing methods to dictate one’s evaluation questions and designs, one should choose one’s methods according to the questions one wants answered.


View all my reviews

Shooting America

On Friday, 14th of December 2012, a young gunman walked into Sandy Hook elementary school in Newtown, Connecticut, and proceeded to shoot and kill 26 people, including 20 children in the ages of 6 and 7 years. This horrific incident was one of the worst mass murders even in the violent history of America. Unfortunately, however, it wasn’t entirely isolated. In fact, this was the third incident of a similar kind to take place in the United States just in 2012. These recent cases include the movie theatre shooting in Aurora, Colorado, on the 20th of July 2012, in which twelve people were killed and 58 injured; and the shooting at a Sikh Temple in Oak Creek, Wisconsin, on August 5, in which a white supremacist killed six people, possibly mistaking the Sikhs for Muslims. Six of the worst mass shootings in America have taken place since 2007.

Could this finally be a turning point—the straw that broke the camel’s back, in the words of Sen. Dianne Feinstein of California—in the American attitude towards gun violence? There are signs of outrage that seem unprecedented; after all, the horror of the slaughter of these little children is just too painful, too hard even to imagine. But don’t hold your breath.

There is much focus on the mental health of the shooter and on providing increased security in schools. This is understandable. However, I do not see this primarily as a mental health issue. This is not meant to belittle the importance of psychological factors or to deny the importance of mental illness as an explanatory factor. America’s mental health care system is clearly broken and there seem to be extraordinarily many sick people who turn to violence. However, there are crazy people everywhere, but in most other places they can be stopped before they are able to commit mass murder. Just recently, a Chinese nutcase attacked a school in his own country. Armed only with a knife, he was able to injure a number of people but not to kill anyone before he was apprehended.

In this latest case in Connecticut, the assailant was in the possession of three advanced pieces of weaponry: Glock and Sig Sauer handguns and an AR-15 Bushmaster semi-automatic rifle. He used this firepower to enter the school and then to kill the kids and their teachers. It is worth noting that apparently he had acquired access to these guns from his mother (who was the first victim of his killing spree), a gun enthusiast.

What surprises me is that so relatively few Americans have arrived at the inevitable conclusion that access to high-powered firearms itself is a threat. While a few people like Feinstein and Dan Gross of the anti-gun violence Brady Campaign have systematically brought the issue of stronger gun control up, the debate still seems to be up in the air. Although most pro-gun politicians have had the sense of laying low since Friday’s tragedy, a number of gun activists, sensing a threat to their God-given right to carry any weapon, have again decided that attack is the best defense. People like John Lott, author of More Guns, Less Crime, has been seen on a number of TV talk shows peddling his claim that everywhere where stricter gun controls have been effected, murder rates have gone up (would someone please look at the statistics behind this implausible claim). Citing the Aurora example, Lott asserts that the shooter actually chose that particular movie theatre because it didn’t allow guns. There were other theatres closer to the murderer’s home but, Lott implies, there might have been armed people there who would have returned the fire, so the shooter was afraid of attacking them. Just imagine a gunfight in a crowded and dark movie theatre.

Another creep, Philip Van Cleave, president of the sinister sounding Virginia Citizens Defense League, actually had the temerity to tell Washington Post on Sunday after the Newtown massacre that guns are fun. Defending the people’s wish to own semi-automatic weapons like the AR-15 (implicated in the three latest massacres mentioned above), he was quoted by the newspaper as saying: “I could ask you why should anyone want a Ferrari? [Bushmasters] are absolutely a blast to shoot with. They’re fast. They’re accurate. … Guns are fun, and some of them are much more cool than others.” Apart from the stunning insensitivity, Van Cleave’s views may not be that rare amongst Americans.

Who exactly Lott, Van Cleave and their ilk envision could have been better armed to fight back in the Sandy Hook elementary school in Newtown is not clear to me. The young teachers who died with their students? Or perhaps the children themselves? Employing armed guards and arming teachers or movie goers or citizens in general so that they can return fire when a deranged person starts shooting at them is such a dystopian vision that few of us I imagine would cherish. And would all of us really want to be trained in handling guns?

Maybe I’m wrong. Maybe many Americans do see this as a desirable way of protecting oneself and one’s liberties. After all, this is a country where large groups of people arm themselves in preparation against a takeover by the socialists in the Federal Government (or even—gasp!—the United Nations) . It would seem inevitable that an even larger number of people walking and driving around with concealed weapons would result in those guns being used when things heat up in, say, a traffic jam or a supermarket line. And the risk of innocent bystanders getting hurt in these altercations seems high.

This happened on Friday, August 24, 2012, when a man shot his former co-worker near Empire State Building in Manhattan. The police killed the assailant immediately following the incident, in the process injuring eight innocent bystanders with ricocheting bullets—and remember, these were highly trained law enforcement officers who are experts in handling firearms.

“Guns don’t kill people; people kill people,” has long been the rallying call of the supporters of the National Rifle Association (NRA), a powerful pro-gun lobby. Regretfully, that’s just not true. There will always be nuts who want to kill people. However, their ability to do so is significantly increased by the availability of guns. Later the same evening, the brother of the Empire State Building victim was interviewed by CNN. His opinion: Don’t turn this into a referendum on firearms; if the killer hadn’t had a gun, he could have used a “baseball bat or whatever.” Perhaps so. People who are inclined to commit premeditated murder do find a way to do so. A baseball bat would do the trick, but it just might be more difficult to carry it concealed to the spot, then un-shield it rapidly when the intended victim is in sight, and to wield it to his head—all on a crowded city street. The intended victim might have a higher chance of escape, too, when he sees the batter approaching. In this particular case, the victim was also physically larger than the assailant, which might have made a difference in a fight without guns.

A more important point is that premeditated murder is, by definition, usually targeted towards a specific individual against whom the would-be murderer bears a grudge. Such murders occur in all countries.

There are two very obvious cases where the availability of guns does create a much larger hazard. One, as we have seen, is indiscriminate mass murder by a lunatic. An individual’s ability to massacre a large number of people is directly correlated with the availability of guns, especially powerful assault weapons. In this year’s cases, the madmen’s ability to kill was multiplied by the availability of such automatic weapons, with clips holding more than 100 rounds of ammunition reducing the need to reload. Such weapons and clips have no legitimate use in private hands, as they are by no stretch of imagination needed for hunting or target practice. Their only purpose is to enable the killing of as many people as possible in close combat.

The other case, which so obviously speaks against having guns around is that the majority of killings in the US happen between family members and people who know each other. Only a small fraction of these are premeditated murders. Most are either accidents (every year many people shoot themselves or their dear ones accidentally when fondling their beloved guns) or happen when arguments—between spouses, friends or colleagues—heat up and a loaded gun happens to be handy. Guns kill people, even when people don’t intend to do so.

In the December 2012 issue of The Atlantic, that appeared on newsstands just before the Connecticut massacre, Jeffrey Goldberg argues that it is too late to install any further gun controls in America. There are already some 280-300 million guns owned by private citizens in America and each year this number is increased by more than 4 million. These are of course stupendous figures, given that the total population of the USA is just around 311 million. I have recently heard that slightly less than half of American adults own a gun. Simple arithmetic thus implies that these people have multiple guns at home. Goldberg draws the conclusions that it would be impossible anymore to regulate the situation through democratic means and, therefore, it would be better to give more guns to law-abiding citizens so that they can defend themselves. This is a saddening view, although it does have a certain logic. Yet, by the same logic it would be futile to attempt to address any similar issue that involves an advanced situation, including nuclear disarmament. His solution would also bring back the Wild West in which disputes were settled with six-shooters.

Except that in the Wild West, guns were quite strictly controlled in towns where the sheriff made sure that gunslingers would check their weapons at the gate. Joe Klein in Time magazine (August 6, 2012) outlined how this free-for-all guns-galore is not an American tradition or what the Founding Fathers expected. Rather it’s a result of a concerted advocacy effort by all kinds of right-wing groups since the 1970s to overturn gun control legislation. In 1993, during Bill Clinton’s presidency, legislation was passed to ban assault weapons, but that too was let expire a decade later.

Of all industrialized countries, the United States has by far the largest amount of guns per capita: 88.8 firearms per 100 people. This is far more than the 54.8 in the second most gun-heavy country in the world, Yemen. America has some 5% of the world’s population but, depending on the estimate, up to half of the world’s firearms in the hands of private citizens. Topping the list with Yemen, which many consider en route to becoming a failed state and which now is the principal host of Al-Qaeda on the Arabian Peninsula, should not be an accolade most Americans would want. Still a large number of Americans seem to be perfectly content with it.

Unfortunately, my own country of origin, Finland, is number 4 on the list (the third place goes to Switzerland, where most men who have served in the army have a rifle in their closet). Why Finland, the seemingly tranquil Nordic country known for its peace-making efforts on the world scene? I suppose the reasons are something similar to those that would be applicable to America, too: a macho culture of rugged individualists. There, too, men go on rampage shooting people.

Another factor that militates against gun control in America is (you guessed it) money. There is a lot of money involved in gun sales and they are not limited to domestic sales, as if the 4 million guns sold in the US annually wouldn’t be enough. Mexico’s drug war is largely fought with American guns: it is estimated that 80% of the guns confiscated from Mexican gangs have been bought legally in the US.

In the American political system, lobbyists for special interest groups play a central role. Politicians find it hard to go against the lobbyist if they want to stay in power. According to the Time article by Joe Klein, NRA has funded a total of US$18.9 million to political parties and candidates running for federal office since 1990. Of this amount, 82% has gone to the Republicans. The New York Times columnist Charles Blow has calculated that the NRA’s financial contributions to politicians in Washington are 4,100 times larger than those by the largest anti-gun organization, the Brady Campaign.

When culture, tradition, corporate interests and money come together, it will be a tough job to go against them and change things when it comes to the prevalence of guns and the ensuing gun violence. This is the reason for my pessimism, but I do hope that I am proven wrong. Perhaps, in the unspeakable tragedy of the Sandy Hook elementary school, there lies a seed of hope for some modest reform. If nothing more, it would seem reasonable to start by banning semi-automatic assault weapons and clips containing tens or hundreds of rounds of ammunition.

Dead Aid: A Flawed Book on an Important Topic

This brief book has received a lot of attention since it was published just a few months ago earlier this year. The author attacks development aid to Africa with a vengeance, calling it the “silent killer of growth.” Despite the billions of dollars worth of development aid to Africa provided by Western governments over the past four decades, poverty and inequality have in fact increased on the continent. This is not the first book to criticize aid, but the fact that the author is an African woman (instead of the usual white males) has struck a chord with many reviewers and the audience. Dambisa Moyo is livid about what she sees as not only wasted money but as an active hindrance to economic development. All the well-meaning but naïve celebrities campaigning for more aid to Africa get an earful from Dr. Moyo, an academic economist who has made her career in the financial sector in the West.

Moyo states the goal of the book in the Introduction: “This book is about the aid-free solution to development: why it is right, why it has worked, why it is the only way forward for the world’s poorest countries” (p. xx). The book is divided into two parts. The first third, ‘The World of Aid,’ provides an history and critique of development aid to Africa and more broadly. Unfortunately, her training in economics—and this being her first book to a general audience—comes through as overuse of numbers and statistics, resulting in numbing and sometimes impenetrable sentences:

“Local debt returned investors 15 per cent in 2006, and 18 per cent in 2007. In the last five years average African credit spreads have collapsed by 250 basis points. What this means is that if a country issues US$100 million in debt, it is saving itself US$2.5 million per year relative to where it was five years ago.” (p. 4)

“Among the top five aid recipients from the Marshall Plan were Great Britain, which received the lion’s share of 24 percent, and France, Italy and Germany, which receive 20, 11 and 10 per cent, respectively. In per capita terms smaller European countries received more support: Norway received US$136 per person, Austria US$131, Greece US$128 and the Netherlands US$111.” (p. 12)

Also her indignation about aid is so rabid that her writing often turns to hyperbole or in other places is painfully simplified:

“This is the vicious cycle of aid. The cycle that chokes off desperately needed investment, instils a culture of dependency, and facilitates rampant and systematic corruption, all with deleterious consequences for growth. The cycle that, in fact, perpetuates underdevelopment, and guarantees economic failure in the poorest aid-dependent countries.” (p. 49)

“In an aid-dependent environment, the talented—the better-educated and more-principled, who should be building the foundations of economic prosperity—become unprincipled and are drawn from productive work towards nefarious activities that undermine the country’s growth prospects.” (p. 50)

All of this makes especially the first part of the book rather tedious. She definitely is no match to more established aid critics, such as New York University professor and former World Banker Bill Easterly whose books The Elusive Quest for Growth and The White Man’s Burden make for thoughtful and entertaining reading.

The second part, ‘A World without Aid,’ luckily lifts the book to a higher level, as Ms. Moyo focuses on the positive rather than bashing what she sees as a crime against Africa. She definitely does not lack in ambition, stating that:

“This book provides a blueprint, a road map, for Africa to wean itself off aid. … What follows is a menu of alternatives to fund economic development across poor countries. If implemented in the most efficient way, each of these solutions will help to dramatically reduce Africa’s dependency on aid.” (p. 75) and “The Dead Aid proposal envisages a gradual (but uncompromising) reduction in systematic aid over a five- to ten-year period.” (p. 76)

Here she launches her ‘Dead Aid strategies’ that, “if embraced wholeheartedly, will not only turn the economic tide in the short term, but also promise longer-term growth” (p. 143). Her blueprint for indigenous economic development in Africa focuses on a set of solutions, notably the development of capital markets, foreign direct investment, trade, and various forms of microfinance. She does also recognize the role of remittances from the overseas diaspora that are so important to many developing countries. Although the remittances are a type of aid—an influx of unearned money into the country from abroad—Ms. Moyo believes this to be less harmful than government to government aid.

Although the solutions presented are basically free-market oriented, the author is principally concerned about how to finance development, rather than the development model itself. She even suggests that it matters little whether the country in question has a capitalist or socialist development strategy. Quoting the Scandinavian experience, she recognizes that governments can raise money on the free markets while using it on a “socialist agenda” of free education and health care (pp. 72-73). In fact, she considers that for such governments using free-market tools to finance development is particularly important.

Furthermore, despite her call for “uncompromising” reduction of aid, she shows certain flexibility in allowing that “more-modest aid programmes that are actually designed to address the critical problems faced by African countries can deliver some economic value” may have some room to be part of Africa’s development financing strategy (p. 76).

And all of her ire is not directed towards aid and the Western nations that provide it, more for their own interest than in Africa’s, in her opinion. She frequently chastises African policymakers for corruption and red tape, reminding us that it is not accidental that FDI avoids Africa.

Dambisa Moyo dedicates an entire chapter, entitled ‘The Chinese are Our Friends,’ to the role of China in Africa. With poorly disguised excitement she describes China’s rapidly emerging engagement on the continent and goes as far as claiming that “in the last sixty years, no country has made as big an impact on the political, economic and social fabric of Africa as China has since the turn of the millennium” (p. 103). She is particularly enthusiastic about the fact that the Chinese involvement is all for commercial purposes—whether FDI or trade—unlike that of the West, which has for long given Africa “something for nothing.” In earlier parts of the book she has fashioned China and other East Asian countries that have developed through their own efforts as role models for Africa. She is well aware of the risks of the relationship with China: that the Chinese companies might underbid local firms or would not hire Africans or that they would have lax safety standards—all the standard issues that worry concerned Westerners when they think of China’s growing influence in Africa. However, Moyo correctly points out that, while Westerners are outraged by Chinese support for Africa’s corrupt and rogue leaders, these same “notorious plunderers and despots” have risen and thrived “under the auspices of Western aid, goodwill and transparency” (p. 108). Furthermore, to her, the pros of Chinese involvement far outweigh the cons:

“Bartering infrastructure for energy reserves is well understood by the Chinese and Africans alike. It’s a trade-off, and there are no illusions as to who does what, to whom and why…To continue to grow at its extraordinarily rapid rate China needs fuel, and Africa has it. But for Africa it’s about survival. In the immediate term, Africa is getting what it needs—quality capital that actually funds investment, jobs for its people and that elusive growth. These are the things that aid promised, but has consistently failed to deliver.” (p. 111)

In the case of trade, Dambisa Moyo is equally harsh on the West as on Africa itself. She rightly reminds us the “elected Western politicians have remained keen to protect their agricultural markets, and win the backing of the powerful farming lobby. The net result is a protective world of trade restrictions and barriers up around the West, to keep African (and other developing regions’) produce out” (p. 115). Yet at the same time she finds that the “most egregious examples come from Africa itself. African countries impose an average tariff of 34 per cent on agricultural products from other African nations, and 21 per cent on their own products” (p. 117).

Regretfully, the many good points that Moyo raises are again undermined by her style. The text is littered with cringe-inducing similes (“seduced by the siren call of aid, African governments sink their ships on the rocks of development demise” – p. 88). And in her fervour she again resorts to hyperbole:

“The West can choose to ignore all of this, but, like it or not, the Chinese are coming. And it is in Africa that their campaign for global dominance will be solidified. Economics comes first, and when they own the banks, the land and the resources across Africa, their crusade will be over. They will have won.” (p. 152)

Quite unnecessarily, in an apparent attempt to liven up the discussion, she invents and uses a fictitious country, the Republic of Dongo, as a dummy against which she places her proposals. Dongo is supposedly some sort of hybrid displaying many of the characteristics of various African countries. However, the decoy adds little more than an additional irritant to the book.

All in all, I read Dead Aid with mixed feelings. Early on, the angry prose and doctrinaire perspective really got to me. Having worked for most of my adult life on international development issues (unlike Dr. Moyo), I was also taken aback by the wholesale dismissal of virtually all development cooperation. Then later when focusing on the alternatives to aid, the book significantly lightened up and the solutions that were proposed started to be much more convincing. There is no denying that the East Asian countries, like China, Korea and Singapore, all experienced rapid development that was not based on aid, at least over an extended period. Surely there are lessons for Africa to be learned there. How countries develop is not straightforward or simple. Each place has its own dynamics and idiosyncrasies to which complex geographical, historical, political and economic factors contribute (see for example Paul Collier’s recent book The Bottom Billion). It is very important that these issues are discussed, without taboos and prejudice. Dambisa Moyo has contributed to the debate with this opinionated and flawed, but still important book.