Behavioral and Brain Sciences

We expect that consensus meetings, where researchers come together to discuss their theoretical viewpoints, prioritize the factors they agree are important to study, standardize their measures, and determine a smallest effect size of interest, will prove to be a more efficient solution to the lack of coordination and integration of claims in science than integrative experiments.

This commentary argues against the indictment of current experimental practices such as piecemeal testing, and the proposed integrated experiment design (IED) approach, which we see as yet another attempt at automating scientific thinking. We identify a number of undesirable features of IED that lead us to believe that its broad application will hinder scientific progress.

To succeed, we posit that research cartography will require high-throughput natural description to identify unknown unknowns in a particular design space. High-throughput natural description, the systematic collection and annotation of representative corpora of real-world stimuli, faces logistical challenges, but these can be overcome by solutions that are deployed in the later stages of integrative experiment design.

Almaatouq et al. propose an "integrative approach" to increase the generalisability and commensurability of experiments. Yet their metascientific approach has one glaring omission (and misinterpretation of) - the role of sample diversity in generalisability. In this commentary, we challenge false notions of subsumed duality between contexts, population, and diversity, and propose modifications to their design space to accommodate sample diversity.

The authors rightly critique existing social sciences approaches. However, they are too quick to dismiss the criticism that their proposed paradigm is atheoretical. Social and cognitive theories are indeed incommensurate, often due to the lack of a unifying framework. Without proper integration with theoretical frameworks, their proposal may merely produce a resource-intensive veneer of thoroughness without substantive improvements to understanding.

Integrative experiment design promises to foster cumulative knowledge by changing how we design experiments, build theories, and conduct research. I support the push to increase commensurability across experimental research but raise several reservations regarding results-driven and large-team-based research. I argue that it is vital to preserve academic diversity and adversarial debate via independent efforts.

This commentary analyzes the extent to which the incommensurability problem can be resolved through the proposed alternative method of integrative experiment design. We suggest that, although one aspect of incommensurability is successfully addressed ( dimensional incommensurability ), the proposed design space method does not yet alleviate another major source of discontinuity, which we call conceptual incommensurability .

The target article argues researchers should be more ambitious, designing studies that systematically and comprehensively explore the space of possible experiments in one fell swoop. We argue that while "systematic" is rarely achievable, "comprehensive" is often enough. Critically, the recent popularization of massive online experiments shows that comprehensive studies are achievable for most cognitive and behavioral research questions.

Integrative experiment design is a needed improvement over ad hoc experiments, but the specific proposed method has limitations. We urge a further break with tradition through the use of an enormous untapped resource: Decades of causal discovery artificial intelligence (AI) literature on optimizing the design of systematic experimentation.

Demonstrating the limitations of the one-at-a-time approach, crowd initiatives reveal the surprisingly powerful role of analytic and design choices in shaping scientific results. At the same time, cross-cultural variability in effects is far below the levels initially expected. This highlights the value of "medium" science, leveraging diverse stimulus sets and extensive robustness checks to achieve integrative tests of competing theories.

Almaatouq et al.'s prescription for more integrative experimental designs is welcome but does not address an equally important problem: Lack of adequate theories. We highlight two features theories ought to satisfy: "Well-specified" and "grounded." We discuss the importance of these features, some positive exemplars, and the complementarity between the target article's prescriptions and improved theorizing.

Almaatouq et al. argue that an "integrative experiment design" approach can help generating cumulative empirical and theoretical knowledge. Here, we discuss the novelty of their approach and scrutinize its promises and pitfalls. We argue that setting up a "design space" may turn out to be theoretically uninformative, inefficient, and even impossible. Designing truly diagnostic experiments provides a better alternative.

Integrative experiment design assumes that we can effectively design a space of factors that cause contextual variation. However, this is impossible to do so in a sufficiently objective way, resulting inevitably in observations laden with surrogate models. Consequently, integrative experiment design may even deepen the problem of incommensurability. In comparison, one-at-a-time approaches make much more tentative assumptions about the factors excluded from experiment design, hence still seem better suited to deal with incommensurability...

No abstract text is available yet for this article.

The constructs of motivation (or needs, motives, etc.) to explain higher-order behavior have burgeoned in psychology. In this article, we critically evaluate such high-level motivation constructs that many researchers define as causal determinants of behavior. We identify a fundamental issue with this predominant view of motivation, which we called the black-box problem. Specifically, high-level motivation constructs have been considered as causally instigating a wide range of higher-order behavior, but this does not explain what they actually are or how behavioral tendencies are generated...

We take issue with Glowacki's assumption that intergroup relations are characterized by positive-sum interactions and suggest to include negative-sum interactions, and between-group independence. As such, peace may be better defined as the absence of negative-sum interactions. Rather than being a consequence of cooperation, peace emerges as a necessary but not sufficient prerequisite for positive (in)direct reciprocity between groups that, in turn, is key to social identities and cultural complexity.

While effective institutional practices are critical for the evolution of peace certain factors deter their effectiveness. In-group and out-group dynamics may make peace difficult between culturally distinct groups. Critical ecological conditions often lead to intractable conflict over resources. And within group conflicts of interest most prominently between generations may inhibit effective peace making.

Glowacki defines peace as harmonious relationships between groups maintained without the threat of violence, where groups can be anything from families to nation states. However, defining such contentious concepts like "peace" and "groups" is a difficult task, and we discuss the implications of Glowacki's definitions for understanding intergroup relationships and their evolutionary history.

Humans are predisposed to form in-groups and out-groups that are remarkably flexible in their definition due largely to the complex language that has evolved in them. Language has allowed for the creation of shared "background stories" that can unite people who do not know each other. Second, the discovery of agriculture has resulted in the critical need to negotiate boundaries, a process that can lead to peace (but also war).

Here we revise Glowacki's model by proposing a simple and empirically tested mechanism that is applicable to a comprehensive set of social interactions. This parsimonious mechanism accounts for the choice of both cooperative and peaceful alternatives and explains when each choice benefits the interacting parties. It is proposed that this mechanism is key to the evolution of both peace and conflict.