Choosing the Future of Science: The Social Organization of Scientific Inquiry

Saturday - Sunday, 20-21 April 2013
Center for Philosophy of Science
817 Cathedral of Learning
University of Pittsburgh
Pittsburgh, PA USA

Abstracts
(Alpha. by Author)

Reorganizing Peer Review in an ArXival Age
Benjamin Almassi, College of Lake County

The growth of arXiv and other e-print platforms for scientific communication provides an opportunity to revisit the uneasy assumption of social-epistemological primacy of scientific peer review. What is lost as scientific communities move from reliance on traditionally peer-reviewed publications to faster, open, largely unfiltered platforms like arXiv? Without the filter of masked review, what other social-epistemic mechanisms are available to gauge work and mitigate biases in collective scientific practice? Here I take up multiple factors, including the range of existing and emerging forms of open access, evidential and non-evidential roles of peer review traditionally conceived, and assorted considerations for and against open access. The development of arXiv as an open site of scientific communication has been responsive to issues of corroboration and gate-keeping, yet issues of gender and author identity biases are relatively neglected. I argue that such biases constitute a persistent evidential and ethical problem for open access absent traditional peer review.

Pharma without Profit: Using Open-Source Principles to Generate Big Data without Big Money
Alexandra Bradner, University of Kentucky

Drug discovery traditionally has occurred behind closed doors in for-profit corporations hoping to develop best-selling medicines that recoup initial research investment, sustain marketing infrastructures, and pass on healthy returns to shareholders. Only Big Pharma has the man- and purchasing-power to synthesize the thousands of molecules needed to find a new drug. Against this, several individual scientists and, more recently, the National Institutes of Health have suggested that this kind of applied genomics work calls for a new form of social organization—open-source, not-for-profit collaboration, which will speed the generation and understanding of new medicines.
This paper argues that drug discovery is a world that should be conceptualized through a care and not a justice lens. This means, first, that open-source discovery networks will have to rethink the standard practices according to which credit is granted to individual scientists and, second, that free-riders—who take data, but do not share their own—are uncaring and unreliable, but not unjust.

Policing Epistemic Communities
Justin Bruner, University of California – Irvine

We examine how particular social arrangements and incentive structures encourage the honest reporting of experimental results and minimize fraudulent scientific work.  In particular we investigate how epistemic communities can achieve this goal by promoting members to self-police the community.  Using some basic tools from game theory, we explore a simple model in which scientists both conduct research and have the option of investigating the findings of their peers.  We find that this system of peer policing can in many cases ensure high levels of honesty.

Theory from Chaos
Michael Dickson, University of South Carolina

I explore several agent-based, evolutionary, models of the simultaneous acquisition and promulgation of theoretical knowledge. The agents consist of a 'signaler-system' and a 'predictor-system' which themselves play a cooperative game of asymmetric information, the point of which is for the agent to 'learn' (under an appropriate learning dynamics) to predict the state of the world. By placing such agents on a grid and allowing them to communicate in various ways, one can study the simultaneous development and dissemination of theoretical knowledge. The surprising result is that widely held successful theories develop under very minimal conditions of cooperation (i.e., chaos) amongst agents.

Emerging Varieties of Research Organization
Elihu Gerson, Tremont Research Institute

Over the last century or more, the number of specialties has been rising steadily. As the number of specialties has grown, the intellectual scope of each has narrowed, so that we now have a large number (hundreds, or even thousands) of relatively narrow specialties. As a result, it is now routine for a specific research problem to require the coordinated efforts of multiple specialties. Moreover, the increasing complexity of research has also increased the number of ancillary occupations needed to carry out the work. These occupations include many kinds of technician, cartographers, collections managers, artists, librarians, computer programmers, archivists, veterinarians, and others. Many of these ancillary occupations deploy skills and credentials comparable to those displayed by principal investigators and faculty members. 

Specialties are embedded in the larger institutional organization of research (and the still larger organization of society) in many ways. For example, major research and teaching organizations are typically structured along specialty lines (e.g., as departments in universities). The interests and concerns of specialties and the organizations that house them are aligned to some degree by this kind of arrangement. 

The proliferation and narrowing of specialties means that this kind of smoothly aligned embedding is becoming more difficult to achieve, especially for newly emerging specialties whose intellectual connections to the system of research institutions are not yet well-established. One effect of this process has been the development of new ways of organizing research that do not fit comfortably into the traditional model of laboratories or centers with a few researchers, located on the campus of a single university or research organization, and clearly part of the broad research program of a single well-defined discipline. Instead, we now see complex projects that span multiple specialties and organizations. These projects exhibit new forms of organization that reflect their more complex relationships to a more complicated world. Two of these new forms of research organization are platforms and junctures. It is certainly the case that additional forms remain to be recognized and described. 

Platforms are coherent ensembles of materials. Methods, equipment, conventions and people that provide a set of conventional resources for supporting a variety of different programs. Computer operating systems such as Unix are an example. The established operating system supports many applications produced by different programmers and used in many different contexts. In addition to such operating systems, the world of research includes a variety of platforms such as model organisms (e.g., Drosophila, Caenorabditis elegans), model taxa, instrumental technologies such as atomic force microscopy or flow cytometry, specialized software systems such as the Rosetta system for analyzing protein structure and specialized classes of theoretical model such as the Price equation used in population biology. 
Junctures are overlapping intersections among multiple lines of research. A juncture includes more than a single project or program. Junctures develop relatively stable alliances that integrate technical aspects of two or more specialties. Models, procedures, and concepts from different specialties are brought to bear on a common set of overlapping research problems. Junctures are not associated with a particular specialty, nor are they housed in any particular department. People in a juncture work on common or overlapping problems. They become obligated to use one another’s results, techniques, and ideas in their own research. Hence, they must learn some of one another’s techniques, concepts and models. 

Junctures thus act as a place where specialties are partially integrated. 
For example, masculinization research projects study female masculinization in a variety of mammalian species. Masculinization researchers come from many specialties, including developmental biology, vertebrate zoology, endocrinology, reproductive biology, biochemistry, medicine, veterinary medicine, biological psychology, neuroscience, morphology, animal behavior, and others. Masculinization research is a stable complex intersection that is like a specialty in many ways Researchers are acquainted with one another and with each other’s work. They see one another at the meetings of many associations; they often collaborate on papers and studies; they referee one another’s papers; they trade reprints, students, samples, gossip, reagents, and other vital resources freely. But the juncture is missing many important organizational characteristics of a specialty: no single department covers all the relevant topics; there are no textbooks, standard courses, or degree programs; there is no single journal, association, or regular meeting; no specialized Internet-based resource serves as a common ground for interested researchers; no sponsor recognizes the juncture as a coherent field of study. 
Platforms and junctures are similar in that they cut across multiple specialties. They are persistent. They are housed and funded by multiple organizations. They establish and enforce conventions of practice for researchers working on a range of problems. Yet they are not traditional specialties. They do not have formal educational programs for perpetuating themselves, and they typically do not have their own learned associations. They are thus more loosely connected to the other parts of the system of research institutions than traditional specialties. Their work relies heavily on informal personal collaboration among researchers. These collaborations can trade off against the stability and solidarity of the established disciplines. 

These new organizational forms raise important questions about the ways in which discovery can be organized and supported reliably. For example, because they cut across specialty boundaries, platforms and junctures support epistemic integration among specialties. At the same time, their independence from any one specialty acts to ensure that this integration is only partial. Another effect of their cross-specialty connections is to bring incompatibilities in style, problem agendas, theories, procedures and concepts more evident and more pressing. The trend created by these emergent forms of research organization is away from traditional coherent and relatively self-contained disciplinary traditions, and toward far more complex network of overlapping and loosely affiliated programs that produce and define partial results that present many problems of integration.

Scientific Networks on Data Landscapes: Question Difficulty, Epistemic Success and Convergence
Patrick Grim (State University of New York at Stony Brook), Daniel Singer (University of Pennsylvania), Aaron Bramson (University of Michigan), William Berger (University of Michigan), Christopher Reade (University of Michigan), Carissa Flocken (University of Michigan), Steven Fisher (University of Michigan)

A scientific community can be thought of as a network of interactive agents attempting to answer questions on the basis of incomplete, conflicting, and sometimes ambiguous data.  The interaction between the structure of the network and the nature of the question under investigation affects epistemic desiderata, including accuracy and speed to community consensus.  Here we build on previous work, both our own and others’, in order to get a firmer grasp on precisely what features of investigatory networks work with what features of the question itself in order to produce epistemic success.

Costs and benefits of cognitive division of labour and epistemic networking: An agent-based simulation study
Rainer Hegselmann, University of Bayreuth

In my contribution I’ll present a simulator that allows to study the interplay of several mechanisms that play an important role in the dynamics of science and the spreading of scientific knowledge in a society.

Key components are:

1. Agents modify their opinion on a certain topic in a social exchange process.
2. A true opinion exists. Some of the agents are looking for the truth and to a certain degree they are successful in their quest for the truth. Some are better than others. Some are not interested at all in the truth.
3. The agents belong to all sorts of different epistemic groups that are net- working all the time. The social exchange process is confined to those others which are socially not too far away in terms of network distance in the net- works that emerges from the ongoing networking.

In the following I give some more details about the three components and their interplay.

Ad 1: In the original and simple version of the BC-model as published in Hegselmann & Krause (2002) we assumed a 1-dimensional, real valued opinion space for a finite set of agents and a discrete time. Let xi(t) be the opinion of agent i at time t. All agents have a confidence interval ε, which is the same for all agents and constant over time. All agents i take seriously all and only those other agents j whose actual opinions are not further away than ε (the confidence interval). All agents simultaneously update their opinion by averaging upon all opinions within their confidence interval. – Thus, a dynamical system models the social exchange process. Obviously there are alternatives to the social exchange process (SEP) as it is modelled in the BC-model, for instance a dynamics driven by the linear De Groot dynamics. Let SEPi   be the resulting opinion of agent i in period t +1 under one such dynamics.

Ad 2: Let T be the true value in the assumed opinion space, for instance the unit interval [0,1]. Then a more or less successful quest for the truth, embedded in a social exchange process can be modelled by the convex combination

xi(t +1) = αi ⋅ T + (1 – αi) ⋅ SEPi   with (0 ≤ αi  ≤ 1)

In the equation αi controls the success of agent i in the quest for the truth. Combining simulations and analytical means Hegselmann & Krause (2006) analysed in detail the resulting dynamics.  One of the central results was that for a whole society to end up at the truth, it is not necessary that all their members are truth seekers.

Ad 3: Epistemic agents, belonging to different epistemic groups, are networking all the time: Truth seekers may try to get close to other truth seekers while at the same time keeping distance to agents that are not interested in the truth. Some exceptionally good truth seekers might try to cluster among themselves, while others try to 'enlighten' the non- or not so good truth seekers. - These and other types of processes are modelled as migration processes on graphs or grids (retangular, hexagonal or irregular) that are generalised versions of models as proposed by James Sakoda and Thomas Schelling decades ago. The conceptual structure of that generalisation is described in Hegselmann (2012).

Taking (1) - (3) together we get a complicated and interwoven social-epistemic network dynamics. The contribution will make use of ENSIM, a simulator that allows to experiment with these processes. The simulator especially allows to analyse cost and benefits of epistemic networking, measured in terms of distance to the truth.

Correcting Publication Bias in Medicine: The Power and Limits of Prestige
Carole Lee, University of Washington – Seattle

Peer reviewed publication provides a system of incentives motivating scientists to formulate new projects and disclose findings in exchange for peer recognition.  However, the incentive to publish has increasingly motivated dysfunctional forms of gamesmanship by individual scientists.  A particularly pernicious form of gaming behavior – namely, publication bias in academic medicine – suggests that, despite efforts by journal editors to close loopholes, they have succeeded and failed in ways that demonstrate the limited effectiveness of peer-reviewed publication – the traditional means for allocating prestige and motivating research within science – to self-govern scientific communication and disclosure.

The Evolution of Consensus Conferences
Miriam Solomon, Temple University

Consensus conferences began in the mid-1970s at the US National Institutes of Health.  They were modeled, in part, on Arthur Kantrowitz’s 1967 idea of “science court” and designed to resolve scientific controversy through group deliberation.  NIH consensus conferences were well received and widely imitated and adapted in the US and overseas.  Consensus conference programs were modified in response to both epistemic concerns and local circumstances.  One of these modifications, called the “Danish model” became paradigmatic and was imported back to the US in the late 1990s as a method for facilitating public participation in science.  This paper argues that consensus conferences are social epistemic rituals.  They claim to “make knowledge” through satisfying ideals of fairness and objectivity.  The assumptions that lie behind such claims deserve critical examination.  Examination of the evolution of consensus conferences helps reveal these assumptions and provides a necessary background for normative assessments and future recommendations. 

Herding and the Quest for Credit
Michael Strevens, New York University

The system for awarding credit in science -- the "priority rule" -- functions, I have proposed elsewhere, to bring about something close to a socially optimal distribution of scientists among scientific research programs. If all goes well, then, potentially fruitful new ideas will be explored, unpromising ideas will be ignored, and fashionable but oversubscribed ideas will be deprived of further resources. Against this optimistic background, the present paper investigates the ways in which things might not go so well, that is, ways in which the priority rule might fail to realize its full potential as an incentive for scientists to work on the right things. Several possible causes of "herding" -- an outcome in which a single research program ends up with a number of researchers well in excess of the optimum -- are considered.

Conservatism and the Scientific State of Nature
Kevin Zollman, Carnegie Mellon University