Appendix I
Introduction
Surveys as a Tool for Training in Scientific Integrity
Michael W. Kalichman
Scientific integrity is about more than rules, regulation, and compliance. Much of what we do as scientists requires decisions that must be made in the absence of clear guidelines. Questions about topics such as data management, publication, and the use of animal or human subjects often represent difficult ethical challenges. To learn about such concepts, which are frequently complex and nuanced, it is essential to have the chance to think actively rather than merely listen to a lecture or read some text. Grappling with tough cases through discussion is one common approach to stimulate an active learning process. Another approach is to use surveys as a tool for promoting discussion.
Surveys, like the scenarios used for case study discussion, require trainees to examine their own perceptions and assumptions. Through the process of this reflection, it is possible to refine existing standards, identify new standards, and develop strategies for responding to difficult questions. The characteristic that distinguishes surveys from case study discussions is that answers are not typically open-ended. Instead, the respondents are asked to answer forced choice questions that are either categorical (e.g. yes/no) or quantitative (e.g. the degree of agreement or disagreement). These answers then can be reduced and summarized for discussion of patterns and correlations within a particular group (e.g. this years students) or between groups (e.g. those who have had vs. those who have not had training in scientific integrity). Although the subtleties in complex cases may not always emerge in discussion, it is often possible with surveys to elicit information about common trends and attitudes that would otherwise be lost. The following includes both general observations about the use of the sample surveys and some specific comments about the use of each.
The surveys included below parallel the topic areas of this text. The following points should be kept in mind when employing them as teaching tools. First, because some surveys overlap, their selection is at the discretion of the instructor. Further, not all surveys will be appropriate to meet the needs of a specific course, instructor, or group of students. Second, these surveys should not be viewed as definitive; instructors may want to develop new surveys to meet specific instructional objectives. Third, nearly all of these surveys are suitable for administration during class or a workshop, but some may be more appropriate as homework assignments. For purposes of homework or distribution, these forms can be found as PDF files at this books web site, www.scientificintegrity.net, and printed for convenient use. Fourth, simply completing these surveys can have value in stimulating reflection on personal values and the normative conduct of science. However, analysis and discussion of survey results is a key part of this exercise. The instructor could do the analyses, but it may be even more valuable to have trainees summarize the data, select results of interest, present their findings, and lead class discussion about interesting results. Usually, it is not necessary for the survey discussants to focus on the responses to each and every item in the survey. Instead, identifying questions that reveal differing attitudes and perceptions on the part of the respondents is desirable. These should be used to stimulate class discussion, allowing the discussants to state their positions and the rationales underlying their responses. Such discussions allow students to invoke their critical thinking skills in articulating their arguments. Equally important, these discussions frequently uncover multiple points of view, many of which have merit and can be appropriately defended.
Authors note: Using surveys to collect information may fall into the category of human subject research. In such cases, Institutional Review Board (IRB) approval must be sought before any work is begun (see chapter 5). The definition of human subjects research centers on the fulfillment of criteria related to the subjects and to the investigational process and goals. Human subjects are defined as living individuals about whom an investigator obtains: (a) information, specimens, or other data through intervention or interaction with the individual, or (b) identifiable private information. The word research is meant to encompass systematic investigation designed to develop or contribute to generalizable knowledge. If you use the surveys in this Appendix as tools for stimulating class discussion, and nothing more, your actions do not constitute contributing to generalizable knowledge. Under such conditions administering these surveys and presenting the resulting data for purposes of discussion does not constitute research. However, the authors of this appendix and this book encourage users to check with their institutions review boards to verify whether such use is exempt, can be expedited for review, or requires full IRB review. If your use of these surveys extends beyond the immediate purpose of classroom instruction, it is likely that IRB review and approval would be needed.
Survey Descriptions
Survey 1: Overview (Chapters 1 and 2)
This survey is modified from one originally used to study perceptions about research misconduct at the University of California, San Diego (2). This survey has potential value as an introductory exercise for a course in scientific integrity. Ideally, students would be asked to complete the questionnaire immediately before or at the beginning of a workshop or the first meeting of the course. In addition to the raw data being of interest (e.g. what percent of respondents believe they have firsthand knowledge of plagiarism), secondary analyses and discussion about the meaning of the answers is at least as important. Examples of specific analyses that might be of interest are: (a) the correlation between position (question #1), years of experience (#2), or experience as an author (#3) and the answers to questions about misconduct experience (#4 11) or (b) discussion of the various possible interpretations of, for example, 10% of respondents reporting firsthand knowledge of data fabrication or falsification. It can also be of interest to compare results of this survey to those that are published (1, 2).
Survey 2: Research Misconduct (Chapters 1 and 2)
The primary focus of this survey is research misconduct as defined by federal regulatory agencies to include fabrication, falsification, and plagiarism. Some areas that may lend themselves to fruitful discussion include distinctions between different kinds of data falsification (#2 vs. #3), different types of plagiarism (#4 6), personal willingness to commit possible misconduct (#7 9), responsibilities for whistleblowing (#10 12), and allocation of blame vs. punishment (#13, 14).
Survey 3: Mentoring (Chapter 3)
The results of this survey can provide the basis for discussions on the responsibilities of mentoring. All of these questions address potential roles for mentors, heads of research groups, and/or thesis supervisors. One possible use of this survey is to have it completed by both students and their thesis supervisors, followed by a presentation and discussion of the results in class.
Survey 4: Publication (Chapter 4)
The initial two questions are based on a brief scenario that potentially distinguishes between knowing what one should do and the willingness to do so. The following questions address two key areas: (1) the reasons for publishing a paper (#3 7) and (2) a variety of publication practices that may be viewed as more or less acceptable (#8 13). For both sets of questions, interesting discussions can result from identifying relative differences and considering possible rationales for those differences.
Survey 5: Authorship (Chapter 4)
This is the second of three surveys on the topic of publication. The opening questions focus on a junior scientist's dilemma about adding a senior scientist's name to the list of authors on a forthcoming publication. The remaining questions highlight both what should be criteria for authorship (#3 9) and what should not be criteria for authorship (#10 15). Typically, these questions will readily reveal a wide range of views about how authorship should be defined.
Survey 6: Peer Review (Chapter 4)
This survey is specifically designed as a homework assignment. The goals are threefold. First, trainees are asked to think about the practice of a manuscript reviewer asking that one of her/his postdocs review a manuscript (#1, 2). Second, trainees are asked to discuss related questions with at least one active investigator (#3 6) as well as offer their own views (#7 10). Third, the trainees are asked to think about when it would not be appropriate to accept an assignment for manuscript review (#11 13).
Survey 7: Human Subjects (Chapter 5)
Research with human subjects is distinguished by the obligation to consider and protect the interests of those who have volunteered to be in a research study. The initial questions in this survey (#1 7) focus on the role and domain of the Institutional Review Board. The remaining questions address the circumstances under which potential research subjects might enroll in a research study (#8 12) as well as whether a study should be approved (#13) or stopped early (#14). Discussions of these questions are likely to reveal perceptions and attitudes that are even mutually incompatible, although all are potentially acceptable under current regulations.
Survey 8: Animal Subjects (Chapter 6)
Although there are many specific regulatory controls for the use of animals in research, this remains an area about which the public and even the biomedical science community are sharply divided. The initial questions (#1 5) consider the role of the Institutional Animal Care and Use Committee in reviewing research with animal subjects. The remainder of the survey form (#6 17) is designed to encourage trainees to think about their personal criteria for accepting or rejecting the use of animals in research. The survey can be completed in class, but it may be useful as a homework assignment to allow for more thoughtful consideration. For the purposes of analysis, it should be of value to compare the relative importance placed on species (where is the cut-off and why?), the adverse consequences of the experiment (pain, distress or discomfort), and the utilitarian value of the studies (increased understanding of the mechanisms of cancer vs. cosmetic safety). It is to be expected that opinions will vary widely, even among scientists who use animals in their research.
Survey 9: Conflicts of Interest (Chapter 7)
Financial conflicts of interest in academia and science have become a matter of serious concern in recent years. The purpose of this survey is to explore the nature of disclosure of conflicts of interest (#1, 2), the reasons that disclosure might be seen as important (#3 5), the extent to which we should worry about non-financial conflicts of interest (#6 9), and possible protections from bias due to conflicts of interest (#10 13). Discussion of these issues will typically help to sharpen rationales about what we are worried about and what solutions are most likely to have an impact.
Survey 10: Collaboration (Chapter 8)
This survey deals with a defining characteristic of modern science: collaborations. The very first question (#1) asks for student perspectives on the importance of collaboration to good research. This is followed by several questions (#2 7) about the risk of problems in collaborations. The remaining questions (#8 13) list issues that might be seen as important to be addressed before initiating a collaboration. Discussions of these questions are likely to reveal that scientists unfortunately tend to not think about these issues until they become a problem.
Survey 11: Data Ownership (Chapter 9)
This survey opens with a case discussion scenario to address some possible reasons for ambivalence about sharing of research data (questions #1 5). Questions about ownership and sharing of data (#6 21) should serve to illustrate the importance of these issues and some of the dilemmas faced by researchers.
Survey 12: Genetic Technology (Chapter 10)
The application of advances in molecular genetics has created ethical challenges as well as fears related to the misuse of these technologies. The survey opens with some questions (#1 3) about what people would and would not want to know about their possible future. It is likely that these questions will lead to sharp divisions of opinion during class discussion. This division of opinion then raises further questions about who should give consent for research studies involving searches for genetic predispositions (#4 8). The difficulty of such questions is further exacerbated when a study risks providing information about a particular behavioral trait in a given racial group (#9 11).
Survey 13: Recordkeeping (Chapter 11)
Although most scientists had the experience of an early lab course in which rigorous standards were proposed for keeping a lab notebook, it is not necessarily the case that these standards for recordkeeping are common practice. The first questions on this survey (#1 9) offer a variety of possible characterizations of how research records should be kept. The focus of the remaining questions (#10 13) is the disposition of those records following completion of the project. Discussion of the answers to these questions will hopefully reveal the variation in common practice, but also encourage a shift toward improved recordkeeping.
References
1. Eastwood S, Derish P, Leash E, Ordway S (1996). Ethical issues in biomedical research: perceptions and practices of postdoctoral research fellows responding to a survey. Science and Engineering Ethics 2: 89‑114.
2. Kalichman MW, Friedman PJ (1992). A pilot study of biomedical trainees' perceptions concerning research ethics. Academic Medicine 67: 769‑775.