Guidelines for SoTL Research

Guidelines for Human Research Participants in Scholarship of Teaching and Learning Research

UWS Leadership Site for the Scholarship of Teaching and Learning

December 2007
Renee A. Meyers, Coordinator

Researchers across all disciplines conduct Scholarship of Teaching and Learning (SoTL) research. However, because SoTL researchers in some disciplines are unaccustomed to needing IRB approval for their disciplinary research (i.e., it does not involve human participants), there has been ambiguity among these SoTL researchers regarding the IRB process, and similar ambiguity among IRB evaluators as to what review category best fits SoTL research. This document is meant as a guideline for those doing Scholarship of Teaching and Learning (SoTL) research, and for those judging its status in the IRB process (also see McKinney, 2007 for additional information). It was not written to dictate decisions or actions. Instead it was written to provide additional information on research practices that may be unfamiliar to many, and to help guide researchers and IRB decision makers on SoTL work. This document was not produced solely by the author. Faculty from across the UW-System in many disciplines including Psychology, English, Education, Political Science, Biology, Information Studies, and Communication all contributed to its content (please see the list of contributors at the end of the document). It is hoped that this document will answer questions, provide information, and clarify ambiguities about the goals and objectives of SoTL research. With that dictate in mind, this whitepaper begins by providing a brief description of SoTL research. Second, it discusses the intersection of IRB review and SoTL research. Third, it offers a set of guidelines for protecting students and their work. Finally, it provides some references and a list of the contributors who shaped this document.

Scholarship of Teaching and Learning Research

Scholarship of Teaching and Learning (SoTL) researchers seek to advance the practice of teaching and learning through scholarly inquiry into student learning. They are interested in systematically investigating questions that improve our understanding of how students learn, and focus on building practitioner knowledge about the practice of teaching. This research is most often conducted in established educational settings (i.e., the classroom) and revolves around normal educational practice. The research process typically involves surveying, interviewing, questioning, or observing students in their own classroom setting. This research often involves a comparison of various instructional techniques, learning tools, curriculum materials, or classroom teaching strategies. It may also include analysis of student papers or assignments. Regardless of the research strategies or techniques, however, the core focus of SoTL research is to understand how our students learn so as to improve instructional practices. In addition, presentation and publication of SoTL research is vital for keeping instructors up-to-date on the latest theory and practices, and to establish a literature so that new research always builds on previous research.

Examples of the types of questions that SoTL researchers might pursue include:

  • How can an instructor help students learn to be more critical or reflective thinkers?
  • How can an instructor help students learn how to use feedback on their assignments so as to make the subsequent assignments better?
  • How do students view group participation, and how does that impact their learning?
  • How can instructors help students learn to think creatively?
  • How do students draw on their prior knowledge to learn about new information or ideas?

As is evident from this sample of questions, SoTL researchers are focused on systematically investigating how their students learn. The results of these investigations provide researchers with information about how to revise their current instructional practices so as to improve student learning outcomes.

SoTL Research and IRB Procedures

In general, inquiry into teaching and learning questions (the central concern of Scholarship of Teaching and Learning (SoTL) research), should be exempt (in contrast to minimal risk or risk) as a review category under Title 45 Code of Federal Regulations Part 46 (aka the Common Rule). SoTL research is typically exempt as a review category because it does not disrupt or manipulate subjects' normal life experiences, incorporate any form of intrusive procedures, or identify students in such a way that it poses a risk to them. However, if you use students under 18 years of age as participants in your research project, you must move from the exempt category of review to a more extensive review. Consultation with the IRB is helpful if you are using underage participants.

The exempt review category does NOT mean that your research is 'exempt from review by IRB.' It means 'exempt from full board review.' All researchers who are working with human participants MUST still file a protocol and receive approval. This includes funded research as well, and now funders are increasingly concerned with IRB approval before funds are dispensed. In short, exempt means only that the protocol is reviewed (typically) by the IRB chair or staff, and not by the full board.

The other categories of review include expedited review and full board review. Research under the expedited review category typically is considered minimal risk, which means that the probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those originally encountered in daily life or during the performance of routine physical or psychological examinations or tests (45 CFR 46, 102). Minimal risk protocols are often reviewed in an expedited manner by a subcommittee of the full board. An example of when the minimal risk category might apply to SoTL research would be if the researcher was collecting data from voice, video, digital, or image recordings for research purposes.

The risk category requires a full board review. Generally, SoTL research will not fall into this higher "risk" category. This may include research on sensitive or protected populations (e.g., minors, prisoners, fetuses, etc.), or research that results in more than minimal risks for the participants, or research that involves intentional deception of the participants. In addition, failing to fully inform subjects is deceptive, whether or not it is intentional. The federal regulations include 8 components of informed consent. For more information, visit the U.S. Department of Health and Human Services Human Subjects webpage. Remember that students cannot give "informed" consent unless they are fully informed.

Keeping these caveats in mind, most SoTL work will fall into one (or a combination) of the following commonly recognized exempt review categories:

  1. Exemption for education. Research conducted in established or commonly accepted educational settings, involving normal educational practices, such as (a) research on regular or special education instructional strategies, or (b) research on the effectiveness of or the comparison among instructional techniques, curricula, or classroom management methods is exempted.
  2. Exemption for research involving educational tests. Research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement) is exempted, unless (a) information obtained is recorded in such a manner that human subjects can be identified directly or through identifiers linked to the subjects; and (c) any disclosure of the subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or reputation.
  3. Exemption for survey or interview procedures. Research involving survey or interview procedures is exempted unless (a) information obtained is recorded in such a manner that human subjects can be identified directly or through identifiers linked to the subjects; and (c) any disclosure of the subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or reputation.
  4. Exemption for research involving observation of public behavior. Research involving observation is exempted unless (a) information obtained is recorded in such a manner that human subjects can be identified directly or through identifiers linked to the subjects; and (c) any disclosure of the subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or reputation.
  5. Research for collection or study of existing data. Research involving the collection or study of existing data, documents, records, pathological specimens, or diagnostic specimens is exempted, if these sources are publicly available or if the information is recorded by the investigator in such a manner that subjects cannot be identified directly or through identifiers linked to the subjects.

As is evident from the sample of questions listed previously, and from the definition of SoTL research provided, it is likely that most SoTL research will fall into the exempt review category. Most SoTL research is conducted in "established or commonly accepted educational settings, involving normal educational practices," typically involves "research on the effectiveness of, or the comparison among, instructional techniques, curricula, or classroom management methods," and involves surveys, interviews, observations, or study of existing data.

The question of whether participants can be identified directly or through identifiers linked to the participants is sometimes a difficult call for the researcher and the IRB decision-makers alike. If a student writes about Wal-Mart in a paper, and everyone in the class knows that this student wrote about Wal-Mart, and as a researcher you want to use parts of this paper in a journal article that you are writing, is that student identifiable? The question of identification also revolves around whether this disclosure of the participants' responses outside the research could reasonably place the participants at risk of criminal or civil liability or be damaging to the participants' financial standing, employability, or reputation. In this case, it is probably unlikely that the student would be identifiable by the readership, and maybe even his/her fellow students, but more importantly, it is unlikely that making this paper public would place the student at risk unless the student works at Wal-Mart and is highly critical of Wal-Mart practices.

Certainly, it is incumbent upon the researcher to make the research participants' evidence that s/he has collected as anonymous as possible before making it public. In the above example, the researcher may well want to black out the title of the organization (i.e., Wal-Mart), or in some way make it impossible for the reader to identify the organization (if possible). In addition, it is important that the researcher explain very clearly to the participants in the Informed Consent form exactly what aspects of their research participation will potentially be made public so that when the students sign the form they understand how their work will be used. Furthermore, it is vital that the instructor very clearly delineate to the students the difference between regular assignments and those assignments that will be used for research, and to be sure that the students realize that they are consenting to the research (above and beyond just doing the assignments). Finally, it is important to remember that researchers do NOT have access to students' confidential data (e.g., overall GPA) without requesting the students' permission to use that data.

In short, if the Informed Consent form is clear and explicit, and the researcher takes every precaution to present/publish the student data in an ethical and confidential manner, then the likelihood of identification and harm are greatly reduced. The IRB decision-makers can often be helpful in guiding researchers in ways to make their data public in both confidential and ethical formats.

Guidelines for Protecting Students and Their Work

Clearly, SoTL researchers must keep student participants' information confidential. The following requirements and recommendations can be useful when preparing the IRB form:

  • Instructors must ask for "informed consent" from the student research participants, and must allow any participant to withdraw from the study at any time. Remember that only those who are 18 years of age or older can give informed consent. If you are using underage participants in your research, it would be wise to consult with your IRB. Informed consent forms are available with the campus IRB materials.
  • Instructors should be careful to exclude all identifiers on student surveys or questionnaires, or other collected evidence, as much as possible.
  • Student papers or assignments that are assessed for research purposes should not be analyzed until after final grades have been posted (if at all possible), and are typically rendered confidential by removing any identifiers before analysis, or having someone other than the instructor of record conduct the analysis.
  • If respondents are to be tracked over time (i.e., repeated measures are needed on each student) an instructor might ask students to create an ID that only the student knows, and to use that same ID on any data that will be collected for the repeated measures analysis. The instructor would not be privy to this ID, but if s/he were afraid that the students might lose this ID or forget where they wrote it down, s/he might have another person outside the class collect the ID information on a sheet of paper, and keep it in privacy (only to be retrieved if a student forgot his/her ID).
  • Sometimes (if the analysis involves student course work) it is useful for the informed consent forms to be collected by someone other than the instructor of record and put into a sealed envelope that is not opened until after semester grades have been filed. In this way, the instructor does not know who has (has not) consented to participate until after the class is completed.
  • The instructor might not ask for consent until the end of the semester when the student can be a better judge of whether s/he wants to allow the instructor to use his/her assignments. Again, these forms are best collected by a third party, and only examined once final grades have been turned in.
  • The instructor may request consent after the semester is over (although this is far more difficult to do).
  • If data is collected in a D2L chat room or discussion board, get informed consent forms from the students, and then indicate to the students when they first sign on that all interactions in the chat room/discussion board will be used for research. Make it clear that participating implies consent. If a student does not consent to having his/her chat room comments included in the research project, make it possible for the student to complete this assignment in another way.
  • Additionally, you might ask students to choose an anonymous username when participating in a chat room or D2L discussion board where interaction data is being collected.

In general, it is unlikely that disclosure of information contained in student assignments will "reasonably place the students at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or reputation." Most student work done in the classroom is not of this nature. Additionally, much SoTL research that is quantitative in nature is published or presented in aggregate form so that individual students are not identifiable. However, SoTL research of a more qualitative nature that analyzes student papers or student discussions, and then uses excerpts or quotes from these student products in publications or public presentations, cannot present in aggregate form. But the researcher can take precautions to remove from these excerpts or quotes obvious student identifiers (name, class, year, etc.), to choose (as much as possible) excerpts or quotes that are not directly linked to students or that pose no risk to the students, and to present/publish the student evidence in the least identifiable form possible. In addition, the researcher can ask on the Informed Consent form whether the student is willing to be cited, and if so, whether they want to be identified or remain anonymous (for a sample IRB Informed Consent Form compiled by Blaine Peden that addresses this concern, email pedenbf@uwec.edu). In the event that students are willing to allow their names to be published with their work, these issues become moot. But in these situations, the researcher must be very clear with the students in the Informed Consent process to make sure that that the students understand what aspects of their work will be made public, how it will be made public, and what that might mean for them.

Certainly, there are exceptions to all rules, but generally SoTL research fits the profile of exempt research under the Federal guidelines. The educational nature of this work, its focus on student learning, and the rare chance that its public disclosure will place the student at risk, all add up to a preferred determination of exempt status. However, it is always advisable to consult with your IRB board members in the process of preparing your application. They can be invaluable resources.

Finally, this document is a work-in-progress, so if you have additional materials that you think should be included, or would like to make suggestions for revisions, please do send those to . Thanks for any help you can provide.

Relevant References

Burman, M. E., & Kleinsasser, A. (2004). Ethical guidelines for use of student work: Moving from teaching's invisibility to inquiry's visibility in the scholarship of teaching and learning. The Journal of General Education, 53 (1), 59-79.
Hutchings, P. (Ed.). (2002). Ethics of inquiry: Issues in the scholarship of teaching and learning. Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching.
McKinney, K. (2007). Enhancing learning through the scholarship of teaching and learning: The challenges and joys of juggling. Bolton, MA: Anker Pub.

Consulting Faculty

  • Elizabeth Buchanan, UWM School of Information Studies, past IRB board member
  • Bill Campbell, UW-River Falls, Director, Grants & Research
  • Bill Cerbin, UW-LaCrosse, Professor of Psychology, Carnegie SoTL Scholar
  • Nancy Chick, UW Colleges, Associate Professor of English
  • Regan Gurung, UW-Green Bay, Associate Professor of Psychology, Chair of International Society of Scholarship of Teaching and Learning Human Ethics Subcommittee.
  • Doug Johnson, UW-River Falls, Interim Associate Vice Chancellor for Academic Affairs
  • John Koker, UW-Oshkosh, Dean, College of Letters and Sciences, Professor of Mathematics
  • Katina Lazarides, Communication and Project Specialist, UW System Leadership Site for the Scholarship of Teaching and Learning
  • Blaine Peden, UW-Eau Claire, Associate Professor of Psychology
  • Linda Tollefsrud, UW-Barron County, Professor of Psychology, IRB board member