IRB: Ethics & Human Research

Reformed Consent: Adapting to New Media and Research Participant Preferences

The principle of respect for persons clearly demands that investigators communicate with potential research participants in a way that fosters comprehension of the information relevant to deciding whether to enroll in a particular study.1 Federal regulations governing research with humans require documentation of consent with the participant’s signature on a printed consent form, but such documentation should not be confused with the consent process itself.2 As noted in a relevant report from the Institute of Medicine,

traditional informed consent often does not appropriately inform and empower the participant, because the information in the consent document increasingly serves institutional rather than participant needs . . . consent forms have been hijacked as ‘disclosure documents’ for the risk management purposes of research organizations.3

A notable side effect of using research consent forms as risk-management tools is that their average length has steadily increased over the past few decades,4 despite longstanding evidence that increased document length hinders participant comprehension of key information.5

Because of their statutorily mandated status, signed printed consent forms are likely to remain a component of the consent process for the foreseeable future. However, the use of printed forms does not preclude the use of additional materials during the consent process to foster more effective communication of relevant information. Multimedia learning tools are potential aids for presenting this information, and suggestions to add video or other multimedia tools to the consent process have been made for at least three decades.6 However, their potential utility has yet to translate into widespread use. One barrier to wider use of multimedia consent aides has been the relative paucity of data from methodologically rigorous and conceptually grounded studies about the effectiveness of these type of consent tools. In this article, we describe the reactions and preferences of laypersons who went through a multimedia-aided consent process for a hypothetical clinical trial.

Multimedia Consent Aids

We reported elsewhere about what to our knowledge remains the only large-scale randomized controlled trial of the effectiveness of a multimedia-aided consent procedure.7 This trial involved participants with serious mental illness, and the multimedia consent aid was a DVD specifically designed to capitalize on established principles from multimedia learning theory.8 All participants in the study had a DSM-IV-TR diagnosis of schizophrenia. Comprehension of the disclosed consent information—as measured with the MacArthur Competence Assessment Tool for Clinical Research9 —was significantly better among participants randomized to the DVD-aided consent process compared to those randomized to the routine consent process.

Other studies of multimedia tools in the consent process have produced mixed results. In their review of randomized controlled studies using audiovisual aides for research consent published between 1966 and 2006, Ryan et al.10 found only four studies meeting their inclusion criteria.11 The studies varied in the nature of the audiovisual consent intervention, study population, and conceptual and methodologic rigor, and their results were inconclusive. A broader review of the literature on computer and multimedia research consent aides was provided in a widely cited report by Flory and Emanuel.12 They reviewed 12 trials of video- or computer-aided consent, though two of the trials should more appropriately be categorized as phases within a single study and were published as such,13 and four others were actually different arms of two studies.14 Thus, the number of available independent studies Flory and Emanuel examined was as few as nine. Three of the nine studies found that computer-based or video tools had beneficial effects on participant comprehension.15 A fourth yielded mixed results depending on the specific comparison examined,16 but patients receiving multimedia consent (video or computer) as a combined group had better understanding than those receiving text-based disclosure (standard or booklet format) as a combined group. Two additional studies found no benefits in regard to initial comprehension, but those receiving computer- or video-aided consent did show better retention.17 In one of these,18 the video did not provide information about the details of a particular clinical trial, but rather involved a videotaped, talk-show format discussion between experts and audience members about HIV vaccine trials that was used as a supplement to the standard consent document. The other study, which revealed no differences in initial comprehension but better retention,19 involved a highly educated sample of pregnant women who demonstrated good comprehension of disclosed information regardless of consent format.

In another study with seemingly negative results, Campbell et al.20 found no significant benefits of video- or computer-aided consent over standard printed consent forms. However, the investigators intentionally minimized interactive discussion/explanations between research staff and participants (the staff were instructed not to give additional information or to respond to possible questions about the material). Moreover, the authors pointed out that “it may be easier for attention to wander when one is passively watching a video.”21 Thus, rather than showing the lack of efficacy for multimedia consent tools, these findings underline the point that multimedia tools should not replace the interpersonal consent interaction and discussion between researchers and participants but rather may be useful methods to facilitate that discussion.22

The ninth independent study included in Flory and Emanuel’s review was reported as having negative findings.23 Yet the enhanced consent process in that study did not actually involve multimedia aids. The experimental enhancement involved presentation of exactly the same text as in the paper consent form (accompanied by an audiotape of a person reading the printed consent aloud), but the text was presented on a computer screen organized in hierarchical tree menus and submenus linked via “hot buttons.” For example, to read the text on side effects, the participant had to click on a heading that was titled “Possible Side Effects.” There is no reason to expect consent text presented on a computer screen to be superior to a printed document, unless the computer software (or other multimedia method) is used in a way that compensates for known strengths of that technology and/or known deficits in standard presentation modes.

Because of the limited number of methodologically sound and conceptually rigorous relevant studies, we agree with the general conclusion of Ryan et al.: that the empirical literature is not yet sufficiently developed to draw definitive conclusions one way or the other about the general effectiveness of or value derived from multimedia consent aids. We think it very likely that multimedia consent aides can have considerable benefit when developed and applied specifically in reference to established multimedia learning principles24 and when used to communicate those aspects of the relevant information for which multimedia presentation has clear advantages over printed text (e.g., some concepts, such as risk probabilities, are better understood when communicated pictorially than via text alone25 ). But replication and additional methodologically rigorous and conceptually grounded research is needed. In many cases, the computer or multimedia consent tools tested in prior research seem to have been developed and implemented without consideration of what these tools can provide beyond the practice of simply discussing and reviewing a printed consent form.

There are many ways to further develop and evaluate the potential utility of multimedia consent tools. One key component in developing better multimedia consent aids (and informed consent procedures in general) is to seek and incorporate input from potential research participants and other stakeholders about their preferences and opinions regarding existing and potential consent materials or procedures. A recent review of studies by Nilsen et al.26 provided evidence supporting this, finding that consumer input when developing patient information materials improved the relevance and comprehensibility of the information. Citing that prior review, Ryan et al. specifically recommended consumer/stakeholder input as a “potentially fruitful avenue for improving research on the use of audio-visual interventions during the informed consent process.”27 Further, as noted by Dubois, seeking input from community members and other stakeholders is a “form of respecting autonomy, or the self-determining and rational dimension of people . . . [and] a form of justice, a way of acknowledging the equal worth of participants by giving them input into how research is conducted.”28

To our knowledge, only one published study has focused on stakeholder “needs assessment” in the research context. Jimison et al.29 conducted a series of focus groups with key stakeholders, including prior research participants with breast cancer or serious mental health conditions (schizophrenia or depression) and researchers or research oversight professionals. The limitations of printed consent forms were widely acknowledged across the stakeholder groups, and their response to multimedia consent materials was generally quite positive. Although the stakeholder groups in the Jimison et al. study were clearly among those from whom input is warranted, we believe it may also be informative to seek opinions about multimedia consent from a broader array of laypersons, i.e., those not specifically identified with a particular serious neuropsychiatric or other medical illness. Thus, using the multimedia materials prepared for our earlier consent study,30 we conducted a qualitative study focused on general reactions and suggestions for multimedia consent from a diverse sample of laypeople. Specific aims for the present study were: 1) determining participants’ overall reactions and preferences regarding multimedia-aided consent; 2) eliciting participant’s opinions about the importance, relevance, or desirability of the specific information that was or was not included in the consent disclosure; and 3) identifying the types of sources other than consent forms that research participants wish to consult when making enrollment decisions.

Study Methods

Thirty adult laypersons participated. We recruited participants through a university-based research center using flyers posted around the campus and via word of mouth. To be included in the study participants had to be 18 years or older and fluent in English. The present sample was recruited and evaluated independently from the prior study of DVD-aided consent;31 none of the present participants or data were included in that study.

Multimedia-Aided Consent.

The multimedia-aided consent procedure and materials were identical to those employed in our earlier study.32 Participants were provided with a consent procedure for a high-risk and high-complexity hypothetical clinical trial of an experimental cognitive enhancer medication. We chose a cognitive enhancer as the hypothetical experimental compound as such drugs may potentially improve cognition even in apparently healthy participants with mild or subclinical cognitive impairment due to a variety of causes, including normal aging. We also felt a high-risk study was appropriate as nonpatients are often involved in early-phase clinical trials of relatively high potential risk. The consent materials included a DVD and a printed consent form, each of which contained the federally mandated elements for consent documents,33 as well as those required by our institutional review board (IRB). The materials were also reviewed by each of the study authors, one of whom (MPC) directs our IRB, and a second (BWP) who has served on the IRB for the past five years; both coauthors reviewed the DVD for compliance with federal regulations governing research with humans and found it contained all the required elements of informed consent.

The participants met individually with a trained staff member who administered the DVD-aided consent procedure. The DVD was presented on a laptop computer screen connected to speakers for audio, and provided a narrated voiceover description of the hypothetical drug trial. In addition to the orally presented material from the narrator, key information was provided visually using summary text highlighting key information in bullet-point format, animated sequences to illustrate key concepts (such as randomized assignment), and short video clips of study procedures (e.g., videos and audios depicting brain MRI procedures, neuropsychological testing, and a blood draw). The research staff member remained with and assisted each participant as he or she watched the 20-minute DVD. In accord with the DVD’s role as consent aid—rather than replacement for an interpersonal consent dialogue—participants were encouraged to ask the staff member questions. Each participant was also encouraged to have the staff member stop the DVD and repeat any segments that were unclear. All participants were given a printed consent form describing the hypothetical study that they could refer to at any time during the consent process.

Qualitative Interview.

Following the multimedia consent process, each participant completed a semistructured qualitative interview with the first author. Each interview was audiotaped and lasted between seven and 26 minutes. Using methods similar to those employed in other qualitative studies,34 these interviews employed a funneling approach in which topics were introduced with broad, open-ended questions followed by increasingly more specific questions to ascertain and clarify participants’ opinions and reactions regarding several themes or aspects related to the multimedia consent process. These included: 1) participants’ overall reactions and preferences with respect to multimedia-aided consent, e.g., determining what participants liked, disliked, or felt neutral about concerning the experience of the DVD-aided consent process and its specific format and content; 2) participants’ views about how important specific information was to their decisions to agree or decline to enroll in the hypothetical trial, (e.g., which of the described risks of the study medication were deemed “most important” by the participants); 3) what additional types of information each participant would want to receive; and 4) what additional sources participants would like to consult to obtain such information. Participants were also encouraged to talk about any other ideas they deemed relevant.

A common dilemma in pharmaceutical trials is how to provide the relevant information (for example, regarding side effects) without diminishing comprehension by overloading participants with an exhaustive list of trivial detail about the study.35 One approach to this dilemma is to determine what a “reasonable volunteer” would consider to be relevant information.36 Toward that goal, the qualitative interview also included queries about the experimental medication’s adverse effects. Participants were asked to list the side effects that “caught their attention” and the reason(s) for the salience of those side effects. Also, when participants spontaneously voiced concerns about the specific risks and adverse effects, the interviewer asked follow-up questions to determine more specifically what those concerns were.

Each participant completed a written questionnaire about age, gender, ethnic background, education, prior experience consenting to research participation (including whether the person had ever participated in a clinical drug trial or other medical study) or to treatment, whether the person owned or otherwise had regular access to a computer, and his or her level and frequency of Internet use.

Data Analyses.

Digital audio files, labeled only with the interviewee’s initials, were transcribed by an unaffiliated transcription service. Using grounded theory analytic methods,37 these documents were then coded by project investigators as follows. The first author read through the transcripts and, in consultation with an expert in qualitative research (LP), compiled a list of codes for each interview based on a priori as well as emergent themes. Coded units ranged from phrase length to paragraph length, and each unit could be assigned one or more codes. This code list was discussed with another investigator, and any disagreements in the suitability of codes or code categories were resolved through refinement of the list. In order to determine intercoder reliability, 10 of the 30 interviews were randomly selected and recoded by a second investigator; 504 out of 510 codes (98.8%) were concordant between the two raters (the remaining six were removed from use in study statistics). Codes for each subject were converted into a Microsoft Excel spreadsheet for analysis, using a template methodology outlined by Crabtree and Miller.38 The presence or absence of codes among participants, as well as the context in which they appeared (i.e., in response to what question particular themes emerged), was ascertained and quantified.

Study Results

Fifteen women and 15 men participated in the study. Their ages ranged from 19 to 68 years (mean = 45 years). Eighteen participants were Caucasians, four were African Americans, and six were from other ethnic backgrounds; two participants declined to indicate an ethnic background. Twenty-three of the 30 participants reported having had prior experience as a research subject, including six who had participated in a clinical trial that involved a medical procedure or medication. Twenty subjects recalled at least one specific prior experience reviewing and signing a printed consent form for research or treatment.

Reactions to DVD-Aided Consent Process.

Participants were asked what aspects of the DVD they liked or disliked. Three or more participants made positive comments in the following categories: repetition/summary (n = 10); organization (n = 8); narrator (calm, articulate, believable [n = 7]); informative/understandable (n = 10); focused attention (n = 7); high-yield (important points emphasized, increased retention [n = 11]); appropriate length/pace (n = 9); put at ease/ empowering (n = 5); and role playing/visual format (n = 12). Three or more participants made negative comments in the following categories: infrequent side effects screen (too fast/no voiceover [n = 12]), narrator (monotone, young, slow
[n = 6]), animation (juvenile, unnecessary, dated [n = 4]); music (dated, loud [n = 3]); and editing (stalls, too repetitive [n = 5]).

When queried about their opinions of the DVD and printed consent formats, only one person indicated a preference for the printed consent, 24 indicated a preference for video-aided consent (14 of whom indicated a preference that video and printed consent form both be provided), and five participants indicated no preference. The qualitative interview included no specific questions about the participant’s perceptions of the effect of consent format on likelihood of enrollment. However, the three subjects who spontaneously commented on this point each indicated that he or she would be more likely to enroll in studies that included video in the consent process. (These three people did not specify whether they meant studies with consent processes that used video as a supplement to a printed form, or in place of one.)

Participants were asked what clarifications or further information they would seek from the researcher or the Internet. Requests for further information fell into the general categories listed in Table 1. Commonly cited points of information included inquiry about specific procedural elements, side effects, and commercial sponsorship. Over half of the participants either mentioned that they did not understand a risk or asked for clarification regarding an adverse effect. This confusion was often based on a lack of a definition of side effects. Notably, subjects preferred different sources of information for different types of information. For instance, 14 subjects indicated that they would request clarification about study procedures from the researcher, while only two indicated they would seek such information on the Internet. Similarly, all questions about the intention of the study were directed to the researcher rather than to the Internet (nine versus zero). In contrast, only one participant sought to ask the researcher about study support, whereas 11 subjects sought such information from the Internet. (“Study support” included pharmaceutical company, research organization, and physician credentials.) Of the 18 subjects who sought further information using an online search engine, eight stated that a search input of the specific study drug name would be their first priority.

All the subjects were asked what they would want to know from a past research subject who had participated in a trial of the study drug. Questions fell into three categories: 25 participants asked about personal effects of the study drug (e.g., side-effect duration); 12 requested further information about the subjects’ characteristics (e.g., lifestyle/occupation or specific role in the trial); and 11 participants asked for an “honest assessment” of the experience (e.g., compliance with dosage instructions or whether it was worth enrolling). When queried what past participant responses would increase their likelihood of enrolling in the hypothetical trial, participants mentioned noticeable benefits (15), no or minimal side effects (12), personal similarity to successful past participants (two), and general endorsement of the study (three).

Discussion

In the present study we used qualitative interviews to evaluate impressions about a multimedia-aided consent process among nonpatient volunteers from the general community. Overall, subjects preferred inclusion of multimedia tools in the consent process; 24 of the 30 participants expressed a favorable reaction to the multimedia-aided consent, indicating a preference for a consent procedure involving only multimedia or one combining multimedia and printed consent forms. Only one participant expressed a preference for printed consent forms alone. (Five participants did not indicate a preference.) There was, however, substantial heterogeneity within the sample concerning reactions to specific components of the multimedia materials. A majority of participants expressed interest in accessing the Internet and/or prior participants as additional informational sources outside the researcher-provided disclosure. The present findings support the notion that laypeople value multimedia methods and access to information via means beyond those typically found in the informed consent process for medical research. This is consistent with the reported reactions of various patient groups and researchers or research oversight professionals.39
As ours was a nonepidemiologic sample, it is of course possible that the findings are not generalizable to the broader community of potential research participants. As most of the participants in our study had access to computers and were familiar with the Internet, their opinions may not represent those who do not use computers. Moreover, the fact that 75% of participants were previously research subjects may reflect a higher rate of research participation than is typical in the general population. On the other hand, persons in the general community who are more inclined to participate in research represent a particularly important segment of the population in regard to the aims of this study. Of note, the participation of healthy “professional research subjects” is an issue of ongoing concern that engenders much ethical discussion.40

Even in the computer-literate sample of participants in our study, there was a range of divergent views about specific components of the multimedia procedure. For example, some subjects appreciated the slow and clear articulation of the narrator, while others found it monotonous. Similarly, the animation—intended to help illustrate the chance of receiving placebo—was labeled as “juvenile” by a couple of participants. These differences are likely attributable to a combination of aesthetic preferences, personal associations, learning styles, and educational background.41 With such diversity of opinion, an individually-tailored consent process seems more appropriate than a uniform procedure.

The link between presentation preferences and individual participant needs is not well understood. In this vein, a pertinent direction for future research would be to link information retention with video design preferences and demographic characteristics. Our sample was relatively diverse: it included a wide age range, equal proportions of men and women, and people from many ethnic backgrounds. However, further studies with larger cultural, socioeconomic, and other demographic subgroups are also warranted to determine whether such factors are connected to specific consent preferences. Once reliable and valid methods are established for determining individual consent preferences, it may be possible to match individual participants to specific consent templates based on a simple screening process. Such templates could include contingency plans that are developed before recruiting participants. The potential utility of an even more active, adaptive approach to consent disclosures also warrants consideration.

The value of computer-aided, adaptive training—in which the presentation of information (via text and/or multimedia) actively adapts to the responses and needs of the student—has been well documented within the broader field of education42 but has been virtually ignored as a potential method of enhancing the informed consent process. A number of studies have employed computers in the consent process but have not tailored the presentation to the responses, needs, preferences, and cognitive learning style of the individual.43 On the other hand, print or multimedia materials are aids to consent and should not bear the full responsibility for adapting to the needs and preferences of the individual participant. The importance of dialogue between researchers and potential participants is difficult to overemphasize.44 As is true of printed consent forms, multimedia learning tools are most effective and appropriate when used to facilitate, not to replace, the consent discussion.45

One question that sometimes arises in regard to enhanced consent is what effect it may have on whether people will consent to participate in a study or stay in it over time. Although the effect on rates of participation was not addressed in the present study, in our randomized comparison of the effectiveness of a multimedia-aided versus routine consent procedure,46 we found that over two-thirds of the subjects agreed to participate. These rates were similar for the multimedia vs. routine consent groups. That study used a hypothetical protocol, so we could not investigate the effects on retention, but it is possible that better informed participants will be less likely to experience unexpected developments that might cause them to withdraw from a study.

Another concern involving the use of multimedia aides is the feasibility or cost of developing or adapting multimedia materials to fit the needs of a specific consent process. However, producing DVDs is neither expensive nor complex. The DVDs we employed were produced completely in-house by staff members who knew about filming and editing. Most basic-system desktop computers include DVD-RW (read/write) drives, and the cost of digital video cameras and user-friendly software for high-quality home DVD production and editing is modest. In addition, many of the materials created—for example, modules explaining common concepts such as randomization or common procedures such as MRIs—can then be used in a variety of studies. Thus, as researchers use multimedia aides over successive protocols, they will likely develop libraries of content modules that can be shared, arranged, and employed across a diverse array of studies.

The results of this study also suggest that participants may value access to the Internet to independently obtain additional information about study medication risks and to ask prior participants about their experiences with the research protocol and study medications. Of note, when offered the possibility of Internet access, participants started requesting new types of information. At this point in the interview, the subjects had exhausted all questions for the researcher. Although it is presently not customary to ask potential participants, “Would you like to use the Internet to look up any information?” the possible benefits of such an approach are substantial. The researcher, in addition to providing links to particularly helpful Web sites, would have the ability to field any questions that arise in regard to new information. Similarly, the researcher could address surprises or correct any misinformation.

Another valuable addition to the standard consent process may be to provide an “exit interview” at the conclusion of a subject’s participation. New potential participants could then be given access to that information and/or the opportunity to discuss the experiences with (willing) prior participants. For instance, a number of our subjects indicated that when deciding whether to participate in a clinical trial, they would give considerable weight to past participants’ judgments. These subjects indicated that they would welcome an opportunity to ask former participants about the experimental treatment’s effect on their daily lives and whether they now consider the research study worth the risks involved. Continued use of such debriefing techniques may help to further discern the level and type of details that “reasonable participants” are likely to value as salient aspects of the initial consent process. There are obviously potential questions of privacy and accuracy of information that would need to be considered; past participants’ names would certainly not be revealed without their informed written consent as approved by the IRB. Nonetheless, given the stated preferences of the subjects in this study, this type of potential communication between former and potential research subjects merits further exploration.

Our study suggests that a range of community stakeholders would value researchers moving beyond the current overemphasis on printed consent forms for research consent. By developing and empirically evaluating supplemental communication aids, researchers can help guide the evolution of informed consent in a manner that more fully honors the concept of respect for persons.

Acknowledgments

This project was supported in part by grants from the National Institute of Mental Health (R01 MH67002, P30 MH66248, and R01 MH64722), the National Institute on Aging (T35 AG26757 and R01AG028827), the John A. Hartford Foundation, the American Federation for Aging Research, and the Department of Veterans Affairs.

This project was reviewed and approved by the Human Research Protections Program (IRB) at the University of California, San Diego, and Veterans Affairs San Diego Healthcare System.

James Henry, BA, is a medical student in the University of California, San Diego, School of Medicine, La Jolla, CA;Barton W. Palmer, PhD, is Professor, Department of Psychiatry, University of California, San Diego, La Jolla, CA, and Research Scientist, Veterans Medical Research Foundation, San Diego, CA;Lawrence Palinkas, PhD, is Professor, School of Social Work, University of Southern California, Los Angeles, CA;Danielle Kukene Glorioso, MSW, is Center Manager, University of California, San Diego, Advanced Center for Innovations in Interventions and Services Research, San Diego, CA;Michael P. Caligiuri, PhD, is Professor, Department of Psychiatry, and Director, Human Research Protections Program, University of California, San Diego; andDilip V. Jeste, MD, is Estelle and Edgar Levi Chair in Aging, Distinguished Professor of Psychiatry and Neurosciences, and Director, Sam and Rose Stein Institute for Research on Aging, University of California, San Diego, La Jolla, CA.

References

1. Beauchamp T, Childress J.Principles of Biomedical Ethics , 4th ed. New York: Oxford University Press, 1994; National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research.The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research , https://www.hhs.gov/ohrp/humansubjects/guidance/belmont.htm.

2. Department of Health and Human Services. Protection of Human Subjects, 45 CFR 46; Food and Drug Administration. Protection of Human Subjects, 21 CFR 50; Department of Health and Human Services. Office for Human Research Protections. Tips on informed consent, https://www.hhs.gov/ohrp/humansubjects/guidance/ictips.htm.

3. Federman D, Hannam K, Lyman Rodriguez L.Responsible Research: A Systems Approach to Protecting Research Participants . Washington, DC: Institute of Medicine, National Academies Press, 2003.

4. Baker MT, Taub HA. Readability of informed consent forms for research in a Veterans Administration medical center.JAMA 1983;250(19):2646-2648; LoVerde ME, Prochazka AV, Byyny RL. Research consent forms: Continued unreadability and increasing length.Journal of  General Internal Medicine 1989;4(5):410-412.

5. Epstein LC, Lasagna L. Obtaining informed consent. Form or substance.Archives of Internal Medicine 1969; 123(6):682-688; Mann T. Informed consent for psychological research: Do subjects comprehend consent forms and understand their legal rights?Psychological Science 1994;5(3):140-143.

6. Barbour GL, Blumenkrantz MJ. Videotape aids informed consent decision.JAMA 1978;240(25):2741-2742; Silva MC, Sorrell JM. Enhancing comprehension of information for informed consent: A review of empirical research.IRB: A Review of Human Subjects Research 1988;10(1):1-5.

7. Jeste DV, Palmer BW, Golshan S, et al. Multimedia consent for research in people with schizophrenia and normal subjects: A randomized controlled trial.Schizophrenia Bulletin advance access published January 31, 2008, https://schizophreniabulletin.oxfordjournals.org/papbyrecent.dtl.

8. Mayer RE.Multimedia Learning . New York: Cambridge University Press, 2001.

9. Appelbaum PS, Grisso T.MacCAT-CR: MacArthur Competence Assessment Tool for Clinical Research . Sarasota, FL: Professional Resource Press, 2001.

10. Ryan RE, Prictor MJ, McLaughlin KJ, Hill SJ. Audio-visual presentation of information for informed consent for participation in clinical trials.Cochrane Database of Systematic Reviews 2008, Issue 1. Art. No.: CD003717.

11. Agre P, Rapkin B. Improving informed consent: A comparison of four consent tools.IRB: Ethics & Human Research 2003; 25(6):1-7; Benson PR, Roth LH, Appelbaum PS, et al. Information disclosure, subject understanding, and informed consent in psychiatric research.Law and Human Behavior 1988;12(4):455-475; Norris DR, Phillips MR. Using instructive videotapes to increase patient comprehension of informed consent.Journal of Clinical Research and Pharmacoepidemiology 1990;4(4):263-268; Weston J, Hannah M, Downes J. Evaluating the benefits of a patient information video during the informed consent process.Patient Education and Counseling 1997;30(3):239-245.

12. Flory J, Emanuel E. Interventions to improve research participants’ understanding in informed consent for research: A systematic review.JAMA 2004;292(13):1593-1601.

13. See ref. 11, Benson et al. 1988.

14. See ref. 11, Agre and Rapkin 2003; Campbell FA, Goldman BD, Boccia ML, Skinner M. The effect of format modifications and reading comprehension on recall of informed consent information by low-income parents: A comparison of print, video, and computer-based presentations.Patient Education and Counseling 2004;53(2):205-216.

15. Dunn LB, Lindamer LA, Palmer BW, et al. Improving understanding of research consent in middle-aged and elderly patients with psychotic disorders.American Journal of Geriatric Psychiatry 2002;10(2):142-150; Wirshing DA, Sergi MJ, Mintz J. A videotape intervention to enhance the informed consent process for medical and psychiatric treatment research.American Journal of Psychiatry 2005;162(1):186-188; Agre P, Campbell FA, Goldman BD, et al. Improving informed consent: The medium is not the message.IRB: Ethics & Human Research 2003;Suppl 25(5):S11-S19.

16. See ref. 11, Agre and Rapkin 2003.

17. See ref. 11, Weston et al. 1997; Fureman I, Meyers K, McLellan AT, et al. Evaluation of a video-supplement to informed consent: Injection drug users and preventive HIV vaccine efficacy trials.AIDS Education and Prevention 1997;9(4):330-341.

18. See ref. 17, Fureman et al. 1997.

19. See ref. 11, Weston et al. 1997.

20. See ref. 14, Campbell et al. 2004.

21. See ref. 14, Campbell et al. 2004.

22. See ref. 7, Jeste et al. 2008.

23. Llewellyn-Thomas HA, Thiel EC, Sem FWC, Woermke DEH. Presenting clinical trial information: A comparison of methods.Patient Education and Counseling 1995; 25(2): 97-107.

24. See ref. 8, Mayer 2001.

25. Fuller R, Dudley N, Blacktop J. Risk communication and older people—understanding of probability and risk information by medical inpatients aged 75 years and older.Age and Ageing 2001;30(6):473-476.

26. Nilsen ES, Myrhaug HT, Johansen M, et al. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material.Cochrane Database of Systematic Reviews 2006, Issue 3. Art. No.: CD004563.

27. See ref. 10, Ryan et al. 2008.

28. DuBois JM.Ethics in Mental Health Research: Principles, Guidance, and Cases . New York: Oxford University Press, 2008, p. 42.

29. Jimison HB, Sher PP, Appleyard R, LeVernois Y. The use of multimedia in the informed consent process.Journal of the American Medical Informatics Association 1998;5(3):245-256.

30. See ref. 7, Jeste et al. 2008.

31. See ref. 7, Jeste et al. 2008.

32. See ref. 7, Jeste et al. 2008.

33. See ref. 2, Department of Health and Human Services.

34. Reichstadt J, Depp CA, Palinkas LA, et al. Building blocks of successful aging: A focus group study of older adults’ perceived contributors to successful aging.American Journal of Geriatric Psychiatry 2007;15(3): 194-201; Palinkas LA, Criado V, Fuentes D, et al. Unmet needs for services for older adults with mental illness: Comparison of views of different stakeholder groups.American Journal of Geriatric Psychiatry 2007;15(6): 530-540.

35. See ref. 5, Epstein and Lasagna 1969; Mann 1994.

36. See ref. 1, Beauchamp and Childress, 1994; National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research 1979.

37. Glaser BG, Strauss A.The Discovery of Grounded Theory: Strategies for Qualitative Research . New York: Aldine de Gruyter, 1967.

38. Crabtree BF, Miller WL, eds.Doing Qualitative Research , 2nd ed. Thousand Oaks, CA: Sage Publications, 1999.

39. See ref. 29, Jimison et al. 1998.

40. Rist M, Mohan WJ. Wanted: Professional research subjects; Rewards commensurate with risks.Hastings Center Report 1976;6(6):28; Abadie R. The ethics debate on compensating drug trial volunteers.Anthropology News 2008;49:24.

41. Chernecky C, Macklin D, Walter J. Internet design preferences of patients with cancer.Oncology Nursing Forum 2006;33(4): 787-792.

42. Brusilovsky P, Kobsa A, Vassileva J, eds.Adaptive Hypertext and Hypermedia . Boston, MA: Kluwer Academic, 1998; Kelley CR. What is adaptive training?Human Factors 1969;11(6):547-556; Roy DD. Assessing validity of Web-based computer adaptive training modules.Journal of the Indian Academy of Applied Psychology 2008;34(1):127-136.

43. See ref. 12, Flory and Emanuel 2004.

44. Palmer BW, Cassidy EL, Dunn LB, et al. Effective use of consent forms and interactive questions in the consent process.IRB: Ethics & Human Research 2008;30(2):8-12; Eyler LT, Jeste DV. Enhancing the informed consent process: A conceptual overview.Behavioral Sciences and the Law 2006;24(4): 553-568.

45. See ref. 12, Flory and Emanuel 2004; see ref. 44, Eyler and Jeste 2006.

46. See ref. 7, Jeste et al. 2008.

James Henry, Barton W. Palmer, Lawrence Palinkas, Danielle Kukene Glorioso, Michael P. Caligiuri, and Dilip V. Jeste, “Reformed Consent: Adapting to New Media and Research Participant Preferences,”IRB: Ethics & Human Research 31, no. 2 (2009): 1-8.