T he effective use of volunteer tutors has recently come to the forefront of literacy research. Sparked by the America Reads Challenge Act of 1997, thousands of volunteer tutors are entering our schools to work with struggling readers. The America Reads Challenge (ARC), a federal grassroots call to action, is an unprecedented national campaign that challenges every American to help all our children learn to read, including those with disabilities and limited English proficiency. ARC proposes having adult volunteers tutor children during and after school. Nationwide, thousands of America Reads tutors serve their communities through Federal Work Study, AmeriCorps, VISTA, National Senior Service Corps, and Learn & Serve America. As a result of increased funding of these service organizations, hundreds of thousands of tutors are being sent into American public schools. Currently, 20,000 students at 1,100 colleges teach elementary students through the Federal Work Study program (Nonte, 1999). Although volunteer tutors are not a new concept (Wasik, 1998), the sudden surge of thousands of volunteers in our public school system is a phenomenon in need of study. For volunteers to make a lasting impact on young children, it is essential that tutoring programs are founded on a solid base of research, and that we know whether, in fact, they work. That is, does adding tutors to schools improve outcomes for children?
Unfortunately, there is a startling lack of evidence regarding the effectiveness of one-on-one tutoring by volunteers. The popularity of Success For All (Slavin, Madden, Dolan, & Wasik, 1996) and Reading Recovery (see Pinnell, DeFord, & Lyons, 1988) has created a general belief that one-on-one tutoring is an effective panacea for poor reading achievement (Wasik, 1998). Little research, however, has actually documented this effectiveness using adult volunteers.
In Wasik's (1998) analysis of 17 volunteer tutoring programs, 2 of the programs provided evaluations that included an experimental-control comparison between well-matched or randomly assigned groups of participants. These two programs were the Howard Street Tutoring Program (Morris, Shaw, & Perney, 1990) and the School Volunteer Project (U. S. Department of Education, 1979). The Howard Street Tutoring Program reported effect sizes of .61 for word recognition and 1.07 for oral reading with the tutored group, performing significantly better than the matched controls. Likewise, the School Volunteer Development Project reported effect sizes of .50 on the Metropolitan Achievement Test, with tutored students outperforming nontutored students in a randomly assigned control group.
Of the tutoring programs reviewed by Wasik (1998), two other programs reported a within-group control design, using number of sessions as the criterion for group assignment. Book Buddies (Invernizzi, Rosemary, Juel, & Richards, 1997) and Reading One-One (Farkas, 1996) both reported significant effects for word recognition in and out of context, with students who received more tutoring sessions outperforming students who received fewer tutoring sessions (Wasik, 1998). Despite evidence that the number of sessions was not correlated with any pretest measure, both programs were criticized for a lack of experimental-control comparisons between equivalent groups (Wasik, 1997). Reading One-One has been further criticized for not truly being a volunteer or National Service tutoring program; Reading One-One tutors are paid an hourly wage and are hired only after rigorous screening to eliminate incompetent, uneducated, or illiterate applicants (Farkas, 1996; Wasik, 1997).
The Book Buddies program (Invernizzi, Juel, & Rosemary, 1997) in Charlottesville, Virginia, uses unpaid community volunteers who are solicited by a volunteer recruiter. Tutors are primarily white, educated, and female. Two thirds of the volunteers tutor for more than two years, and a fourth have tutored since the program's inception in the early 1990s. The program operates in all six of Charlottesville's elementary schools and has received accolades at the state and national level. Program evaluations of the first three Charlottesville Book Buddies cohorts reported effect sizes of 1.24 for word recognition (Invernizzi, Rosemary, Juel & Richards, 1997), and results show consistent improvement from one year to the next.
Despite this, Book Buddies has remained an untested model for the America Reads Challenge. The purpose of this study was to put Book Buddies to the test, using tutors typically employed in America Reads and National Service (AmeriCorp, VISTA, Federal Work Study tutors) and a rigorous experimental vs. control group design. Further, because of the immense need for resources that are effective in urban schools, the Book Buddies model was tested in a challenging, high-poverty urban setting.
Wasik (1998) identified characteristics of successful tutoring programs. Two characteristics significantly affect the design of the tutoring program. First, successful volunteer tutoring programs assign a reading specialist to write and supervise lessons and to provide ongoing training and feedback to the tutors. These individuals use their reading expertise to assess students regularly and prepare daily lesson plans that meet students' changing instructional needs. Through constant observation, reading specialists provide detailed feedback and specific information to the tutors. In addition, reading specialists teach volunteers about the reading process and about children who have difficulty learning to read (Wasik, 1998).
Although a reading specialist costs a project extra money, the consequences of an unsupervised program can result in wasted time and money. One program, funded by more than $1 million in Federal Work-Study grants, placed over 1,300 Federal Work-Study students in 16 schools. Believing that a reading specialist was too expensive, untrained VISTA members wrote the lesson plans. Observations and evaluations of the program revealed inappropriate lesson plans and questionable instruction. One tutoring interaction was recorded:
During one session, a community volunteer asked a first-grader to complete the spelling of the word "panda." The student's guesses at the final vowel included o, r, d, e, o, u, d, and o. The tutor followed up each wrong guess with "No," or "How did I tell you to spell this?" or "Now, what would that make it? pan-dOOO." There was no lack of effort on the part of the student or affection from the tutor. But neither had a clue about how to get over that phonetic hurdle." Better training and supervision are now at the top of the agenda for their comprehensive evaluation. (Chaddock, 1998, p. A1)
A second critical area identified by Wasik (1998) is the need for a structured lesson plan. If significant progress is to be made, best practices research and the use of quality materials should drive the components and content of the lesson plan. Tutoring programs consistently mentioned in the tutoring literature share many of the basic components associated with best practices in early literacy instruction. These common practices include: (a) rereading easy, familiar material to build fluency; (b) word study emphasizing letter-sound relationships and common spelling patterns; (c) writing; and (d) learning to read a new text in each lesson. Lessons carried out in Book Buddies conform to both of the components identified as critical for success.
Beginning in 1991, the Charlottesville City schools, the McGuffey Reading Center of the University of Virginia, and the Charlottesville community formed a partnership. Its goals were to improve the reading and writing skills of at-risk children and to solidify the community's involvement in the education of its children (Invernizzi, Juel, & Rosemary, 1997). The result was Book Buddies, a first-grade intervention program that uses community volunteers as tutors. At each school, Book Buddies uses a triad composed of the child, the volunteer tutor, and a reading specialist, who acts as the Book Buddies coordinator. Tutorials are scheduled to avoid conflict with classroom instruction, and each Book Buddy first-grade student is tutored for 45 minutes twice a week. Book Buddies achieves substantial effect size results in comparison to other tutorial programs, and produces results "comparable to those found with professionally trained teachers" (Snow, Burns & Griffin, 1998, p. 260).
Book Buddies coordinators supervise each tutorial and provide ongoing training and support to the volunteers throughout the year of tutoring. In addition, the Book Buddies coordinator assesses each student individually, writes individualized lesson plans for each one, and gathers the appropriate materials for each lesson. Each coordinator supervises approximately 15 volunteer tutors and their respective tutees. Following four years of successful implementation, the school division funded salaries for nine Book Buddies coordinators, who each work approximately 17 hours a week. Coordinators were initially funded through grants obtained by the program originators, Marcia Invernizzi and Connie Juel.
The volunteers for the Charlottesville program are primarily unpaid volunteers from the Charlottesville community. They attend formal training sessions two to three times a year, and receive on-site training on a day-to-day basis from their reading coordinator. The volunteers follow the tutoring lesson plan written for them by the Book Buddies coordinator and write comments on the lesson plan that inform the coordinator of specific areas of success or difficulty for the student.
Classroom teachers recommend first-grade students for the Book Buddies program. Selection is determined by the Title I referral process and by the children's summed scored on the Phonological Awareness Literacy Screening (PALS; Invernizzi, Meier, Swank, & Juel, 1998). A summed score is derived by adding together a student's score on rhyme and beginning sound awareness tasks, alphabet naming, letter-sound identification, spelling, and word recognition tasks.
In 1999, after its first seven years of operation, Book Buddies had served over 1,000 first-grade children in Charlottesville City Schools. This local success has spawned many other Book Buddies programs in and around the state. However, without evidence from experimental studies, it is impossible to determine whether Book Buddies is a viable model for replication by America Reads and National Service organizations. The popularity of Book Buddies and the potential for tutoring programs provided by the America Reads Challenge provide an important opportunity to evaluate the effectiveness of this program using a rigorous research design. The purpose of this study was to determine whether the Book Buddies program causes clear improvements in young children's reading skills when implemented in an urban setting.
The Book Buddies program was placed in an America Reads project site in the South Bronx, New York, in Congressional District 16, the poorest congressional district in the nation. Most students from this school district and their families live in Community District 1, School District 7. The population of the community is 67% Hispanic, 31% non-Hispanic black, 2% non-Hispanic white, and 1% other. An estimated 18% of the population aged five and older do not speak English well or at all. According to the New York City Board of Education's school profiles, 93% of the students in are eligible to receive free lunch. More than half of the population is unemployed and dependent on some form of government assistance.
The full-time Tutor Trainer Coordinator (TTC) was an experienced Book Buddies tutor in the Charlottesville program and had graduated with a minor in Reading Education. The TTC supervised the tutoring four days a week and spent the fifth day creating materials, conducting long-range planning, and supervising the tutoring of students who had been absent during the week. Our TTC worked for seven months before beginning maternity leave. At that time, a Master of Education reading specialist began running the project, first by shadowing the TTC for several weeks, and then taking the project on herself. Two individuals help the TTC--a full time VISTA worker and two part-time Federal Work-Study students. These individuals help the TTC gather materials for the tutoring boxes and tutor one student each.
The 15 Book Buddies in the Bronx tutors were all part of the Retired Seniors Volunteer Project (RSVP), a division of the Experience Corps, an America Reads project of the Community Service Society of New York. At the time of the project, all of the volunteers were 55 years old or older, with 45% of the tutors between the ages of 55 and 64. Seventy-eight percent were female and 22% were male; 82% were African-American, 15% were Hispanic, and 3% were Caucasian. Forty-seven percent of volunteers attended or had graduated from college, and 42% had graduated from high school. Tutors worked 12 hours a week, and received a small stipend for their participation.
The recruitment strategies included a collaborative effort with the American Association of Retired Persons. Efforts included conducting a direct mailing to its membership, targeting local print and electronic media, and posting fliers at local agencies and houses of worship. In placement, recruitment, and support of volunteers, RSVP staff was sensitive to language and cultural differences. To articulate clear expectations about the assignment, each volunteer was provided with a job description outlining tasks and a handbook defining program goals, policies, and procedures.
Tutors attended a formal three-day workshop before they began tutoring. During this workshop, tutors learned about each part of the Book Buddies lesson plan. They viewed videotapes of each component of the lesson plan, previewed books and word study materials, and participated in role-plays with a partner. Each day of training ended with a short quiz covering that day's aspect of the tutoring plan. In addition, the TTC held informal one-hour training sessions with the 15 tutors twice a week throughout the year. The informal training sessions focused on the four main parts of the lesson plan. Videos, handouts, and live demonstrations were provided for the tutors, which provided excellent opportunities for tutors to discuss their concerns and uncertainties. The Book Buddies manual (Johnston, Invernizzi, & Juel, 1998) was made available for checkout, and tutors were referred to the manual during training sessions.
First-grade children were selected from one elementary school in the South Bronx. The population of this school is 99% minority, and 99% of students were eligible for free or reduced-price lunch. Sixty-nine percent of the students in the first-grade classrooms were Hispanic, 30% were non-Hispanic African-American, and 1% were unknown.
Students were originally referred for tutoring based on the end of kindergarten schoolwide assessment program and teacher referral. After conducting our own assessments using PALS (Invernizzi et al., 1998), 55 of the lowest scoring first-graders were selected. Two children were omitted because of behavioral concerns. Students were matched by pretest summed scores, then randomly assigned to either treatment (Cohort A) or control group (Cohort B) conditions. The two groups were comparable on demographic factors and on school attendance. During the treatment period, no additional services were provided to any of the first-grade students. The study participants included 55 first-grade students, with 28 in the treatment group and 27 in the control group. In the experimental group, there were 11 females and 17 males. In the control group, there were 12 females and 15 males.
Students circle the picture of a word that rhymes with the picture of a target word. The examiner and student complete three practice items together before completing ten assessment items. Practice and assessment items are completed orally, with the examiner identifying each picture for the student. Students receive 1 point for each correctly circled item. There are 10 possible points.
Students circle the picture of a word that begins with the same sound as the picture of a target word. The examiner and student complete three practice items together before completing ten assessment items. Practice and assessment items are completed orally, with the examiner identifying each picture for the student. Students receive 1 point for each correctly circled item. There are 10 possible points.
Students provide the letter sound for 23 upper-case letters and the digraphs ch, sh, and th . A pronunciation guide is provided for examiners. Only short vowel sounds are accepted as correct responses. When a child gives the long vowel sound, the examiner probes by asking the student, "What other sound does this letter make?" Letters are presented in random order. Students receive 1 point for each correctly named letter sound. There are 26 possible points.
Students spell five consonant-vowel-consonant words. Spellings are scored by number of correct or acceptable phonemes. A scoring guide details acceptable phoneme representations. Students receive 1 point for each correct or acceptable phoneme, and entire words spelled correctly receive one bonus point. There are 20 possible points.
Students in the treatment and control groups were posttested on two measures, assessing their reading accuracy by determining (a) the number of words read correctly in one minute from a primer-level text, and (b) the score on a standardized measure of letter and word recognition.
The student read a passage from Little Bear (Minarik, 1978) for one minute. The examiner kept a running record of the student's reading. The score was the total number of words correctly read. This assessment was conducted in January and June. The reading in context assessment was not completed in September because the students were unable to read.
The WRAT Reading subtest consists of a letter reading section and a word reading section. For letter reading, students name 15 letters of the alphabet. For word reading, students are asked to pronounce 42 words. One point is given for each correct letter and word. A maximum of 57 points can be earned.
In September, the researchers and the TTC assessed the 55 first-grade students determined to be the most at risk for reading difficulties by the three first-grade teachers. Students were matched on the PALS pretest summed scores and randomly assigned to Cohort A or Cohort B. Cohort A students received tutoring from September to January. Cohort B, the control group for the first phase of the study, received classroom instruction only from September to January, and received tutoring from January to June. Each Book Buddies student received 40 Book Buddies lessons. A description of the tutoring follows.
Book Buddies lessons consist of reading, writing, and phonics (Johnston et al., 1997). Tutors follow a sequence of core activities in a four-part lesson plan described in a volunteer tutoring manual (Johnston et al., 1998). The 45-minute tutoring lessons include (a) rereading familiar books, (b) word study, (c) writing, and (d) reading a new book.
Book Buddies lessons begin with the repeated reading of three or four familiar books followed by independent reading of the new book from the previous session. The goal of the rereading portion is to provide the student with a successful warm-up for the tutoring session and to develop fluency and automatic word recognition (Samuels, 1979).
Word study involves the pacing of instruction in alphabet, spelling, and phonics in accordance with the developmental word knowledge of the child (Bear, Invernizzi, Templeton, & Johnston, 1999). Derived from developmental spelling theory and using a compare-and-contrast approach to word features, word study instruction consists of sorting tasks, first with picture cards, then with word cards as words become known.
The writing component involves writing for sounds. During the writing portion, students learn to segment their speech and to match letters to those segmented sounds. Tutors often dictate sentences from familiar texts, or children compose their own sentences about the books they have read.
A new book is introduced at the end of each Book Buddies lesson. The new book is chosen by the TTC, and incorporates the word study feature on which the child is working. After a thorough book preview, the student reads the book independently. If needed, the tutor supports the child with choral and echo reading during the first attempt.
We assessed the outcomes of the tutoring program by comparing the differences between the group receiving tutoring in the first half of the year to those in the control condition. Then, after the students in the control condition received tutoring in the second half of the year, we compared the differences again. For the comparisons, we used PALS, one standardized measure of letter identification and word reading in isolation, and one measure of reading in context. A split-plot analysis of variance was conducted for the variables of interest. The homogeneity of variance, or sphericity, assumption associated with split-plot analyses was tested using Mauchly's Test of Sphericity, p > .05, and the results are discussed with confidence of equal variance.
Both cohorts made progress during the year on basic measures of the alphabetic principle. In September, the first-grade students could name an average of 18 lower-case letters, identify six letter sounds, and read three words from a preprimer list of words. By June, these students could name an average of 25 letters of the alphabet, identify 20 letter sounds, and read on an instructional level (16 words) from a preprimer list of words. The PALS results by cohort are presented in Table 1. Comparisons that yielded statistically significant differences between cohorts are indicated.
A split-plot analysis of variance failed to reveal a statistically significant difference between group main effect on the WRAT, F(1, 46) = 2.93, p = .09. However, both the within-group effect, or differences across time [F(2, 92) = 69.47, p < .001] and cohort x time interaction were found to be statistically significant, F(2, 92) = 3.52, p < .05. Three simple effect post-hoc analyses were conducted to determine which cohort scored highest on the WRAT at each of the three time points. Type I error rates were controlled among these nonorthogonal contrasts through Bonferroni adjustments (i.e., p value calculated at .05/3 = .0167).
Results indicated that there were no statistically significant differences between the groups in September, F(1, 92) = 1.55, p > .0167. In January, there were significant differences between the groups, F(1,92) = 24.38, p < .0167, with the tutored group outperforming the nontutored group. By June, there were no significant differences between the groups, F(1, 92) = 1.62, p > .0167, indicating that, following their period of tutoring, the previously nontutored group (Cohort B) progressed to the point where there were no significant differences between them and the group who received tutoring in the first half of the year (Cohort A). The pattern of results is illustrated by Figure 1.
A split-plot analysis of variance failed to reveal a statistically significant difference between the group main effect for one minute of correct reading in context, F(1, 46) = .660, p = .421. However, both the within-group main effect, or differences across time [F(1, 46) = 65.83, p < .001] and cohort x time interaction were found to be statistically significant, F(1, 46) = 4.26, p < .05. Two simple post-hoc analyses were conducted to determine which cohort scored highest on the one minute of reading in context at each of the two time points. Type I error rates were controlled among these nonorthogonal contrasts through Bonferroni adjustments (i.e., .05/2 = .025).
Results indicated that there were statistically significant differences between the groups in January, F(1, 46) = 17.12, p < .025, with the tutored group (Cohort A) outperforming the nontutored group (Cohort B) on the measure of accurate reading in context. By June, there were no statistically significant differences between the groups, F(1, 46) = .1069, p < .025, suggesting that, following tutoring, the two groups performed equally on the measure of word reading in context. These results follow the same pattern illustrated by Figure 1.
The purpose of this study was to implement and evaluate the Book Buddies model using National Service volunteers as tutors for high-risk first-grade students in a challenging, high-poverty urban setting. Using a design with a randomly assigned control group, the results show that tutoring sessions produced clear effects, regardless of whether they were provided in the fall or spring of first grade. The results from the January testing show that the children who received 40 Book Buddies lessons significantly surpassed the students in the control group on measures of accurate word reading in context, letter identification, and word reading in isolation. These findings are similar to other control group studies (e.g., Morris et al., 1990) that reveal positive effects for the tutored group over the control group. The findings from the June testing suggest that the group of students who received tutoring during the second half of the school year were able to progress to the point where there were no distinguishable differences between the two groups at the end of the year.
The findings are straightforward. First, the Book Buddies model of using adult volunteer tutors to tutor high-risk first graders in a closely supervised, one-on-one structured format is not only possible, but also effective. Unlike the tutors in the parent program, the tutors in Book Buddies in the Bronx were not homogeneously white, female, and highly educated. To the contrary, the RSVP AmeriCorp seniors who participated in the Book Buddies in the Bronx project formed a diverse group; fewer than half attended college at all, and 15% spoke English as their second language. The fact that significant intervention effects were obtained in one of poorest school districts in America using a diverse group of National Service volunteers to deliver instruction is promising indeed.
The second finding contributes to discussions about school resources and intervention planning. Researchers who participate in early intervention efforts would probably hope that the students who received their 40 Book Buddies lessons during the first half of the year would not only surpass their peers in the control group, but also continue their growth after the treatment had ended. After all, the students in Cohort A had cracked the alphabetic code and they had all the necessary prerequisites in place to benefit from regular classroom instruction. Theoretically, Cohort A students should have spent the second half of their year putting their alphabetic knowledge to work as they navigated the waters of first-grade reading instruction. But Cohort B, consisting of students who were significantly behind their peers in January on measures of alphabet recognition, letter sounds, spelling, and word recognition, essentially caught up to Cohort A after receiving 40 Book Buddies sessions. The good news is that both groups benefited from the 40 sessions of one-on-one Book Buddies tutoring, regardless of the point in the school year at which the tutoring began. Schools with limited resources might be able to serve more students if tutoring could be planned for two successive cohorts.
Why didn't Cohort A outperform Cohort B at the end of the year? The answer to that question is beyond the scope of this study, but one thing is powerfully clear: Tutoring alone did not enable the students to read at end-of-year first-grade level by June. Although both groups made tremendous progress, students in both groups achieved, on average, only a preprimer reading level after one year of classroom instruction and 40 sessions of one-on-one tutoring. These results provide another demonstration that there is no panacea for helping students at risk for school failure. When beneficial early intervention programs are provided, they will need to be followed by additional high-quality experiences if students are to stand a chance. Other support structures (i.e., improved classroom instruction, continued one-on-one assistance) must be in place to sustain the rapid growth and truly make a difference. As Moats (1999) described, while tutors and parents can contribute to reading success, classroom instruction must be viewed as the critical factor in children's reading success.
The results of this study contribute to our understanding of the constraints and conditions of implementing effective volunteer tutoring in challenging circumstances. Specifically, this study underscores the importance of key components for ensuring quality tutoring for the America Reads Challenge: (a) the use of research-based instructional elements to produce significant progress in reading achievement; (b) the need for a well-structured lesson plan in which the content and delivery of instruction are carefully planned; (c) the need for intensive, ongoing training and supervision of tutoring under the guidance of a knowledgeable reading specialist; (d) the need for frequent and regular tutoring sessions; and (e) the need for careful evaluation, assessment, monitoring and reinforcement of progress.