Teachers are given the opportunity to instruct their students using various methods. The methods that teachers use can use will depend on several factors, such as what is needed for the curriculum and who the audience is. In order for teachers to be
effective in their classrooms, it is important that practical and effective textbooks are available. This paper will discuss three methods that can be used to measure the reliability of a tenth grade history textbook. The methods that will be discussed in this paper are: the “Fry Reliability Formula”, the Cloze Procedure, and the Content Area Reading Inventory method (CARI).
Fry Reliability Formula
According to Weitzel (2007), a gentleman named Edward Fry first developed a this method for teachers in order to determine the readability of materials used in classes. “The Fry readability method is a manual tool that determines the grade level of writing by analyzing three, 100-word passages from a selected writing, taking the average number of syllables and the average number of sentences for each passage and plotting those numbers on a Fry graph,”(Weitzel, 2007).
Scripts of the textbook: World History Patterns of
Integration
Passage 1:
“The Polynesian voyaging canoe one of the great ocean-going craft of the ancient world, was the means by which generations of adventurous voyagers were able to extend the human frontier far out into the Pacific, discovering a vast realm of Oceanic Islands. By 1000 B.C., when Mediterranean sailors were sailing in their land-locked sea, the immediate ancestors of the Polynesians had reached the previous uninhabited archipelagos of Fiji, Tonga, and Samoa, in the middle of the Pacific Ocean. Their descendants went on to settle all the habitable islands in a large triangular section bounded by the Hawaiian Archipelago” (Beck, 2005 p.s11).
Passage 2:
“The Assyrians employed military force to control a vast empire. In contrast, the Persians based their empire on tolerance and diplomacy. They relied on a strong military to back up their policies. Ancient Persia included what today is Iran. The Persian Homeland Indo-Europeans first migrated from central Europe and southern Russia to the mountain plateaus east of the Fertile Crescent around 1000 B.C. This area extended from the Caspian Sea in the North to the Persian Gulf in the South. In addition to the fertile farm land, ancient Iran boasted a wealth of minerals. These included copper, lead, gold, silver and gleaming blue lapis lazuli” (Beck, 2005 p.99).
Passage 3:
“The Greeks Invented drama as an art form an built their first theaters in the West. Theatrical productions in Athens were both an expression of civic pride and a tribute to the gods. As part of their civic duty, wealthy citizens bore the cost of producing the plays. Actors used colorful costumes, masks and sets to dramatize the stories. The plays were about leadership, justice, and the duties they owed to the gods. They often included a chorus, that danced, sang, and recited poetry. Tragedy and Comedy The Greeks wrote two kinds of drama”(Beck, 2005 p. 136).
Assessment
100 Word Passages Syllables Sentences
Passage 1 179 3
Passage 2 170 6
Passage 3 140 7
Average Average
163 5.3
Using the Fry Method three randomly selected passages were taken from the Textbook, World History Patterns of Interaction. Each passage was one hundred words. The syllables of each passage were counted as were the number of sentences. The average number of syllables was 163 per 100 words. The average number of sentences per 100 words was 5.3. Based on the Fry Graph, the textbook is written for tenth grade students, whose ages are typically 15 to 16 years of age. The Graph shows that the average number of syllables and sentences in the textbook that was examined was actually written for students who were between 18 or 19 years old (see Table 1).
Table 1: Fry’s Formula using 10th Grade World History Textbook
(source: Schrock, 2007)
Cloze Procedure
According to Lapp, Flood, & Farnan, (1996) the Cloze Procedure looks at material to determine whether or not it suitable for students. “The cloze procedure consists of the systematic deletion of words from selected passages of text” (Lapp, 1996 et al., p 394). What this means is that teachers can decide if a textbook should be used in their classroom.
According to Saskatoon Public Schools (2006), there are several reasons for teachers to use the Cloze Procedure:
• “To determine which cueing systems readers effectively employ to construct meaning from print”
• For assessment purposes to establish a student’s progress of the material.
• To allow students to keep track of what they are reading.
• To reinforce students “to think critically and analytically about text and content”.
Cloze tests are usually used by teachers to make sure the assignments are appropriate for the students. According to Lapp et al. (1996), “Cloze tests typically are constructed using a passage from a selected textbook or other materials students will be expected to read during content area instruction.” What this means is that the teacher will choose a reading selection to review prior to given instruction. Additionally, the teacher will evaluate whether the readings will be effective. Lapp et al. (1996) recommend for teachers to select “a 250-300 word passage containing complete thought units or paragraphs be selected, and that the passages from the front of the textbook be used because later passages are likely to be laden with concepts built on previously read material”. In other words, Cloze tests use a specific number of words to establish whether a textbook is conducive to lessons. Additionally, the earlier sections of the textbooks are preferred for teachers to evaluate.
The Cloze test was given to two high school sophomores. The test examines the students’ responses. The students were scored in three categories: The scores were rated as Frustration, Instructional, and Independent. Depending on what the student scores determines how a student is categorized. The students were administered the test. Student number 1 scored a _40__%, student number 2 scored a _39__%. This means both student #1 and #2 are categorized as reading in their Frustration level and did not comprehending their instruction. The results of the Cloze test were predictable due to the advanced level of the textbook. The results are similar to Fry’s assessment because the textbook is written for a higher age group. Moreover, the sophomore students were 15 years old and were not able to comprehend the textbooks. The results are different because the students were low on their score; however, they did comprehend some of the material in the passages. The explanation for the discrepancies is due to the fact that the teacher may have been instructing the students to a higher level and due to the fact that the students may not have been adequately prepared for the assignment or materials that were covered.
Cloze Test Given to Students
Passage 1:
“The Polynesian v_____ canoe one of the great ocean-going craft of the ancient world, was the means by which generations of a_________ voyagers were able to extend the human frontier far out into the Pacific, discovering a vast realm of O_____ Islands. By 1000 B.C., when M______ sailors were s______ in their land-locked sea, the immediate ancestors of the Polynesians had reached the previous uninhabited archipelagos of Fiji, Tonga, and Samoa, in the middle of the Pacific Ocean. Their d________ went on to settle all the habitable islands in a large triangular section bounded by the Hawaiian Archipelago” (Beck, 2005 p.s11).
Passage 2:
“The Assyrians e______ military force to control a vast e______. In contrast, the Persians based their empire on tolerance and d_____. They relied on a s______ military to back up their policies. A______ Persia included what today is Iran. The Persian H______ Indo-Europeans first migrated from central Europe and southern Russia to t__ mountain plateaus east of the Fertile Cr_____ around 1000 B.C. This area extended from the C_____ Sea in the North to the Persian Gulf in the South. In addition to the f______ farm land, ancient Iran boasted a wealth of minerals. These included copper, lead, gold, silver and gleaming blue lapis lazuli” (Beck, 2005 p.99).
Passage 3:
“The Greeks I_____ drama as an art form an built their first t_______ in the West. Theatrical productions in Athens were both an e________ of civic pride and a tribute to the gods. As part of their civic duty, wealthy citizens bore the cost of p_______ the plays. Actors used colorful costumes, masks and s_____ to dramatize the stories. The plays were about leadership, justice, and the d_____ they owed to the gods. They often included a c____, that danced, sang, and recited poetry. Tragedy and Comedy The Greeks w____ two kinds of drama” (Beck et al., 2005, p. 136).
Content Area Reading Inventory
Another effective evaluation tool teachers can implement when reviewing their textbooks is the Content Area Reading Inventory (CARI). This method allows “teachers to discover to the extent to which students have developed the reading ability and study skills necessary to understand and learn successfully from content area materials” (Lapp et al., 1996). In other words, this method helps teachers to assess their students’ progress in class. CARI has three specific sections for assisting teachers with student assessments. The first section uses sections of the book and additional resources. The second section reviews the comprehension of new vocabulary. Finally, the third section assesses whether or not students are able to understand what they have read (Lapp et al., 1996).
Passage Selected
“The Polynesian voyaging canoe one of the great ocean-going craft of the ancient world, was the means by which generations of adventurous voyagers were able to extend the human frontier far out into the Pacific, discovering a vast realm of Oceanic Islands. By 1000 B.C., when Mediterranean sailors were sailing in their land-locked sea, the immediate ancestors of the Polynesians had reached the previous uninhabited archipelagos of Fiji, Tonga, and Samoa, in the middle of the Pacific Ocean. Their descendants went on to settle all the habitable islands in a large triangular section bounded by the Hawaiian Archipelago” (Beck et al., 2005, p.s11).
Nine Question Evaluation Given to Students
1. Describe what the word “uninhabited” means from the following excerpt “when Mediterranean sailors were sailing in their land-locked sea, the immediate ancestors of the Polynesians had reached the previous uninhabited archipelagos of Fiji, Tonga, and Samoa, in the middle of the Pacific Ocean”. _________________________________________________________________
2. Explain the meaning of voyagers from the following excerpt. “The Polynesian voyaging canoe one of the great ocean-going craft of the ancient world, was the means by which generations of adventurous voyagers were able to extend the human frontier far out into the Pacific, discovering a vast realm of Oceanic Islands.”_____________________________________________________________________________________________________________________________________________
3. Explain the meaning of land-locked from the following excerpt. “By 1000 B.C., when Mediterranean sailors were sailing in their land-locked sea.”
4. Identify the main mode of transport used by the Polynesian voyagers.______________________________________________________________________________________________________________________________________________
5. Name three islands the Polynesian Voyagers visited. ______________________________________________________________________________________________________________________________________________________
6. What year did this passage describe? ________________________
7. Why was the Polynesian Voyage significant?
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
8. Why do you think some of the islands were uninhabited prior to the immediate ancestors to the Polynesians?
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
9. What other means of transport do you think the Polynesians used prior to canoes?
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Student #1
1. Correct
2. Correct
3. Correct
4. Correct
5. Correct
6. Correct
7. Correct
8. Correct
9. incorrect
Student #2
1. correct
2. correct
3. correct
4. correct
5. correct
6. correct
7. correct
8. incorrect
9.incorrect
Student #3
1. correct
2. correct
3. correct
4. correct
5. correct
6. correct
7. correct
8. incorrect
9.incorrect
Student #4
1. incorrect
2. incorrect
3. correct
4.correct
5.correct
6.correct
7.incorrect
8. incorrect
9.incorrect
Student #5
1. correct
2. correct
3.correct
4.correct
5.correct
6. blank
7. blank
8. blank
9.blank
Results
All of the students were correct in their first five answers, except for student #4. All students missed question #9. Additionally, it appears that most students had difficulties with the inferential questions, questions 7-9. Student #5 ran did not complete the last four questions. It appears that the class strengths are in vocabulary concepts and facts. The weakness for the group as a whole is in independent thought. Furthermore, student #4, received 4/9 correct which is a 44%. It appears this students has difficulties with both vocabulary concepts and with inferential questions and may need additional assistance. Moreover, lesson plans will need to be created for students to continue to use a higher level of thinking in order to assist students with their learning.
Conclusion
Teachers have many evaluation tools to utilize in their classrooms. One of the most important assessments teachers can use is making sure the textbooks are beneficial to the students learning goals. This paper discussed three methods that can be used by teachers to measure the reliability of a tenth grade history textbook. The methods reviewed were: the “Fry Reliability Formula”, the Cloze Procedure, and the Content Area Reading Inventory method. Although all of these methods can be used by teachers when evaluating the usefulness of a school textbook for students, teachers should also be mindful that these are only a few of the many methods that can be used for evaluating textbooks.