Texas Academic Skills Program

Annual Report on the TASP

and the Effectiveness of Remediation

July 1996


Executive Summary

The Texas Academic Skills Program (TASP)

This report includes both a trend analysis of year-end test results and a cohort analysis of academic performance. Trend analysis, which is based in this report on TASP Test performance alone, is appropriate for making comparisons of student performance among groups over time. The cohort analysis, which includes data from both locally required remediation and TASP-required remediation, should be used only as a snapshot of the progress of a particular group of students at a given point in time. Any group comparisons from the cohort data should be made only when cohorts have been in existence for a similar time period.

Trend Data

Trend data show student performance on the TASP Test by ethnicities and academic subjects at Texas public postsecondary institutions for academic years 1989-90 through 1994-95. The raw numbers of all students except White students attempting the TASP Test continue to increase, and the proportion of all students except Whites attempting the TASP Test continues to increase. Minority students are accounting for proportionately more of the entering and tested freshmen students, reflecting the changing demographics of Texas.

Comparison of annual results in 1993-94 and thereafter with the prior years' results is difficult because of several programmatic changes implemented September 1993: exemptions from the Texas Academic Skills Program resulting from high performance on other test instruments, changes to the test instrument in anticipation of increasing standards, and the rescission of the "15-hour rule" for attempting the test. These programmatic alterations have caused a dramatic change in the tested population, and less prepared students are now attempting the examination, resulting in lower pass rates when compared with previous years.

The data still support some general conclusions:

Not included in these analyses are students exempt from the TASP through college credit earned prior to the 1989-90 academic year, students participating in TASP-waived certificate programs, students attending independent or out-of-state institutions, high school students who attempt the TASP Test, and students attending upper-division or medical institutions where numbers are so few that student privacy might be violated if TASP score data were released.

Cohort Data

Each cohort consists of first-time entering students who remain in higher education for a least two semesters through the following fall semester (semesters need not be contiguous). The same students are excluded from the cohort studies as are excluded from the trend analysis, as well as students attending only one semester of an academic year.

The success of students requiring remediation is compared to the success of students not requiring remediation as measured on grade point averages above 2.0 (C average), passing college-level coursework with a D or higher, retention rates, and highest award earned (cohorts 1989-95, 1990-96, and 1991-97).

Cohort studies, when complete, allow for a given cohort to be analyzed over a six-year period, the time in which most students can be expected to complete an undergraduate education. Future reports will follow the cohorts in progress presented here, as well as new cohorts as they come into existence until their sixth year. Only the first cohort has actually completed the six-year study period.

Statewide, students who completed required remediation are performing in college at levels generally comparable to those students who did not require remediation. Data show that when students complete remediation, they:

After a six-year period, students who complete remediation earn certificates and associate degrees at a higher rate than students who do not require remediation. Students who do not require remediation earn proportionately more baccalaureate degrees, but students who complete remediation remain in school longer and may perform at the same level if given additional time to complete their degrees.


INTRODUCTION

Background

The Texas Higher Education Coordinating Board in 1985 appointed the Committee on Testing to determine how many Texas students were entering college inadequately prepared for college-level work. The committee reported in A Generation of Failure: The Case for Testing and Remediation in Texas Higher Education (July 1986) that an estimated 30 percent of the students entering Texas public higher education each year could not read, write, or compute at levels needed to perform effectively in higher education, causing some of those college students to leave higher education because they lack needed skills, not because they lack ability. The committee found that other students may receive degrees without having mastered basic reading, writing, and mathematics skills.

The Committee on Testing recommended that:

Accountability for such an extensive and important education initiative was a major concern of the committee. In 1986, the recommendations provided by the Committee on Testing were adopted by the Coordinating Board and sent to the Texas Legislature. During the 1987 legislative session, the recommendations became law under Section 51.306 of the Texas Education Code (Appendix A). The legislation that created the Texas Academic Skills Program incorporated all of the significant Committee on Testing recommendations. The legislature and Board rules require that if skills deficiencies are identified, the student is required to participate in continuous remediation until he or she masters all sections of the examination. The institutions must offer advising programs for all students and remedial programs for students with demonstrated skills deficiencies.

While the TASP is the first statewide skills assessment and remediation program in Texas, many public colleges and universities had a long history of diagnostic assessment and academic assistance for under-prepared entering students. When implementing the TASP, the Coordinating Board took advantage of this institutional experience in assessment and remediation by adopting rules allowing colleges and universities to continue administering diagnostic examinations already in place to entering students. Students are not required by the state to enter remediation after failing a local placement examination, though institutions may require it.

The Coordinating Board adopted rules to address specific issues related to the administration of the TASP, including rules governing when students take the test and what action is needed if performance indicates a need for remedial intervention. Continuous remediation is required by Coordinating Board rules, although no particular program is prescribed. The Coordinating Board believes that students benefit most from early and intensive remediation. Board rules require a student to participate in remediation every semester of enrollment until all sections of the TASP examination are mastered. Although students needing help in more than one content area do not have to remediate in all areas at once, they must continuously work on skills development in at least one area each semester they are enrolled.

The individual instructional needs of skills-deficient students are determined by the institutions. Developmental educators and academic advisors on each campus, with appropriate educational experience, resources, and knowledge of the background of each student, determine the best way to meet student needs. Institutions are encouraged to provide a wide variety of remedial options to students, including classroom instruction, tutorials, self-paced classes, computerized instruction, and self-directed study.

Evaluations of advising and remedial programs are also conducted to determine program effectiveness. Two major reports are produced as a result of the Texas Academic Skills Program: the Annual Report on the TASP and the Effectiveness of Remediation and the Report on Academic Advising. This volume contains the fifth Annual Report on the TASP and the Effectiveness of Remediation.

Although the testing component of the TASP usually receives most of the public's attention, the TASP is much more than a test. Skills assessment is accompanied by academic advising and placement into courses at appropriate levels, or remediation if necessary. The law simply states that students must be offered remediation and that they must master all content areas on the examination before enrolling in upper-division course work beyond 60 semester credit hours or graduating from a two-year college.

More than $22.6 million in remedial program start-up funding was appropriated to institutions during the 1990-1991 biennium. The Legislature appropriated $11.6 million for noncourse-based remediation during the 1992-1993 biennium, $11.9 million for the 1994-1995 biennium, and $11.7 million for the 1996-1997 biennium. These appropriations are in addition to formula funding for course-based remediation.

The TASP Test

More than 3,000 Texas higher education faculty and administrators were involved in the development of the TASP Test. They served on key committees and contributed to the validation and standard setting processes.

Three content committees -- one each for reading, writing, and mathematics -- determined which skills and sub-skills should be tested and reviewed test materials and test items for accuracy, appropriateness, and skill match. A separate committee studied all test materials and test items to insure that they were free of bias. Finally, regional panels provided an additional review of the materials.

Through surveys, several thousand Texas faculty were asked if the skills to be tested represented preparation necessary for success in an undergraduate degree or certificate program. Results of the validation survey indicated the chosen skills were necessary.

Following the development and validation of the skills and test items, the items were field-tested on freshmen students attending public colleges and universities. The content and bias review committees reviewed and approved changes to items found to be unsatisfactory.

Each section of the TASP Test is designed to measure a student's skills in relation to a standard of competence established by the Texas Higher Education Coordinating Board. For a complete list of the 29 tested skills, refer to Appendix B.

The reading section is comprised of selections of 300- to 750-word reading passages similar to those found in course materials (e.g., textbooks, lab manuals, articles) students are likely to encounter during their first year in college. Students respond to approximately 40 multiple-choice questions matched to about seven reading selections.

The mathematics section consists of approximately 50 multiple-choice questions covering fundamental mathematics, algebra, geometry, and problem solving. Test questions focus on a student's ability to perform mathematical operations and solve problems. Appropriate formulas are provided to students for use in performing some calculations. Students may not use calculators during the test.

The writing section consists of two parts -- approximately 40 multiple-choice questions and a writing sample. Passing the writing section is based primarily on the essay; the multiple-choice is used only if the student receives a failing grade on the essay from one of two scorers. The multiple-choice section assesses students' ability to recognize various elements of effective writing. Writing assignment topics are similar to typical in-class essay assignments. Students are not allowed to use dictionaries during the test.

The TASP Test is administered six times per year at more than 100 testing centers across the state. Standardized test administration procedures ensure security, uniformity, and fairness. At the beginning of a test session, each student receives a test booklet that contains all three sections of the examination. The sections are not timed and students may work on the sections in any order they choose. A total of five hours is given for a test session, during which time a student may work on one, two, or all three sections. If a student is retaking the examination, only those sections failed must be repeated. The total testing time is provided for all students regardless of the number of sections they choose to attempt. In compliance with the federal Americans With Disabilities Act of 1990 and the federal Rehabilitation Act of 1973, students with disabilities may be entitled to special accommodations for the TASP Test. Accommodations can involve changes to the test environment or to the test materials. Students must provide documentation of their disability to get appropriate accommodations.

Multiple-choice test questions are machine scored. Writing samples are holistically scored by trained scorers from across the state. The writing scorers look for effective communication of a complete message to a specified audience for a stated purpose. Appropriateness, unity and focus, development, organization, sentence structure, usage, and mechanical conventions are the characteristics considered in scoring writing samples. Failing essays are also analytically scored.

Each test form is constructed in a way that allows computation of a total score for each section as well as diagnostic information about performance on individual skills or groups of skills. Each student receives an individual score report that provides information about total score for each section (pass/fail) and an indication of strengths or weaknesses for each skill area. Students may choose up to three institutions to receive a copy of the score information.

Program Changes Effective 1993

A number of programmatic changes implemented in the 1993 fall semester affected the Texas Academic Skills Program. These changes included exemptions from the TASP for students who perform at specified standards on the Scholastic Assessment Test (SAT), American College Test (ACT), or the Texas Assessment of Academic Skills (TAAS). Another change was the rescission of the rule by which students could wait until their 15th college hour to take the TASP Test. Now, all students must attempt the test by their ninth college hour. Finally, changes to the assessment instrument made for a longer and slightly more difficult test.

These programmatic changes affected the TASP Test passing rate for academic years 1993-95. A study was completed to insure that these changes affected no demographic group's pass rate disproportionately. The results of that study were reported in the fourth Annual Report on the TASP and the Effectiveness of Remediation (August 1995).

Testing Instrument Changes

Effective with the fall semester 1993, changes were made to the reading and the mathematics portions of the TASP Test, and a new type of prompt on the essay portion of the writing test was introduced to reflect a more realistic college writing assignment. All of these changes were field tested before implementation and equated after implementation to assure that no demographic group would be disproportionately affected.

Exemptions

Texas Education Code, Sect. 51.306(m) exempts students from the Texas Academic Skills Program based on exceptionally high performance on either the Texas Assessment of Academic Skills (TAAS), the Scholastic Assessment Test (SAT), or the American College Test (ACT). Texas public institutions of higher education determine which students meet the performance levels set by the Coordinating Board.

The number of students sitting for their initial attempt on the TASP Test during academic year 1994-95 was 81,490. There were 12,064 exemptions awarded based on high performance on the TAAS, SAT, or ACT. This gave an exemption rate of 8.4 percent. The exemption rate in the 1993-94 academic year was 5.5 percent.

The exemptions depress the TASP Test pass rate because the top tier of students, all of whom presumably would have passed the TASP Test, are not tested. When the exempted students are factored into the pass rate, the pass rate on initial attempt increases by nearly six percentage points, and the overall pass rate for all attempts increases by four percentage points.

Rescinding the "15-Hour Rule"

Texas Education Code, Sect. 51.306 requires students to attempt the TASP Test in the semester in which they complete their ninth semester credit hour (SCH). Prior to the fall 1993 semester, Coordinating Board rules allowed an institution to permit its students to wait until they had completed 15 SCH to attempt the TASP Test if the incoming students were administered a local placement examination upon entry and placed into appropriate remediation as needed.

To comply with the statute, the "15-Hour Rule" was rescinded effective in the fall semester 1993. As a result, it appears that a less-prepared student population is being tested, lowering the passing rate statewide. The nine-hour rule for all institutions now captures many TASP-tested students who otherwise would have had an additional semester preparation, particularly at the two-year community and technical colleges, where many students enter for limited retraining or personal enrichment only. Many of these students now test under the statute at nine hours, perhaps failing a section or more, and never attempt a retest because they have achieved their limited goal and have left higher education.

The effect of the reduction of remediation hours taken. These results provide an objective demonstration that remediation positively impacts student performance where there is the manifested need for improvement in the preparedness of beginning students.

In conclusion, until the fall 1993 semester, the TASP Test had a common passing standard based on common test content and a common tested population, and trend comparisons were valid. One of these elements, the tested population, changed in 1993-94. Because of this change, comparisons of academic years 1993-94 and 1994-95 with previous years should be approached with caution.

TREND ANALYSIS

Annual TASP Test performance data allow for a trend analysis of that data. The data are from the institutional CBM 002 TASP reports to the Coordinating Board and from National Evaluation Systems (NES), Inc. data tapes. NES is the contractor responsible for the publication, administration, scoring and score reporting of the TASP Test. All trend data figures refer to students at Texas public postsecondary institutions during the academic years 1989-1990 through 1994-1995. The figures reflect all attempts -- initial or repeat -- to complete the test section or battery.

Figure 1 shows by academic year the raw number of students who were administered the TASP Test and the number of students successfully completing all three sections (passing the TASP Test) during that academic year. The percent of students passing the TASP Test is computed by dividing the "Passing" figure by the "Tested" figure. The proportion of students passing the TASP Test has generally averaged about 65 percent each year until 1993-94, when program changes that altered the tested population were implemented, resulting in a lower overall passing rate (see Program Changes Effective 1993).

Figure 1

Figure 2 illustrates the proportion of all students passing the individual subject matter tests of reading, mathematics, and writing. The downward trend for passing the reading portion continues. The trend in mathematics, however, shows an increase in the pass rate of 2.4 percent over the preceding year. The trend in writing success was fairly steady, but shows a decrease over the last year. The recent increase in mathematics performance and the lack of increase in reading or writing performance may reflect a national trend that has emphasized mathematics instruction, possibly at the expense of the language arts (U. S. Department of Education, Condition of Education, 1996).

Figure 2

Figures 3, 4, and 5 show total test data by ethnicity. Increases in the raw numbers and in the proportion of Blacks, Hispanics, and Asians attempting the TASP Test are evident (Figures 3 and 4). Proportions passing continue to increase steadily for Blacks and Hispanics, before and after the implementation of the programmatic changes in 1993-94. The passing trend for Asians continues to decrease (Figure 5).

Figure 3

While the raw numbers of White students dominates in Figure 3, Whites have accounted for a steadily decreasing proportion of the students tested (Figure 4).

Figure 4

Although for 1994-95 the overall passing rate of all students remained unchanged from 1993-94, and although passing rates for all students by subject except mathematics declined slightly or remained "flat," there were slight increases in the passing rates of Blacks and Hispanics (Figure 5). Pass rates by subject matter and ethnicity are given in Appendix C; overall pass rates are given in Figures 2 and 5).

Figure 5

Future trends are expected to show another overall decline in the pass rate as a result of the increased standards that became effective fall 1995. The new standard requires a student to answer one additional item correctly (67 percent of the items total) in reading and three additional items correctly (56 percent of the items total) in mathematics. It is expected that the pass rate will fall below 50 percent overall. However, changes in high school graduation requirements should positively affect the pass rate in the near future.

COHORT STUDIES

Introduction

A "cohort" is defined as a "group of individuals having a statistical factor in common in a demographic study" (Webster's Ninth New Collegiate Dictionary). The cohorts studied here are first-time entering freshmen who remain in school for at least two semesters, not necessarily contiguously, through the following fall semester. The cohort data presented below do not represent trends; rather, the data mark the progress of the cohort to date. When data are given for a cohort over several years, the latest data given should be considered the most accurate. Trend inferences may be made across cohorts only when looking at the same time period (e.g., three years' data) for each cohort.

Higher Education in Texas, 1993

The present study concerns the 29 public universities and 69 public community and technical college campuses submitting TASP reports to the Coordinating Board from 1989 to 1993, the entry date of the latest cohort included in the study. Texas ranked third nationally in the number of private and public institutions of higher education in 1991 and fourth in 1993 ("Facts on Higher Education in Texas," 1991, 1993, 1995). Public universities accounted for approximately 50 percent of the student enrollment in public higher education in Texas, while community and technical colleges accounted for the remaining 50 percent.

Freshmen Enrollment

In 1993, freshmen (both full-time and part-time) accounted for 44 percent of the student body headcount, with 24 percent attending four-year institutions and 76 percent attending two-year institutions. First-time, full-time (FTFT) entering freshmen accounted for 10 percent of the total student population. Approximately 50 percent of FTFT freshmen attended four-year institutions, and approximately 50 percent attended two-year institutions.

Fall-to-Fall Retention

Four-year institutions retained almost 70 percent of their FTFT freshmen from one fall semester to the next fall semester. Community and technical colleges retained slightly more than 50 percent of their freshmen.

Graduation

Over the six-year period 1987 - 1993, 71,977 community and technical college students transferred to a four-year institution, and 13,516 (18.8 percent) earned bachelor degrees. Community and technical colleges awarded 33,982 associate degrees and certificates. At public universities, 48.3 percent of first-time, full-time freshmen (enrolled in 12 or more semester credit hours) graduated within six years (1994 Statistical Report, Texas Higher Education Coordinating Board).

These general figures should be considered with the cohort data below. For example, the graduation figures cited above are six-year figures for first-time, full-time freshmen who entered fall 1987. At the community and technical colleges, FTFT students account for only slightly more than 12 percent of the student population. More than 85 percent enrolled part-time and may still be attending school part-time or may have completed a personal training or re-training goal short of earning a degree or certificate and left Texas public higher education. Presently there is no way to distinguish between students in the cohort data who attend part-time and those who have left higher education, and there is no way to exclude these students from the cohort database. This may cause a misleading and inaccurate remediation completion rate.

Design

Three principles were established for assessing the effectiveness of remediation. These principles guided the development of the evaluation criteria and the data collection system.

The first principle requires student academic progress to be monitored over time. Students must be given a sufficient amount of time for learning to take place. Indicators of academic progress must be collected at several points during the students' academic careers for accurate measurement of progress.

Second, multiple indicators of student progress and performance are needed. Because the law requires that all skills-deficient students participate in remediation, controlled experimental studies designed to isolate the effects of remedial programs cannot be conducted. Multiple indicators of effectiveness help compensate for the lack of experimental control. Any single indicator, such as grade point average, is subject to influences beyond the effects of remediation. Patterns found in multiple indicators, however, can lead to informed judgments regarding program effectiveness.

Finally, the performance of skills-deficient students must be compared to the performance of students who do not require remedial interventions. An effective remedial program should enable students to earn similar grades -- remediated students will perform with a degree of proficiency of no less than 80 percent of college-ready students, as determined by the TASP Evaluation Committee -- and to progress toward graduation comparably with students who did not require remedial programs.

The following indicators, used in combination, form the basis of comparison between students requiring remediation and college-ready students who do not:

Since the implementation of the TASP in 1989, each of these performance measures has been collected annually for all students enrolled in a Texas public college or university. Students can be individually identified, allowing a continuous record of enrollment and performance throughout their academic careers, even as they transfer from one institution to another or leave public higher education and then return. Data for those students who transfer to out-of-state or to independent institutions, however, are not available.

Evaluation Model

For this evaluation, students were categorized as: "Remediation Not Required," "TASP Required Remediation," "Locally Required Remediation," and "Untested." Generally, all entering students must take the TASP Test prior to the end of the semester in which they accumulate nine or more college-level semester credit hours. However, to assist with placement decisions, institutions may elect to give a local diagnostic placement test to students upon entry. "Untested" students include students in TASP-waived certificate programs. In each category, the students were organized by subject area and ethnicity.

This Annual Report on the TASP and the Effectiveness of Remediation reports on five cohorts, beginning with academic year 1989-90. The cohort for any academic year is defined as all students entering Texas public postsecondary education for the first time during the academic year and attending for at least one additional semester through the next fall semester. Final results are reported for the sixth year of study for those students in the 1989-95 cohort. Interim results are reported for the fifth year of study for the 1990-96 cohort, the fourth year of study for 1991-97 cohort, the third year for the 1992-98 cohort, and the second year for the 1993-99 cohort. Cohorts are reported for the first time after two years of existence. Each cohort will be followed for six years.

The first Annual Report on the Effectiveness of Remediation (August 1992) examined students who were TASP-tested only. Under guidance from the advisory TASP Evaluation Committee, subsequent reports analyze data for all students placed in remediation, through either TASP testing or through local placement testing. Cohort results are reported by subject area, ethnicity, and institution, with statewide two-year institution and four-year institution summaries. These results are available on the Internet via file transfer protocol (Appendix D). Statewide cohort data are reproduced in Appendix E.

Only the 1989-95 cohort has completed the six-year study period. No specific conclusions will be drawn on the effectiveness of remedial efforts for the other cohorts until they complete their respective six-year study period. Final reports on the subsequent cohorts will be provided as they complete the study. Trends across the cohorts may be analyzed at that time.

Completed Cohort Studies

Only one cohort study is complete. Cohort 1989-95 contains 123,080 students, of whom 83,603 were tested for basic skills. Final data show that 53.6 percent of the tested students entered remediation. Of those students, 54.1 percent completed the required remediation. After six years, the following results are shown:

Table 1
Indicator One: Retention in Higher Education
Remaining in School After:

Two Years Three Years Four Years Five Years



Six Years
Total
Cohort 1989-95
91%* 55.2% 45.6% 34.9% 28.8%
Remediation
Completed
89% * 70.6% 59.0% 47.9% 40.4%

* Cohort 1989-95 two year retention data computed for TASP-tested students only.

The findings for this indicator are inconsistent because of the way data were initially computed for TASP-tested students only (see Two Years results), rather than for all students in the cohort (years three through six). For this reason, it is not possible to compare initial fall-to-fall freshman retention rates. However, we can see from subsequent years that if a student completes remediation, that student is retained at a rate 12 to 15 percentage points higher than for the cohort total.

Additionally, examining data across different cohorts for the same time period shows results in these subsequent years (years 3 through 5) that are similar, as explained in the section "Interim Cohort Findings To Date".

Table 2
Indicator Two: Grade Point Average
Earning "C" or Better Grade Point

Two Years
Writing Math
Three Years
Writing Math
Four Years
Writing Math
Five Years
Writing Math
Six Years
Writing Math
Cohort 1989-95
Remediation
Not Required
77% Overall 77.7% Overall 78.2% 77.6% 78.5% 78.2% 79.1% 78.7%
Remediation
Completed
74% Overall 68.3% Overall 70.6% 71.8% 70.7% 72.1% 71.4% 73.1%

Final data gathered for the sixth year of existence for Cohort 1989-95 are consistent with earlier data and show that students who complete remediation in writing or mathematics earn overall grade point averages within 10 or fewer percentage points of students who did not require any remediation in those subject areas. Separate writing and mathematics rates were not initially computed for this first cohort.

Table 3
Indicators Three and Four: First College-Level Class Grade (English/Mathematics)
Passing Classroom Performance

Two Years
Writing Math
Three Years
Writing Math
Four Years
Writing Math
Five Years
Writing Math
Six Years
Writing Math
Cohort 1989-95
Remediation
Not Required
85.0% 73.0% 85.4% 71.8% 84.8% 71.2% 84.4% 70.1% 84.1% 69.8%
Remediation
Completed
82.0% 64.0% 67.7% 52.3% 75.1% 54.2% 74.5% 53.5% 74.2% 53.8%

Final data gathered for the sixth year of existence for Cohort 1989-95 are consistent with earlier data and show that students who complete remediation in writing or mathematics pass their first college-level writing class at pass rates within 10 or fewer percentage points of students who did not need remediation in writing. Performance by students completing mathematics remediation in their first college-level mathematics class is consistent but not as high as for students who did not require remediation in mathematics. Remediated students' pass rates are within 20 percentage points of students not requiring mathematics remediation. Note: Two percent of the students in English and five percent of the students in mathematics completed their first college- level mathematics or English class prior to or while concurrently enrolled in remediation for that subject. This indicator, then, should be used as a simple comparison between two student groups -- students for whom remediation is required who eventually complete remediation and students for whom remediation is not required -- regardless of when the student took the first college-level class in relation to when the remediated student completed that remediation.

Table 4
Indicator Five: Highest Certificate/Degree Earned

Four Years Five Years Six Years
Cohort 1989-95
Remediation Not Required
Certificate
Associates
Bachelors
0.9%
5.7%
7.9%
1.3%
6.9%
23.2%
1.3%
6.8%
22.8%
Remediation Completed
Certificate
Associates
Bachelors
1.9%
7.3%
1.0%
2.2%
9.6%
5.0%
2.3%
9.5%
4.9%

Final data gathered for the sixth year of existence for Cohort 1989-95 are consistent with earlier data and show that students who complete remediation earn a larger proportion of certificates and associate degrees, and they earn them sooner than students not requiring remediation. Students who do not require any remediation continue their education through the baccalaureate degree at a much higher proportion than students who require and complete remediation. Variation from the fifth year's data to the sixth year's data is due to record updating. Percentages for either year round to the same nearest whole percentage.

Table 5 Cohort Outcomes Indicators by Ethnicity

1989-95 (6 yrs. data) Remediation Not Required
White Black Hispanic Other
By Subject:
Mathematics 60.5% 30.0% 38.9% 61.1%
Reading 74.4% 39.4% 49.6% 66.7%
Writing 67.6% 33.0% 40.6% 43.3%
GPA: Mathematics 81.0% 66.6% 71.5% 84.9%
Reading 80.4% 66.4% 70.9% 84.4%
Writing 81.3% 67.9% 72.0% 84.9%
Pass First College-Level Math 70.8% 63.4% 65.9% 74.1%
Pass First College-Level English 84.7% 82.5% 82.3% 83.5%
Remediation Completed
By Subject:
Mathematics 45.3% 26.7% 32.7% 31.9%
Reading 26.2% 26.8% 27.2% 38.0%
Writing 36.8% 28.3% 33.1% 42.0%
GPA: Mathematics 76.0% 63.6% 69.6% 76.1%
Reading 69.3% 66.7% 69.5% 74.6%
Writing 72.4% 67.5% 70.3% 75.5%
Pass Math 54.8% 50.5% 51.6% 62.7%
Pass English 74.0% 78.1% 71.6% 78.4%

Table 5 shows by ethnicity for the 1989-95 cohort the proportions of students completing or not requiring remediation by subject area, earning grade point averages (GPA) of at least 2.0 (C grade average) in each of the three areas, and passing their first college-level mathematics or English class with a grade of D or higher.

Interim Cohort Findings To Date

The cohorts studied here are a sampling from the total freshman class. Numbers of tested/not tested students and enter/complete remediation are from the cohorts, as presented in Table 6.

Table 6
Cohort Comparison Numbers (as of Academic Year 1994-95)

Cohort
1990-96
Cohort
1991-97
Cohort
1992-98
Cohort
1993-99
Freshman
Headcount
346,584 349,668 351,710 351,561
Total Cohort 146,979 148,756 142,665 144,355
Not Tested
Population
33,272 27,840 23,295 24,990
Tested
Population
113,645 120,916 119,370 119,365
Enter
Remediation
57,673
(50.7%)
59,580
(49.3%)
57,944
(48.5%)
55,671
(46.6%)
Complete
Remediation
32,829 (56.9%) 33,911 (56.9%) 32,135 (55.5%) 29,066 (52.2%)

Statistics comparing students who complete remediation with the total cohort population show the proportion for each 1) still in school and 2) by highest award earned (Table 7). Classroom performance and grade point average earned is then shown by tested subject matter for students who required remediation compared to students who did not require remediation (Table 8). (Note: Overall student grade point average is computed by the subject area of interest. Any student may be included in both the mathematics and the composition subject areas if the student failed both sections of either the local placement or TASP Test.)

Table 7
Interim Retention and Award Earned Outcome Measures

Remaining
In School After

Two Years Three Years Four Years Five Years
Cohort 1990-96
Total Cohort
73.0% 55.9% 46.1% 39.5%
Remediation
Completed
84.7% 70.1% 57.7% 49.1%
Cohort 1991-97
Total Cohort
71.2% 55.7% 48.4%
Remediation
Completed
83.8% 67.8% 56.8%
Cohort 1992-98
Total Cohort
70.0% 57.2%
Remediation
Completed
81.5% 66.6%
Cohort 1993-99
Total Cohort
72.6%
Remediation
Completed
83.3%
Highest Award Earned
Cohort 1990-96
Remediation Not Required
Certificate


1.3% 1.4%
Associates


4.2% 6.3%
Bachelors


4.1% 9.9%
Remediation Completed
Certificate


2.0% 2.0%
Associates


7.8% 7.7%
Bachelors


1.3% 1.3%
Cohort 1991-97
Remediation Not Required
Certificate


1.1%
Associates


4.4%
Bachelors


1.9%
Remediation Completed
Certificate


1.7%
Associates


4.2%
Bachelors


1.9%


Table 8
Grade Point Earned and Class Performance Outcomes

Grade Point
Earned "C" or Better
Two Years Three Years Four Years Five Years

Writing Math Writing Math Writing Math Writing Math
Cohort 1990-96
Remediation
Not Required



76.8% Overall 77.5% 77.4% 78.0% 77.9% 78.6% 78.5%
Remediation
Completed
66.3% Overall 70.0% 71.5% 70.3% 71.7% 70.8% 72.1%
Cohort 1991-97
Remediation
Not Required
78.1% 78.0% 77.8% 78.1% 78.2% 78.5%
Remediation
Completed
71.7% 71.8% 70.6% 71.1% 70.5% 71.5%
Cohort 1992-98
Remediation
Not Required
78.3% 78.9% 78.2% 79.2%
Remediation
Completed
70.4% 71.4% 70.5% 71.1%
Cohort 1993-99
Remediation
Not Required
76.7% 78.0%
Remediation
Completed
71.9% 72.1%
Passing Classroom
Performance
Writing Math Writing Math Writing Math Writing Math
Cohort 1990-96
Remediation
Not Required
85.4% 72.5% 84.7% 71.5% 84.4% 70.7% 84.1% 70.2%
Remediation
Completed
70.3% 54.0% 77.7% 54.9% 76.5% 54.2% 76.1% 54.2%
Cohort 1991-97
Remediation
Not Required
85.9% 72.7% 85.3% 71.4% 84.9% 70.6%
Remediation
Completed
77.2% 54.1% 75.7% 53.2% 74.9% 53.4%
Cohort 1992-98
Remediation
Not Required
85.2% 72.2% 84.5% 71.1%
Remediation
Completed
76.5% 57.2% 75.5% 56.9%
Cohort 1993-99
Remediation
Not Required
84.1% 71.9%
Remediation
Completed
76.4% 60.9%


Conclusions and Cautions

Conclusions regarding completed cohorts are given in the section for "Completed Cohort Studies." None of the interim cohorts reported here is complete. All data are interim, and all conclusions based on these data should be approached with caution. However, some overall similarities can be seen between the completed 1989 cohort and the interim cohorts. Statewide, students who completed required remediation are generally performing in college at levels comparable to those students who did not require remediation. Data presented in Tables 7 and 8 reveal that students who complete remediation generally:

Table 7 shows that students who complete remediation stay in school at a higher rate and earn certificates and associate degrees at a higher rate than students not requiring remediation. However, students who do not require any remediation earn baccalaureate degrees at a much higher rate than students who require and complete remediation.

The data as presented here support only limited trend analysis because of differences in time references. Such trend analyses are possible only when enough data has been collected to compare the different cohorts over the same amount of time (for instance, looking at Cohort 1990-96 through Cohort 1992-98 after three years' time in Table 8). More accurate comparisons will be possible only after each cohort completes the six-year study period. However, not much additional information is gleaned from Table 8 after the initial reporting.

Finally, the percentage of students completing remediation may be considered low by some readers (52 - 57 percent; see Table 6). Why students do not complete remediation, which is part of the larger question of why students leave higher education, is a continual concern for educators. Remediation completion rates may be low for a number of reasons. Students most often cite such personal reasons as finances for leaving higher education rather than poor academic performance (How College Affects Students, Ernest Pascarella and Patrick Terenzini, Jossey-Bass, 1991 and What Matters in College: Four Critical Years Revisited, Alexander Astin, Jossey-Bass, 1993). Remediation completion rates by ethnicity are reported in Appendix F.

The cohort data reported here include part-time students. These students, particularly at community colleges, may remain in school only long enough to achieve a personal goal of retraining or personal enrichment and, in the process, become TASP liable. Having achieved their personal goal, the students leave higher education without completing remediation. The issue of how to deal with this anomaly has not been satisfactorily resolved.

As a cohort study, the fifth Annual Report on the TASP and the Effectiveness of Remediation does not analyze TASP Test results from all TASP-tested students nor all locally tested students (see the definition of cohort). Each cohort should be seen as a subset of all students entering Texas higher education, and the data presented as a "snapshot" of that subset at a given point in time. The study of several of the cohorts presented here is still under way, and each will not be complete until the cohort exists for six years. No specific conclusions concerning the effectiveness of remediation programs in Texas public postsecondary institutions should be made based upon these interim reports.

The Texas Academic Skills Program is not just an examination, but a larger academic support program that includes local remedial efforts. Students attend the same remediation classes regardless of whether or not placement is based on the TASP Test or a local placement test. Data from the cohort analyses therefore represent all remediation efforts and not only TASP-required remediation.


Appendix C
Percent Passing
TASP Subject Test by Ethnicity

Academic
Year 1989-90
Academic
Year 1990-91
Academic
Year 1991-92
Academic
Year 1992-93
Academic
Year 1993-94**
Academic
Year 1994-95
Reading
Whites 93.6% 93.0% 92.6% 92.7% 87.8% 86.6%
Blacks 72.4% 73.0% 73.6% 73.9% 64.4% 63.9%
Hispanics 80.3% 80.2% 79.6% 79.6% 70.4% 69.8%
Asians 76.7% 71.9% 69.3% 68.7% 54.2% 53.0%
Others 77.0% 72.8% 72.1% 72.1% 59.8% 59.2%
TOTAL 88.5% 87.3% 86.2% 85.8% 78.0% 76.7%
Mathematics
Whites 84.4% 83.7% 82.5% 82.8% 72.9% 75.3%
Blacks 54.7% 58.5% 56.9% 58.3% 41.4% 46.0%
Hispanics 68.7% 70.7% 68.4% 69.5% 55.7% 59.0%
Asians 89.8% 89.5% 88.4% 90.8% 83.5% 83.7%
Others 85.2% 86.0% 85.2% 86.5% 79.1% 77.9%
TOTAL 78.7% 78.8% 76.8% 77.3% 65.3% 67.7%
Writing
Whites 86.2% 88.5% 89.2% 90.2% 85.8% 84.0%
Blacks 60.6% 66.4% 67.9% 68.9% 62.9% 59.5%
Hispanics 68.5% 73.4% 74.9% 75.4% 69.7% 65.5%
Asians 60.3% 59.0% 61.0% 57.5% 48.9% 47.9%
Others 60.0% 57.8% 61.5% 66.3% 51.6% 49.2%
TOTAL 79.3% 81.4% 81.7% 81.8% 75.9% 72.9%

** See Program Changes Effective 1993


APPENDIX D

FILE TRANSFER PROTOCOL (FTP)

Data for cohort 1989-1995 (six years' data), cohort 1990-96 (five years' data), cohort 1991-97 (four years' data), cohort 1992-98 (three years' data), and cohort 1993-99 (two years' data) for each institution are available in Lotus compatible (.wk1) spreadsheets on the Internet anonymous FTP server at:

info.thecb.texas.gov
(192.16.72.17)

Look in the misc/taspdata/taspsr or misc/taspdata/taspjr directories for the four-year and two-year schools, respectively. The files are named according to fice code, e.g., 003541.WK1 is Angelo State University.

For statewide information, see Appendix E.

Remember to retrieve these spreadsheets as binary files! The extension WK1 must be capitalized, also.

If you are unfamiliar with FTP, contact your local network manager or systems operator (sysop). If you have questions about the data, contact

braselml@thecb.state.tx.us


APPENDIX E
Cohort Data: Statewide Summaries

Statewide summaries are available for all cohorts by all senior institutions, all junior institutions, and all institutions.

To retrieve statewide summaries for all senior institutions, click here.

To retrieve statewide summaries for all junior institutions, click here.

To retrieve statewide summaries for all institutions, click here.


APPENDIX F
Cohort Outcome Indicators by Ethnicity

Remediation Not Required Remediation Required Remediation Completed
White Black Hispanic Other White Black Hispanic Other White Black Hispanic Other
1990-96 (5 yrs data)
Mathematics 63.2% 35.1% 41.8% 69.0% 36.8% 64.9% 58.2% 31.0% 47.7% 31.8% 35.8% 38.1%
Reading 76.6% 46.1% 52.2% 61.0% 23.4% 53.9% 47.8% 39.0% 28.3% 30.8% 29.5% 41.9%
Writing 70.8% 39.8% 43.5% 50.9% 29.2% 60.2% 56.5% 49.1% 39.3% 33.5% 37.5% 46.3%
GPA: Math 81.0% 65.2% 70.7% 85.8%



75.9% 61.5% 67.6% 75.6%
Reading 80.4% 64.6% 69.9% 85.1%



69.2% 66.4% 68.4% 77.0%
Writing 81.2% 65.6% 70.3% 85.4%



71.0% 67.5% 70.3% 76.4%
Pass Math 71.5% 62.3% 65.3% 76.0%



55.4% 51.5% 52.6% 63.3%
Pass English 85.0% 82.5% 81.5% 82.7%



75.6% 78.5% 74.8% 80.2%
1991-97 (4 yrs data)
Mathematics 64.6% 35.8% 43.4% 69.5% 35.4% 64.2% 56.6% 30.5% 47.7% 30.4% 36.3% 38.3%
Reading 77.9% 47.0% 53.7% 61.6% 22.1% 53.0% 46.3% 38.4% 27.7% 29.7% 31.1% 40.9%
Writing 72.7% 42.3% 46.1% 52.8% 27.3% 57.7% 53.9% 47.2% 38.9% 32.5% 38.5% 43.6%
GPA: Math 81.4% 64.4% 70.1% 86.3%



74.9% 61.8% 67.3% 75.7%
Reading 80.5% 63.5% 69.8% 85.2%



68.1% 65.7% 68.9% 75.6%
Writing 81.2% 64.3% 69.9% 85.3%



70.7% 68.0% 70.1% 74.9%
Pass Math 72.0% 61.7% 66.7% 75.5%



53.1% 51.1% 52.6% 66.1%
Pass English 85.6% 83.4% 83.0% 84.3%



74.6% 75.8% 73.4% 79.9%
1992-98 (3 yrs data)
Mathematics 65.4% 37.7% 41.3% 70.0% 34.6% 62.3% 58.7% 30.0% 45.9% 25.1% 33.5% 33.5%
Reading 78.8% 49.8% 51.7% 61.3% 21.2% 50.2% 48.3% 38.7% 27.6% 25.6% 29.9% 37.7%
Writing 74.6% 45.7% 45.1% 53.0% 25.4% 54.3% 54.9% 47.0% 37.6% 28.4% 37.0% 40.3%
GPA: Math 81.5% 65.2% 71.9% 85.8%



73.7% 63.3% 67.5% 76.3%
Reading 80.3% 64.5% 70.9% 85.2%



68.6% 71.4% 68.8% 76.9%
Writing 80.7% 64.7% 71.1% 85.6%



70.6% 69.2% 70.0% 67.3%
Pass Math 72.3% 62.0% 67.4% 75.4%



57.8% 52.6% 54.7% 67.4%
Pass English 85.1% 82.3% 82.8% 84.8%



75.0% 77.5% 74.4% 78.9%
1993-99 (2 yrs data)
Mathematics 65.6% 39.1% 46.2% 68.7% 34.4% 60.9% 53.8% 31.3% 41.2% 21.0% 31.8% 33.0%
Reading 79.2% 53.1% 57.4% 60.7% 20.8% 46.9% 42.6% 39.3% 26.3% 22.5% 29.4% 31.5%
Writing 76.4% 50.2% 53.3% 52.9% 23.6% 49.8% 46.7% 47.1% 23.9% 23.4% 33.9% 33.2%
GPA: Math 80.8% 64.3% 70.3% 84.8%



74.4% 64.7% 69.2% 76.9%
Reading 79.7% 63.5% 69.8% 84.4%



69.9% 69.8% 70.1% 79.5%
Writing 79.7% 62.6% 69.6% 85.2%



71.4% 71.8% 71.5% 75.4%
Pass Math 73.9% 62.2% 67.1% 74.9%



61.6% 57.4% 59.0% 69.3%
Pass English 85.3% 81.0% 82.0% 83.1%



76.7% 77.2% 74.2% 81.8%


Appendix G: Glossary

Cohort Subset of higher education students defined as first-time entering students who remain in school for at least two semesters through the following fall semester (not necessarily contiguously). Overall cohort "N" (number) is computed by adding the unduplicated counts plus "untested."
Cohort Analysis Study of a subset of students after a given amount of time on a given set of outcomes. Not a trend analysis unless the cohorts are studied at the same point in time (after the same amount of time has elapsed). As presented in this study, the cohort analysis may be used for limited two- to three-year trends.
Cohort Data / Study Numbers based on the performance of cohort students from the time of their entry into the cohort through the next six years. Cohorts identified to date are Cohorts 1989-95, 1990-96, 1991-97, 1992-98, and 1993-99, representing the beginning and final years of the study period for the cohort.
Ethnicity Self-reported student data describing students' ethnic origin. "Others" includes Asian students, Native American students, and foreign students.
Exemption Legislation passed by the 73rd Texas Legislature allowed for students to be exempted from the TASP (but not local placement testing) based on high performance on the Scholastic Assessment Test (SAT), American College Test (ACT), or the Texas Assessment of Academic Skills (TAAS) test. Exemption cut-score levels are set by the Texas Higher Education Coordinating Board.
Grade Point Average (GPA) Outcome Number of student with a GPA equal to or better than a 2.0 (C) average after completion of the first college- level mathematics or English class.
Highest Award Earned Number of certificates, associate or bachelor degrees earned to date by cohort students. This should not be confused with the "graduation rate" published by the Texas Higher Education Coordinating Board.
Locally Required Remediation Cohort students who have passed TASP testing but have failed a local placement test in the subject. "Unduplicated count" represents students liable for remediation locally in one or more than one subject.
N: Number of students.
Passing First College-Level Class Outcome Passes are divided by attempts for the given academic subject. A passing grade is "D" or higher. Computed for English and mathematics (there is presently no measure for first college-level reading course).
Placement Testing Tests used to diagnose students' preparedness for college coursework. The TASP Test sets the statewide minimum; local placement requirements may be higher.
Remediation Completed Data for those students who complete remediation are summarized in this report for purposes of comparison with students who do not require remediation. Detailed numbers are available from the TASP Office staff, who compile this separate analysis of cohorts.
Remediation Not Required Cohort students who passed the TASP Test by subject shown. "Unduplicated count" represents students who passed more than one section.
Remediation Required All remediation required through TASP and local testing.
Retention Students remaining in school. Students identified as retained either at the original (local) institution or by transfer to another institution are summed. That sum is divided by either the cohort number N (overall retention) or by the ethnicity number N (retention by ethnic group).
Semesters in Remediation Number of semesters of remediation provided.
TASP Liable Students required to take the TASP Test. Unless waived or exempted, since 1989 all students attending a public postsecondary institution for more than nine semester credit hours have been required to take the TASP Test. A "non-attempt" on a section of the TASP Test is considered the same as a failure in the tested subject.
TASP Required Remediation Remediation required of cohort students who either have failed both local and TASP placement testing in subject shown or have passed local placement testing and have not yet attempted the TASP Test. "Unduplicated count" refers to students liable for remediation in one or more than one subject.
TASP Untested Students waived from the TASP requirement by exemption or enrollment in special certificate programs. The untested students are subtracted from the overall cohort N (number) to arrive at the tested N (number), or the number of students who had to take the TASP examination.
TASP-Waived Certificate Students enrolled in certain certificate programs with less than nine general education hours are waived from the TASP.
Tested Population Cohort students required to take the TASP Test.
Trend Analysis Comparisons of student performance for an academic year with student performance from subsequent academic years on a common set of measures to show improving or declining performance. A cohort analysis is not a trend analysis unless cohorts are compared at the same point of time duration.


APPENDIX H
SUGGESTED READINGS

Alpert, Richard T., William Phillip Gorth, and Richard Allen. Assessing Basic Academic Skills in Higher Education: The Texas Approach. National Evaluation Systems, Inc.: Lawrence Erlbaum Associates, 1989.

Astin, Alexander. What Matters in College: Four Critical Years Revisited. Jossey-Bass, 1991.

Hardesty, Robert L., et al. A generation of failure: The case for testing and remediation in Texas higher education. Final report and recommendations of the committee on testing to the Coordinating Board, Texas College and University System. Texas Higher Education Coordinating Board, July 17, 1986.

Matthews, Joan M., Ronald G. Swanson, and Richard M. Kerker (eds.). From Politics to Policy: A Case Study in Educational Reform. Praeger, 1991.

Pascarella, Ernest and Patrick Terenzini. How College Affects Students. Jossey-Bass, 1991.

Report of the developmental education work group. Texas Association of Junior and Community College Instructional Administrators: May 1995.

Skinner, Elizabeth Fisk and Stephen Carter. A second chance for Texans: Remedial education in two-year colleges. Texas Association of Junior and Community College Instructional Administrators: National Center for Postsecondary Governance and Finance, Research Center at Arizona State University, June 1987.


FOR MORE INFORMATION, CONTACT Ray Fischer

Program Director

TASP Office

Texas Higher Education Coordinating Board