Daniels Educational Consultanting

Education Week

By Caralee J. Adams February 24, 2011

 The anxiety of high school juniors—and their parents—over taking college-entrance exams is creating a market force for the test-prep industry.

Dozens of companies are flooding mailboxes with offers to hire tutors and enroll in classes, including online courses to make preparation for the high-stakes tests more convenient and customized.

Those in the business claim students will improve their performance, and many offer money-back guarantees. Yet outside research shows coaching has minimal positive effects, although there hasn’t been a randomized, controlled experiment to isolate the impact of test prep.

No federal agency has stepped in to provide industry oversight, so experts suggest that consumers do their homework before shelling out money and make sure the prep service is the right fit for their child. For some students, one-on-one tutoring is the most cost-effective; others do best in a classroom. And for many, a $25 test-prep book or free online tests is all they need.

Derek Briggs, chairman of the research and evaluation methodology program in the School of Education at the University of Colorado at Boulder. Briggs’ research has found that there was a positive—but slight—bump in students’ scores as a result of SAT prep courses.

—Nathan W. Armes for Education Week

“Parents are sacrificing, even borrowing on their credit cards to pay these high prices for prep courses,” said Dave Berry, a co-founder of and senior adviser for College Confidential, a college-admissions website. “In fact, kids—if they are dedicated, that’s a big if—can get the prep books and do the exercises and most likely increase their scores to within a reasonable degree of the amount they could get through the prep courses.”

No matter what a student chooses, counselors caution that students keep in mind test scores are just one part of the college-admissions decision. According to surveys by the National Association for College Admission Counselors, in Arlington, Va., SAT and ACT scores have consistently ranked third in importance, behind grades and strength of curriculum.

“High SATs do not get you into college, a strong academic record does,” said Debra Shaver, the director of admissions at Smith College in Northampton, Mass. “Students should concentrate more on their homework and worry less about SATs.”

Lack of Evidence

Derek Briggs, the chairman of the research and evaluation methodology program in the school of education at the University of Colorado at Boulder, has examined how much SAT score increases can be attributed to the effect of test prep.

Test-prep programs generally include three elements: a review of test content, practice on test questions, and orientation to the format of the test. In 2009, in cooperation with NACAC, Mr. Briggs reviewed three national data sets and found the average effect of commercial coaching is positive, but slight. Test-score bumps were more in the neighborhood of 30 points (on a 1,600-point scale at the time), far from what some in the industry claim. He does point out that there may be specific programs that are more effective than others, but evidence to support that is weak.

Related Blog

Visit this blog.

Considering the results of Mr. Briggs’ and earlier studies, NACAC concludes that test-prep activities and coaching have a “minimal positive effect on both the SAT and the ACT.”

Mr. Briggs noted that to some selective colleges, 30 points matter among high-scorers. Admissions officers are “naturally drawn to a number,” and it can be one of the first filters in the process, he said. However, the New York City-based College Board, which sponsors the SAT, cautions institutions about making admissions decisions about otherwise qualified students based only on small differences in test scores.

A 2006 review of 10 online websites providing SAT test-prep services by Consumer Reports revealed that 11th graders who used the sites saw their scores improve by an average of 38 points or a 1.6 percent gain.

 Promises and Approaches

Some companies have backed off specific marketing claims. Last May,The Princeton Review, based in New York City, announced it would stop using claims about average test scores in its marketing materials. Kaplan Test Prep has ended its use of testimonials in which test-takers talk about their large score gains.

“In many ways, [claims are] misleading,” said Kristen Campbell, the executive director of college-prep programs at Kaplan, also in New York City. “Instead of throwing up wild claims of points, we tell students we are committed to getting a high score.” Since 2002, her company has offered students their money back or the chance to take the class again if their scores do not improve after taking the course. For competitive reasons, Kaplan won’t release the number that take the company up on the offer other than to say the percentage is low.

Others promise specific point gains and take to task research that disputes the impact of test preparation.

Jake Neuberg and Ramit Varma, who founded Revolution Prep in California eight years ago, said research doesn’t differentiate test-prep methods. With their approach of covering content, as well as managing test anxiety and increasing student motivation, they promise a 200-point gain on the SAT. Students who don’t improve that much can sign up again for free. About 4.5 percent of students repeat the course because their scores did not go up by the guarantee or they had a scheduling conflict, Mr. Varma said.

Although services range from $300 to $900, Santa Monica-based Revolution doesn’t turn away low-income students who can’t pay. About 10 percent of students participate on scholarship.

In Bethesda, Md., Arvin Vohra founded a company and created an approach he calls the “The Vohra Method” in which students’ needs are targeted and addressed in intensive seminars. He guarantees that if, after 24 weeks of training at four hours a week ($25 per hour), the student’s SAT score improves by less than 380 points, and the score is less than 2,250—out of a total of 2,400—the student can take eight more weeks for free. For those who complete the full training, nearly 100 percent meet the 380-point gain, he said.

Postcards from StudyPoint, a Boston-based company that offers in-home tutoring for $70 to $140 in several cities, advertise 200-point SAT gains. Co-founder Richard Enos says the average increase is 171 points, based on comparisons of actual SAT or PSAT tests. “Part of what programs provide is the structure, accountability, and feedback that keep students engaged and motivated,” he said.

Inexpensive Alternatives

As an alternative to high-priced test prep, Number2.com is a free online service where students can take practice tests, get tailored feedback, and find tutorials for each section of the test. Launched in 1999 and sold in 2002 to Xap Corp., the site is an attempt to remove the price barrier, said co-founder Eric Loken, a professor who teaches research methodology at Pennsylvania State University.

His advice to students: “Don’t buy the hype. Too often, people assume because something costs $500, it must be worth it.” Many test-prep classes involve a series of practice tests that students can take at home for free, he said.

Test-prep courses serve a purpose, but more important is the rigor of the classes students take and practicing the test, said Steve Schneider, a counselor at Sheboygan South High School, in Sheboygan, Wis., and the American School Counselor Association’s secondary-level vice president.

“I never tell a family they shouldn’t do it,” he said. “For some, it’s not an issue to drop a couple hundred dollars. But there are more economical and impactful ways to improve your score.”

The best preparation for college-admissions tests is good math and English classes in high school, said Lisa Sohmer, the director of college counseling at the Garden School in Jackson Heights, N.Y. “Students get from these [prep] classes precisely what they put into them,” she said. “They are not a cure-all. They are simply a tool.”

John Boshoven, a counselor in the Ann Arbor public schools in Ann Arbor, Mich., where everyone takes the ACT during the school day, encourages students to take the test once first to see where they need improvement. His school offers free after-school sample tests and others study on their own. “In the Midwest, we don’t have the test mania or college mania as in other parts of the country,” said Mr. Boshoven.

Michigan is one of about a dozen states that offer the ACT or SAT for free to all juniors, and many include free test prep.

Davis Nguyen, 18, a first-generation American from Hampton, Ga., said his Vietnamese family didn’t have the money for a test-prep course. So he made a chart and studied daily for 120 hours total for the SAT. He improved his score by 300 points from one sitting to the next. Mr. Nguyen said he paced himself like a marathon runner and had to keep with the training. “For my future, I had to stay motivated,” he said.

No Short-Cuts

The SAT is designed to measure the academic skills students learn throughout high school and their ability to apply that knowledge, said Angela Maria Garcia, the executive director of SAT publications and information at the College Board. “The best way to get ready is to do well in school, take challenging courses, and read,” she said. “There really is no short-cut to prepare for the SAT.”

The College Board encourages students to use the free resources on its website, such as the SAT question of the day and practice tests to get familiar with the format.

Students often improve their score by merely taking the test a second time, as they gain more knowledge in school and are more comfortable with its structure. Colleges then take the best mix of scores from each sitting.

The Iowa City, Iowa-based ACT Inc. has developed a software program and a more extensive booklet for review, but they’re really not essential, said Jon Erickson, the senior vice president for education services. “We have always been a little bit offended by test prep,” he said. “It’s seen as a last-ditch effort and doesn’t have much effect.”

Others disagree, such as Robert Schaeffer, the public education director for Jamaica Plain, Mass.-based National Center for Fair and Open Testing, or FairTest, which advocates colleges make admission tests optional. “If test-coaching didn’t work, it would be the only human endeavor you can’t improve,” said Mr. Schaeffer. While companies may exaggerate the claims to enhance their economic self-interest, test-makers also underestimate commercial test prep to protect their product, he said.

Keeping Perspective

Test scores matter more than colleges want to admit, yet they don’t trump transcripts, and parents and students need to keep perspective, said Pamela Horne, the assistant vice president for enrollment management and the dean of admissions at Purdue University in West Lafayette, Ind.

“There are companies who are in the business of ensuring that students are anxious and motivated to think this is life and death, as opposed to the idea that in the U.S. there is a place at the table of higher education for every single student,” Ms. Horne said.

Coverage of “deeper learning” that will prepare students to tackle complex issues in fundamentally different ways is supported in part by a grant from the William and Flora Hewlett Foundation, at www.hewlett.org. Vol. 30, Issue 22