How Good Are Zillow's Home-Price Estimates?

Since its launch, the Web site has made millions of Americans familiar with computer-generated estimates of home values. But just how accurate is it?
By: James R. Hagerty: The Wall Street Journal Online
In the year since its launch, Zillow Inc. has made millions of Americans familiar with computer-generated estimates of home values, created a new online addiction and become a staple of dinner-party chatter.

But just how accurate is it? A Wall Street Journal analysis of 1,000 recent home sales shows that Zillow's "Zestimates" often are very good, frequently within a few percentage points of the actual price paid. But when Zillow is bad, it can be terrible - off the mark by more than 25% on one in 10 homes. In one case it was off by $2 million.

Zillow, based in Seattle, operates a Web site that offers free estimates and other online tools for real-estate buyers and sellers. It draws revenue from online advertising.

The Journal looked at transaction prices recorded for 1,000 recent home sales in seven states, using data from First American Real Estate Solutions, a data provider in Santa Ana, Calif., and compared those prices with Zillow estimates, which didn't yet reflect the sales. The median difference between the Zillow estimate and the actual price was 7.8%. (That was close to the 7.2% median "margin of error" reported by Zillow itself on all transactions involving homes whose value it has estimated.)

The estimates were about equally split between ones that were too high and those below the mark.

A four-bedroom, five-bath house in Fall City, Wash. Zillow estimate: $661,756 Sold for: $2,690,000

Zillow came within 5% of the price in a third of the transactions studied by The Journal. It was more than 25% off target on 11% of them. In 34 of the 1,000 transactions, Zillow was off by more than 50%.

Zillow had estimated that a four-bedroom, 7,600-square-foot home in Fall City, Wash., was valued at $661,756. The home, built last year, sold in early January for $2.7 million. "If you don't visit the property, you're never going to know that it's in an exclusive, gated part of the neighborhood," says Maria Danieli, who represented the sellers. Ms. Danieli says Zillow may be fine for "cookie-cutter" neighborhoods but "they can't compute" the values of the luxury homes she sells.

Zillow executives acknowledge that the estimates can be way off in some cases. The estimate "is a starting point" for people trying to figure out how much a home should cost, says Amy Bohutinsky, a spokeswoman for the company. "We don't recommend it as the final word."

Sometimes the estimates take big lurches in brief periods as Zillow's computer analyzes the latest home-sales data, updated weekly. "My God!" said Jonathan Miller after he looked up his own house in Darien, Conn., on Zillow last week. "My value has dropped 25% in six months. There's no way - that would be the market collapsing!" Zillow has the house pegged at $1,442,851, down from about $2.1 million last July. Mr. Miller, chief executive officer of Miller Samuel, an appraisal firm based in New York, watches his local market closely and figures his home is valued at around $1.9 million.

Zillow can be quite accurate in some markets, Mr. Miller says, but he argues that the estimates are hit or miss. He suggests that Zillow should produce only an estimated price range rather than an exact figure: "When you go down to the $1 level, you're implying precision." Ms. Bohutinsky, the Zillow spokeswoman, notes that Zillow produces both a range and a precise estimate and says users like both.

Zillow also missed the target for Josh Benton, a management consultant at Kurt Salmon Associates in Atlanta. He sold a home last fall for about 15% more than Zillow's estimate. Still, Mr. Benton says he found Zillow useful for getting a sense of the relative value of houses in a neighborhood. And he liked the site's aerial views of neighborhoods as a research aid. "Overall, it's an excellent site," he says.

El Cerrito, Calif. Zillow estimate: $544,361 Sold for: $80,000

Zillow's estimates come from a proprietary computer program that takes into account sale prices for nearby homes that appear comparable, the size and other physical attributes of the home, its past sales history and tax-assessment data, says Stan Humphries, vice president of data and analytics.

Zillow tends to work best for midrange homes in areas where there are a lot of comparable houses, he says. It is less accurate for low- and high-end homes because there are fewer of those and thus less data available from comparable sales, known as "comps." Values of rural homes are hard to gauge for the same reason. Partly for that reason, none of the Web sites can offer 100% coverage of U.S. homes; Zillow says it has estimates on about 57% of all homes.

Even where there are numerous apparent "comps," computer programs like Zillow's can stumble when vital information is missing. Data fed into the computer, for instance, may not reflect the fact that a house has just been remodeled, destroyed by fire or put into foreclosure. Reported prices can be misleading, too. Sometimes homes are sold between family members for a token price, or sellers offer incentives to buyers, such as help with closing costs, that aren't reflected in the recorded price.

Zillow isn't the only site offering such free estimates. Others include RealEstate.com, RealEstateABC.com and Reply.com. But Zillow's site gets more traffic than those rivals. All of these sites appear to have overestimated the value of a house on Olivant Street in a tough area of Pittsburgh that sold for $700 last year in an auction of foreclosed homes. As of early this month, Zillow estimated the value at about $33,000, RealEstate.com at $57,000, Reply.com at $69,000 and RealEstateABC.com at $86,000. Several nearby houses are abandoned or boarded up, blighting the block - something computer models don't take into account.

Northbrook, Ill. Zillow estimate: $450,565 Sold for: $970,000

Zillow lets users try to correct for things computers might miss. For instance, people can use Zillow's "my estimator" tool to account for the value of remodeling or to choose what they regard as the most relevant "comps," screening out those that aren't really similar homes.

Making estimates is particularly difficult in some areas. Some states, including Texas, don't provide public records showing housing transaction prices. To improve its performance in Texas, Zillow last month began tapping sales data from multiple-listing services, databases of property for sale through real-estate brokers.

New York City also is a difficult area for Zillow and other estimate providers because of the large number of cooperative buildings. Tax assessments are made on the co-op buildings rather than individual units, removing one indicator of value. And there is less public information on such things as square footage in individual homes, Zillow says, though the company is finding alternative sources for such data.

Real-estate agents and appraisers tend to sneer at Web site valuations and insist that consumers still need their local expertise to get a true idea of values. Masood Samereie, an agent at Century 21 Hartford Properties in San Francisco, says one of his clients last year lost his chance to buy an attractive home because, relying on Zillow, he made an unrealistically low offer.