This paper does not address mass appraising techniques; it introduces the basic concepts of market modeling to predict the hypothetical sale of a single property. The purpose is to educate appraisers on market modeling, define some basic terms associated with market modeling and dispel some common misnomers.
A regression application that runs in MS Excel can be downloaded by clicking here, along with a short instruction booklet you can download seprately by clicking here. This application will run until the end of this calendar year. This should give interested appraisers some working knowledge of regression analysis. Part II of this writing will specifically address regression analysis.
Editor's note: Download a PDF of this article by clicking here.
I developed an Expert Market system several years ago utilizing a diminishing rate of return engine. I remember demonstrating it to the Appraisal Standards Committee of the Appraisal Institute. They were very impressed with it, but didn't know quite what to do with it; neither did I. Since then I have experimented with artificial intelligence, and regression analysis.
I have developed and in my market area offer a low end product to lenders for low risk loans such as home equity line of credits and first mortgages with low loan to value ratios. I have formed a unique perspective of market modeling, both from the point of view of an appraiser and of a mathematician (if the love of mathematics makes one a mathematician). Most of the work I have done required that I learn to program computers with Microsoft Visual Basic for Applications (VBA) which is the programming language of MS Office.
This computer programming experience gives me a pretty good idea of what the relationship between man and machine really is. In order to follow my line of thinking you must leave behind most everything you have been told or taught about market modeling.
I will introduce several concepts here that are not taken from text books, but rather from my experiences. I am not a statistician, nor is it necessary for an appraiser to be one to use a computer-based market modeling system. However, we will borrow a few tricks of their trade. The term "market modeling" includes mathematical and/or deductive logic (intelligence) analysis that are operated and managed by the human brain and/or computer programs.
IT'S ALIVE, IT'S ALIVE! ACTUALLY, NO IT'S NOT
The human brain is just as comfortable carrying out regression and algorithms computations as the computer processor is at making decisions with computations which imitate the activity of the neurons in the human brain. In fact, the concepts of regression and algorithms were developed hundreds of years before the existence of the modern computer. There is no difference in a market model developed and applied by a person or by a computer.
In my first appraisal job I replaced a retiring appraiser. This appraiser chose not to use a calculator, requiring sometimes as many as three pages of hand calculations which were saved in the file folder along with the appraisal and field notes. An analysis performed with the use of a calculator is absolutely, positively no different than one performed totally by hand calculations (it just takes longer and may have more math errors). If you understand that a computer is just a fancy calculator that can be directed by the real property appraiser via a computer program then you understand that a computer does not perform the analysis "independent" of humans.
The reason this concept may be hard to grasp is that an appraiser typically does not do the programming. But I can assure you that some human did. Two artificial intelligence programs may come to very different conclusions considering the same data. This is because they were programmed to behave differently. One does not adjust the outputs of these computers by having a two way conversation with them; one does this by modifying the programming. The intelligence of the computer is in the programming. It does have a processor which is the workhorse of the computer. Basically, the machine alone (without programming) is only smart enough to know if a circuit is open or closed, and expresses this as either a "1" or "0". This is why computer languages are binary as opposed to the decimal system. The strength or worth of the computer is its speed which has reached a level that is beyond human comprehension.
A model is an explanation of an outcome. The usefulness of a model is its ability to predict some event.
A model predicting when an egg in an incubator will hatch might consider the temperature and rotation of the eggs. A model addressing the number of wins and losses for a college football team might consider the average age, weight, time to run the 40, years played at college level, etc. You are concerned with predicting the sales price of a hypothetical sale within a particular real property sub-market.
You have always been a market modeler and have been performing all of the steps and procedures of market modeling. Every direct comparison approach is based on a market model that he/she has developed in his/her head while performing one or more appraisals in a specific market. For example, when an appraiser makes a line item adjustment of $100.00 per square foot to a comparable when the size difference is equal to or greater than 200 square feet and $50.00 when less than 200 square feet, he/she is applying a part of a market model. The appraiser typically applies this part of the model along with other parts relating to lot size, car storage, quality, age, etc. on a line-by-line basis in the adjustment grid. The computer applies a market model in a similar fashion, but likes to know all of the model components up front and then apply them all at one time.
The appraiser then makes, and always has made, inferences that the comparables used represent the market place reasonably well. The appraiser does not typically discuss the "reliability" of the value opinion, but knows some appraisals are more reliable than others. Most appraisers are like the Scarecrow, Tin-man, and Cowardly Lion in the Wizard of OZ; they understand market modeling, but never knew they did. Appraisers just lack an understanding of computer programming to feel comfortable with automated tools.
Since market modeling has always been performed by appraisers and automated market modeling is only different because it is done with the aid of a sophisticated calculator (computer), there is really only one concept of market modeling. The current set of appraisal principles and rules that you understand and utilize on a day to day basis cover most aspects of market modeling.
There is no difference in output whether the same analysis is performed by hand or by a computer. So automated market modeling is a tool, not a two headed-Martian. This article discusses market modeling tools to assist the appraiser in their everyday appraisal work. The dreaded AVM, which in my opinion plagues society today, works within the same mathematical laws as the methods we will be discussing, but typically has a very low level of scope of work.
Admittedly, the computer, like the calculator, is much faster than those performed by hand-making certain analysis economically feasible. The feasibility of buying the software needed for running advanced market modeling analysis is based on the number of times the output is needed and the fee that can be realized for that output.
PHASES OF MARKET MODELING
There are two phases of market modeling; developing a model and applying the model. Testing the model's fit or accuracy as a predictor is a part of first phase (developing a model).
A market model is a description of how a particular market behaves. Models are expressed mathematically. Mathematics is a form of communicating the aspects of the market's behavior allowing for that behavior to be analyzed. Appraisers are taught to build market models by a technique called "paired sales". However, because the technique of paired sales require otherwise similar properties to be analyzed, they are relatively rare. Most appraisers tend to build their market models over time (doing many appraisals in that market over a period of years). This is why the Uniform Standards of Professional Appraisal Practice and Fannie Mae require the appraiser to consider his/her experience in the geographic location of the market in terms of competency.
Now, "black box" market models operate on a wide array of market model building techniques. Some attempt to build their model based on "expert" rules within which their eventual model must conform. Their rules minimize the number of paths that are considered by the computer in the calculations. The most mysterious is where the AVM is taught to develop a market model based on the data presented to it by a heuristic search for the "correct" prediction.
The model in these cases is not directly revealed by the computer, but one does exist. The model can be indirectly found by considering the difference in the output while changing one variable at a time. There are basically two types of analyses involved in an AVM; regression and artificial intelligence. Index systems are not addressed here as they are not true AVMs. There is really no "black box" or incomprehensible activity going on. Remember a computer operates on the relationship of open or closed. All of the millions of computations performed in seconds were directed by the computer programmer. The belief or appearance that the computer forms a market model independent of human interaction is where the "A" in AVM comes from.
But, the computer developed a market model under the direction of the computer programmer (a human). An analogy is whether homemade ice-cream made with a hand crank is any different when that same recipe is made while powered by an electronic motor. This is not to belittle the level of artificial intelligence that results from sophisticated computer programming. One has but to challenge the computer to a game of chess to be wowed!
Many developers and operators of AVM systems tend to use the concept of the black-box much like a gambler uses shells to hide a pea; as much for deception as concealment. This is not always because they may be hiding some inappropriate activity; it is also done to protect their intellectual property. They do not want to disclose how their AVM works.
Which leads to a final issue: All market models whether developed by appraisers or an AVM have a scope of work which can be disclosed.
ALL GOD'S CHILDREN GOT A SCOPE OF WORK
It is time to require AVMs to report the scope of work with which they produce a value and educate users to the effect of the scope of work on the reliability of the AVM's value prediction. If not enforceable by state regulatory commissions/boards then it should be by consumer protection laws. Even the sunscreen I use tells me what is in it.
For example, it is a fallacy that AVMs are totally impartial. If you understand that there is little difference between what you do by hand and what the AVM is programmed to do then you know it is just as easy for a programmer to provide instructions for the AVM to come out a little higher across the board as it is for the appraiser to manipulate his/her value opinions. They are only as impartial as their programmer or owner is.
There is a lot at stake if a lender with a national or regional presence stops using a particular AVM provider because they perceive its predictions to be too conservative. How can users of AVMs check for such inappropriate programming if the valuation process cannot be recreated based on its scope of work? Until AVMs are forced to disclose their scope of work, it will be difficult to know which of them is selling a real remedy and which might be selling snake oil.
Appraisers who decide to work with some type of AVM must decide upon a targeted accuracy of the product, or products as two or more lines could be offered. AVMs are not limited to the type that gives a value in three seconds and typically has a very low scope of work. The following chart shows the products currently available to lenders based on their cost in fees (Y-axis) and their accuracy (X-Axis). Notice the void of services between the two red lines.
Maybe lenders choose to use unreliable AVMs because they have no other low end choices available. Naming an upper end AVM product is difficult because of the stigma that all AVMs have a very limited scope of work. Remember, an AVM can be no better than the quality of the data used in it. An automated analysis, like the one downloadable in conjunction with this writing, can be used as an addendum to shore up an appraisal.
THE TIME HAS COME
The time has come for appraisers to utilize the power of their quality data and their computer to enhance their daily work product. The effectiveness of the analysis is governed by the quality of the data available for analysis. A car's body could be built to function in outer space, but until the fossil fuel burning engine is replaced with a rocket engine there is really no point. This reasoning also applies to AVMs; there is no point in having an engine that has the capability to analyze high quality data that does not exist. While some very sophisticated market modeling programs exist, most feed off public records and operate with a minimal scope of work.
You have data sources that are superior to public tax records. These include access to local MLS, your own sales databases, and databases shared with other appraisers. The lab has emphasized projects like GIS and GPS which will improve the quality of the data of appraisers. The peer-to-peer data sharing features we are exploring would allow two or more appraisers to share their data. Having enough good quality data on hand to run through an automated analysis will distance the appraiser from the typical AVM provider (which primarily relies on tax records).
Regression is a mathematical procedure that is relatively easy to understand and recognized as legitimate by many users of appraisal services. I have created a regression program that runs in MS Excel. If you have Excel loaded on your computer you can download the application and the user instructions. It is set to run until the end of this year. It is presented in a module that is an addition to an appraisal and is not intended to be a stand alone appraisal.
Play around with it, send us your feedback, and watch for the second part of this paper which will be entitled "Regression Analysis". I think you will be surprised to find out that much of what you have been taught about regression analysis is not true in regards to real property valuation. Our journey will be a search for the truths concerning regression and exploring other options for incorporating similar tools into our daily appraisal practice.
(Wait a minute! Where is all of the discussion on statistics? Statistics deals with analyzing and making predictions with "large" amounts of data. Appraisers will typically be considering from 15 to 50 comparables at a time, which does not qualify as large numbers in the over-all scheme of things. For example, the appraiser is better served by considering the average absolute variance of the model's predictions from the sales prices, than the standard deviation of the predictions. The appraiser's knowledge of markets, their operation, and boundaries is the key ingredient here, not the factor. The only statistical knowledge you need is an understanding of covariance, which we will address in Part II.)