System Design and Self-Selection Bias

Not all populations are created equal. Blindly designing a system without thinking about the pressures involved in the data you collect (or the people who will participate) can easily result in harm to society.

As an example, in a public online polls respondents are more likely to have strong opinions. Potential respondents who don’t have firm positions are less likely to see value in providing answers, and will be less likely to put effort into it. Drawing conclusions from an online poll anyone can respond to will incorrectly lead to the finding that people are very polarized on issues. Scientific polls have safeguards to prevent this sort of bias.

Self-selection bias in system design isn’t always obvious, so I want to discuss a more nuanced case.

There’s a trend in the US for developing new financial instruments. Markets in the US are not well regulated, so it is very easy for entrepreneurs to develop new types of financial contracts as means of making money. One such case is instruments mimicking reverse mortgages. For example, Point lets homeowners sell a percentage of their home in return for cash.

Homes and Liquidity

In economics, liquid assets refers to things that people can exchange without the item losing value. In practice, this means anyone can easily determine the item’s value and can exchange it easily. Cash is a liquid asset because its value is literally printed on the bill or coin and nearly anyone will accept it in exchange for goods immediately. The opposite, illiquid assets, refers to items for which this is difficult. For example, exchanging a home can take weeks and it takes a professional hours to determine a fair value.

For Point, its selling point is the option for homeowners to exchange a portion of their illiquid assets – ownership of their home – in exchange for liquid assets – cash. Fortunately Point is honest that they may offer less for the portion of the home that they buy. They may only offer $90,000 for 20% of a home which has been appraised at $500,000. In these agreements the homeowner’s net worth immediately decreases significantly – possibly by tens of thousands of dollars. (Just think of what would happen if the owner sold to Point, then immediately bought back that 20% in the case above.)

Self-Selection Pressures

In designing any system, we have to consider the pressures which will influence the statistical properties of the decisions it makes.

Aside: Homo Economicus and Homo Psychologicus

Homo economicus is a simplification of humans for economic modeling. It presents a human-like agent that acts rationally in its own self interest according to available information.

Homo psychologicus is a perturbation of homo economicus that takes into accounts the psychological factors all humans fall prey to. These models are more complex since they require more variables, but should be taken into account in situations where humans are less likely to act in their rational self interest.

Homeowner Selection

In this case, we must first ask: What sort of homeowner is likely to accept an offer from Point?

This is where self-selection bias comes into play. In the long term, it’s obvious that statistically the better option is to hold onto full ownership. (If it were not, Point would not have a business model.) So if we assume an owner has a base level of financial savviness, they will hold on to full ownership if they have the financial means to do so. Thus, we can expect many participating homeowners to feel some pressure to get cash quickly. We can then assume they are less likely to be financially stable – more likely to have a credit payment or a bill they need to pay off quickly. They are not reducible to homo economicus but must be modeled as homo psychologicus. They will be more prone to mistakes in reasoning and more impacted by biases.

Offer Selection

Next: What sorts of offers is Point likely to make?

This is a second source of selection pressure – it impacts the statistics of the offers Point is likely to make. Success for Point’s valuation algorithm is the money Point makes, so if functioning optimally the algorithm will offer the lowest amount the owner will agree to. As a business Point is likely to be functioning close to homo economicus, so we can assume they will make this rational decision.

Aside: Weapon of Math Destruction

The logic which goes into this offer cannot be appealed, and there is no way for the owner to know how the value was calculated – it is hidden behind an unquestionable proprietary algorithm. This algorithm is unlikely to be “fair” to the homeowner – its purpose is to make money for Point.  Simply by measuring success this way and not being auditable, the algorithm will be predatory (even if no humans involved had this intention). This unaccountability makes it a Weapon of Math Destruction.

Agreement Selection

Combining these two selection pressures lets us answer: What sort of agreements are likely to be made and accepted?

Point is most likely to enter into agreements with owners who have undervalued their own home. The greater the disparity between an owner’s valuation and what Point really thinks the value is, the more incentive Point has to make a deal. If the value Point offers is sufficiently lower than what a homeowner thinks it is, a homeowner will always reject the offer. If the owner does not have a reasonable understanding of their home’s value, they are more likely to think the offer is a good one. Additionally, if the owner’s rationality is compromised they are more likely to enter into a deal that is not in their best interests. For deals between two homo economicus we would expect deals only to be made when both parties rationally perceive they will profit. Since it is likely many owners will not be acting purely rationally due to other factors in their life, we can’t make this assumption. There will be owners who enter deals which hurt them.

The Sunk Cost Effect

By the time owners get to this stage, they have spent several hundred dollars in getting their home appraised and hours of their time. Point at most has wasted some of its employees’ time. It’s worth it to Point as they absorb some of this cost by offering other homeowners lower prices. But for the owner, they now have an appraisal that they wouldn’t otherwise have. For many the time, money, and effort they already expended will make them more likely to accept the offer even if it is below what they are comfortable with – this is the sunk cost effect. Since many potential participants already had some pressure to get money quickly, this has made their situation more dire and they are less likely to act rationally.

Conclusion

Should you use Point? That depends on your measure of success. If success is making a good return on investment, that depends on whether you can use the increased liquidity to make more than what you (effectively) paid Point and it is better than similar financial arrangements (e.g. reverse mortgage). If you are against systems that have the potential to harm society, you have to decide whether you trust Point has accounted for the damage it could do.

Point has the capacity to harm society if it isn’t careful. We can’t measure the impact on society since its valuation algorithms are hidden and Point is unlikely to share its data with researchers. Even if it cost them nothing to find out, it is likely they would choose not to know¹ whether their system caused harm. They would also be unlikely to share if they did know.

By default, the homeowners who self-select to enter agreements with Point will not be in a good financial situation, and so will generally lose net worth in the agreement. We can expect Point’s algorithm will move net worth from people with lower wealth to people with higher wealth – exacerbating the current wealth inequality problems our society faces. While it is possible for individuals to recoup the loss they incurred by purchasing the fast liquidity, this is not the default. If a homeowner has need of liquidity urgently, they aren’t likely to be using it in a way that will gain value (like investments) – they are more likely to need it for a large high-interest debt or unexpected bill.

It is important to consider these factors in any system being designed. It is possible Point has mitigated the issues I’ve described. This would require them to have a drive to actively ensure they aren’t being predatory of people in tough financial spots. This isn’t something done passively, but a professional responsibility they would have to choose to take. In the absence of evidence or any insight into their transactions and offer methodology, we simply can’t know.