Please Lie

You ought to tell more white lies.

This may seem like strange moral advice, but we live in strange times. Times of ubiquitous surveillance, where everyone's purchases, their web browsing habits, their social media use, the TV shows and movies they watch, and a hundred other data streams are fed into hidden, unaccountable computer models to determine whether you will get a job offer, a loan, an insurance policy, and how hard companies will be able to squeeze you when they sell to you^[0].

It is relatively obvious that such surveillance may change what's in your best interests, although part of the problem with this system is that it's so hard to tell what is and isn't in your interests anymore. But it may not be obvious—to someone unfamiliar with the computer models—how it affects your moral duties, and in particular how it imposes a duty to lie.

A brief discussion of these models is necessary. You have probably heard the terms artificial intelligence and machine learning. These are broad terms for a large class of technologies, which mostly fall under two umbrellas—statistical models and neural networks. These technologies differ in important ways, but they share a common operating principle: they find correlations in large data sets and use them to develop heuristics.

This is similar to much of human learning. When applied to large groups of people, we call the resulting heuristics stereotypes, and we are often uncomfortable with their moral implications. The primary difference between the computer's heuristics and our stereotypes is the depth and breadth of information available to it. Instead of having a stereotype that young black men are likely to commit crimes, for example, it may decide that young black men who have lived in certain zip codes and shop at Walmart are unusually likely to commit crimes, while those who have lived in other zip codes and shop at vintage record stores are less likely to. Often, machine learning models will have tens of thousands of such heuristics, and this precision makes them powerful. They really will zone in on which groups are likely to commit a crime, or pay back a loan, or get injured, or are willing to pay the most for a given product.

Because these models are based on correlations, your behavior impacts how the model views everyone. When the model learns something new about you—say you make a purchase, or an insurance claim, or you just go another year without getting arrested—the model will take everything it knows about you—your age, gender, race, zip code, credit history, political views, hobbies, favorite TV shows, who you're friends with on social media, etcetera—and update its predictions about everyone else who has any of these characteristics in common with you. Perhaps you buy overpriced headphones, and the model predicts it can overcharge your Facebook friends (and their friends) who watch the same TV shows as you for audio equipment. Perhaps you are healthy and shop at Trader Joe's, and the model slightly decreases health insurance premiums for other Trader Joe's shoppers while slightly increasing them for everyone else. Perhaps you become a regular at a cafe, and the model concludes that people at that cafe are likely to vote Democrat and are fruitful targets for Republican turnout-suppression advertisements.

These examples are clearly cherry-picked, but they fall into broad trends brought on by such models:

They make wealthy companies wealthier. Wealthy companies are able to gather more data and develop better models than smaller competitors, and can use these to more effectively advertise to customers, and to do what economists call price discrimination—charging different people different prices because some are willing to pay more than others. Amazon is particularly notorious for this, and while to my knowledge they have backed off on literally charging different prices to different consumers at the same time and in the same neighborhood, they adjust prices so frequently that the effect is the same. The consequence of this, according to classical economic theory, is that while some consumers will pay less than they would previously, as the company's price discrimination improves, all surplus value will eventually be captured by the company. Classical economics also says that price discrimination should be impossible without barriers to entry—which we have already seen that these models provide.

They reinforce the powerful. Besides making the wealthy wealthier, they allow political campaigns to parley advertising dollars into votes more effectively. Outside democracies they are far more powerful, providing tools to detect dissidents and to target disinformation campaigns^[1].

They entrench poverty. Companies charge poor people more for things, because they can't afford to delay their purchases, or because they're less savvy consumers. Recall my opening examples of how these models are used for employment, loans, and insurance policies, and combine that with the fact that people in historically poor groups are statistically riskier to employ, loan to or insure, so will be punished by these models. More directly concerning is that these models are sometimes used by the US criminal justice system in pre-trial release hearings and even sentencing, and they heavily penalize black people^[2].

More abstractly, I expect they will have a chilling effect on society. Eventually people will realize that they're being penalized for many of their day-to-day actions, but they still won't know what they're being penalized for. Uncertainty breeds hesitance. I expect people will become reluctant to talk online, reluctant to shop, reluctant to subscribe to newspapers and magazines. Or they'll become duplicitous, trying to shape their online lives and public habits in a way they think favors them. Being surrounded by either, or both, will be pretty intolerable.

Abstaining from training these models is very, very hard. You can't shop online or use a credit card, you can't use social media, can't sign up for anything requiring a Facebook login^[3], can't use store discount cards, can't subscribe to anything, can't use map-based services on your phone, etcetera. You'll be at a significant disadvantage in life and most people will regard you as a social pariah. And even if you do abstain, you'll be alone, and it will make very little difference.

Fortunately there is a much easier and more effective way: lie. When a store asks for your phone number or your zip code, lie. Lie on your Facebook profile, or use a second Facebook profile when browsing and signing up for things^[4]. When using a ride-sharing service, put in the address next door to your destination. If you have the technical skill to do so, spoof your location on your phone.

This is far better than abstention. As models become more precise in their predictions, they are relying on smaller and smaller signals in the underlying data sets. A random lie will tend to be a strong signal in a different direction than any real trend, exerting disproportionate influence on the model. Instead of merely negating your own influence on the model, you're counteracting other people's too.

0. ^

   Incidentally, almost everyone I've talked to about this either does not believe this is happening (often likening it to a conspiracy theory), but that if it were it would be horrifying, or is well aware of and untroubled by it. The latter is much more common in Silicon Valley; after all they can hardly deny the explicit purpose of so many of the tools and businesses they have built.

1. ^

   Disinformation campaigns are particularly useful for those in a position of power, because confusion and disorganization undermine any attempt to change the status quo. For much more on this and other authoritarian uses of technology, I recommend the work of Zeynep Tufekci.

2. ^

   The linked article claims that a particular model is "biased against blacks", which is true in a loose, colloquial sense of the word, but is not true in the sense that statisticians use the word. What the article really shows is that there is disparate impact—black people suffer much more from use of this model than white people do, even those who will not jump bail or re-offend. The paper on which the article is based shows that the model has similar precision and recall for black and white people, meaning it is not biased in the statistical sense. But it also shows that the false positive rate is much higher for black people. This is a natural statistical consequence of the fact that the model is imprecise and that black people jump bail or re-offend at much higher rates than white people. But it's still noteworthy from a public policy point of view. It would be silly to dismiss the article because it doesn't show statistical bias—after all, a model that only used race as input and rated black people twice as risky as white people would not be biased in the statistical sense but most people would agree that it is racist.

3. ^

   For example, most dating apps require a Facebook login. This may seem pretty trivial unless you are a single 20-something in an area with an unfavorable gender ratio, in which case it will seem anything but.

4. ^

   This is against Facebook's terms of service, but they can go fuck themselves.