A place to cache linked articles (think custom and personal wayback machine)
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

index.md 20KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292
  1. title: Forget privacy: you're terrible at targeting anyway
  2. url: https://apenwarr.ca/log/20190201
  3. hash_url: abe583967cb53a01298d4b073cb406ad
  4. <p><b>Forget privacy: you're terrible at targeting anyway</b></p>
  5. <p>I don't mind letting your programs see my private data as long as I get
  6. something useful in exchange. But that's not what happens.</p>
  7. <p>A former co-worker told me once: "Everyone loves collecting data,
  8. but nobody loves analyzing it later." This claim is almost shocking, but
  9. people who have been involved in data collection and analysis have all seen
  10. it. It starts with a brilliant idea: we'll collect information about
  11. every click someone makes on every page in our app! And we'll track how
  12. long they hesitate over a particular choice! And how often they use the
  13. back button! How many seconds they watch our intro video before they abort!
  14. How many times they reshare our social media post!</p>
  15. <p>And then they do track all that. Tracking it all is easy. Add some log
  16. events, dump them into a database, off we go.</p>
  17. <p>But then what? Well, after that, we have to analyze it. And as someone who
  18. has <a href="/log/20160328">analyzed</a> a <a href="/log/20171213">lot</a> of <a href="/log/20180918">data</a>
  19. about various things, let me tell you: being a data analyst is difficult
  20. and mostly unrewarding (except financially).</p>
  21. <p>See, the problem is there's almost no way to know if you're right. (It's
  22. also not clear what the definition of "right" is, which I'll get to in a bit.)
  23. There are almost never any easy conclusions, just hard ones, and the hard
  24. ones are error prone. What analysts don't talk about is how many incorrect
  25. charts (and therefore conclusions) get made on the way to making correct
  26. ones. Or ones we think are correct. A good chart is so incredibly
  27. persuasive that it almost doesn't even matter if it's right, as long as what
  28. you want is to persuade someone... which is probably why newpapers,
  29. magazines, and lobbyists publish so many misleading charts.</p>
  30. <p>But let's leave errors aside for the moment. Let's assume, very
  31. unrealistically, that we as a profession are good at analyzing things. What
  32. then?</p>
  33. <p>Well, then, let's get rich on targeted ads and personalized recommendation
  34. algorithms. It's what everyone else does!</p>
  35. <p>Or do they?</p>
  36. <p>The state of personalized recommendations is surprisingly terrible. At this
  37. point, the top recommendation is always a clickbait rage-creating
  38. article about movie stars or whatever Trump did or didn't do in the last 6
  39. hours. Or if not an article, then a video or documentary. That's not what I
  40. want to read or to watch, but I sometimes get sucked in anyway, and then
  41. it's recommendation apocalypse time, because the algorithm now thinks I
  42. <em>like</em> reading about Trump, and now <em>everything</em> is Trump. Never give
  43. positive feedback to an AI.</p>
  44. <p>This is, by the way, the dirty secret of the machine learning movement:
  45. almost everything produced by ML could have been produced, more cheaply,
  46. using a very dumb heuristic you coded up by hand, because mostly the ML is
  47. trained by feeding it examples of what humans did while following a very
  48. dumb heuristic. There's no magic here. If you use ML to teach a computer
  49. how to sort through resumes, it will recommend you interview people with
  50. male, white-sounding names, because it turns out that's <a href="https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazonscraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G">what your HR
  51. department already
  52. does</a>.
  53. If you ask it what video a person like you wants to see next, it will
  54. recommend some political propaganda crap, because 50% of the time 90% of the
  55. people <em>do</em> watch that next, because they can't help themselves, and that's
  56. a pretty good success rate.</p>
  57. <p>(Side note: there really are some excellent uses of ML out there, for things
  58. traditional algorithms are bad at, like image processing or winning at
  59. strategy games. That's wonderful, but chances are good that <em>your</em> pet ML
  60. application is an expensive replacement for a dumb heuristic.)</p>
  61. <p>Someone who works on web search once told me that they already have an
  62. algorithm that guarantees the maximum click-through rate for any web search:
  63. just return a page full of porn links. (Someone else said you can reverse
  64. this to make a porn detector: any link which has a high click-through
  65. rate, regardless of which query it's answering, is probably porn.)</p>
  66. <p>Now, the thing is, legitimate-seeming businesses can't just give you porn
  67. links all the time, because that's Not Safe For Work, so the job of most
  68. modern recommendation algorithms is to return the closest thing to porn that
  69. is still Safe For Work. In other words, celebrities (ideally attractive
  70. ones, or at least controversial ones), or politics, or both. They walk that
  71. line as closely as they can, because that's the local maximum for their
  72. profitability. Sometimes they accidentally cross that line, and then have
  73. to apologize or pay a token fine, and then go back to what they were doing.</p>
  74. <p>This makes me sad, but okay, it's just math. And maybe human nature. And
  75. maybe capitalism. Whatever. I might not like it, but I understand it.</p>
  76. <p>My complaint is that none of the above had <em>anything</em> to do with hoarding
  77. my personal information.</p>
  78. <p><b>The hottest recommendations have nothing to do with me</b></p>
  79. <p>Let's be clear: the best targeted ads I will ever see are the ones I get from
  80. a search engine when it serves an ad for exactly the thing I was searching
  81. for. Everybody wins: I find what I wanted, the vendor helps me buy their
  82. thing, and the search engine gets paid for connecting us. I don't know
  83. anybody who complains about this sort of ad. It's a good ad.</p>
  84. <p>And it, too, had nothing to do with my personal information!</p>
  85. <p>Google was serving targeted search ads decades ago, before it ever occurred
  86. to them to ask me to log in. Even today you can still use every search
  87. engine web site without logging in. They all still serve ads targeted to
  88. your search keyword. It's an excellent business.</p>
  89. <p>There's another kind of ad that works well on me. I play video games
  90. sometimes, and I use Steam, and sometimes I browse through games on Steam
  91. and star the ones I'm considering buying. Later, when those games go on
  92. sale, Steam emails me to tell me they are on sale, and sometimes then I buy
  93. them. Again, everybody wins: I got a game I wanted (at a discount!), the
  94. game maker gets paid, and Steam gets paid for connecting us. And I can
  95. disable the emails if I want, but I don't want, because they are good ads.</p>
  96. <p>But nobody had to profile me to make that happen! Steam has my account, and
  97. I <em>told</em> it what games I wanted and then it sold me <em>those</em> games. That's
  98. not profiling, that's just remembering a list that I explicitly
  99. handed to you.</p>
  100. <p>Amazon shows a box that suggests I might want to re-buy certain kinds of
  101. consumable products that I've bought in the past. This is useful too, and
  102. requires no profiling other than remembering the transactions we've had with
  103. each other in the past, which they kinda have to do anyway. And again,
  104. everybody wins.</p>
  105. <p>Now, Amazon also recommends products <em>like</em> the ones I've bought before, or
  106. looked at before. That's, say, 20% useful. If I just bought a computer
  107. monitor, and you know I did because I bought it from you, then you might as
  108. well stop selling them to me. But for a few days after I buy any
  109. electronics they also keep offering to sell me USB cables, and
  110. they're probably right. So okay, 20% useful targeting is better than 0%
  111. useful. I give Amazon some credit for building a useful profile of me,
  112. although it's specifically a profile of stuff I did on <em>their</em> site and
  113. which they keep to themselves. That doesn't seem too invasive. Nobody is
  114. surprised that Amazon remembers what I bought or browsed on their
  115. site.</p>
  116. <p>Worse is when (non-Amazon) vendors get the idea that I might want something.
  117. (They get this idea because I visited their web site and looked at it.)
  118. So their advertising partner chases me around the web trying to sell me the
  119. same thing. They do that, even if I <em>already</em> bought it. Ironically, this
  120. is because of a half-hearted attempt to <em>protect</em> my privacy. The vendor
  121. doesn't give information about me or my transactions to their advertising
  122. partner (because there's an excellent chance it would land them in legal
  123. trouble eventually), so the advertising partner doesn't know that I bought
  124. it. All they know (because of the advertising partner's tracker gadget on
  125. the vendor's web site) is that I looked at it, so they keep advertising it
  126. to me just in case.</p>
  127. <p>But okay, now we're starting to get somewhere interesting. The advertiser
  128. has a tracker that it places on multiple sites and tracks me around. So it
  129. doesn't know what I bought, but it does know what I looked at, probably over
  130. a long period of time, across many sites.</p>
  131. <p>Using this information, its painstakingly trained AI makes conclusions about
  132. which other things I might want to look at, based on...</p>
  133. <p>...well, based on what? People similar to me? Things my Facebook friends
  134. like to look at? Some complicated matrix-driven formula humans can't
  135. possibly comprehend, but which is 10% better?</p>
  136. <p>Probably not. Probably what it does is infer my gender, age, income level,
  137. and marital status. After that, it sells me cars and gadgets if I'm a guy,
  138. and fashion if I'm a woman. Not because all guys like cars and gadgets, but
  139. because some very uncreative human got into the loop and said "please sell
  140. my car mostly to men" and "please sell my fashion items mostly to women."
  141. Maybe the AI infers the wrong demographic information (I know Google has
  142. mine wrong) but it doesn't really matter, because it's usually mostly right,
  143. which is better than 0% right, and advertisers get some mostly
  144. demographically targeted ads, which is better than 0% targeted ads.</p>
  145. <p>You <em>know</em> this is how it works, right? It has to be. You can infer it
  146. from how bad the ads are. Anyone can, in a few seconds, think of some stuff
  147. they really want to buy which The Algorithm has failed to offer them, all
  148. while Outbrain makes zillions of dollars sending links about car insurance
  149. to non-car-owning Manhattanites. It might as well be a 1990s late-night TV
  150. infomercial, where all they knew for sure about my demographic profile is
  151. that I was still awake.</p>
  152. <p>You tracked me everywhere I go, logging it forever, begging for someone to
  153. steal your database, desperately fearing that some new EU privacy regulation
  154. might destroy your business... for <em>this</em>?</p>
  155. <p><b>Statistical Astrology</b></p>
  156. <p>Of course, it's not really as simple as that. There is not just one
  157. advertising company tracking me across every web site I visit. There are...
  158. many advertising companies tracking me across every web site I visit. Some
  159. of them don't even do advertising, they just do tracking, and they sell that
  160. tracking data to advertisers who supposedly use it to do better targeting.</p>
  161. <p>This whole ecosystem is amazing. Let's look at online news web sites. Why
  162. do they load so slowly nowadays? Trackers. No, not ads - trackers. They
  163. only have a few ads, which mostly don't take that long to load. But they
  164. have a <em>lot</em> of trackers, because each tracker will pay them a tiny bit of
  165. money to be allowed to track each page view. If you're a giant publisher
  166. teetering on the edge of bankruptcy and you have 25 trackers on your web site
  167. already, but tracker company #26 calls you and says they'll pay you $50k a
  168. year if you add their tracker too, are you going to say no? Your page runs
  169. like sludge already, so making it 1/25th more sludgy won't change anything,
  170. but that $50k might.</p>
  171. <p>("Ad blockers" remove annoying ads, but they also speed up the web, mostly
  172. because they remove trackers. Embarrassingly, the trackers themselves don't
  173. even need to cause a slowdown, but they always do, because their developers
  174. are invariably idiots who each need to load thousands of lines of javascript
  175. to do what could be done in two. But that's another story.)</p>
  176. <p>Then the ad sellers, and ad networks, buy the tracking data from all the
  177. trackers. The more tracking data they have, the better they can target ads,
  178. right? I guess.</p>
  179. <p>The brilliant bit here is that each of the trackers has a bit of data about
  180. you, but not all of it, because not every tracker is on every web site. But
  181. on the other hand, cross-referencing individuals between trackers is kinda
  182. hard, because none of them wants to give away their secret sauce. So each
  183. ad seller tries their best to cross-reference the data from all the tracker
  184. data they buy, but it mostly doesn't work. Let's say there are 25 trackers
  185. each tracking a million users, probably with a ton of overlap. In a sane
  186. world we'd guess that there are, at most, a few million distinct users. But
  187. in an insane world where you can't <em>prove</em> if there's an overlap, it could be
  188. as many as 25 million distinct users! The more tracker data your ad network
  189. buys, the more information you have! Probably! And that means better
  190. targeting! Maybe! And so you should buy ads from our network instead of
  191. the other network with less data! I guess!</p>
  192. <p>None of this works. They are still trying to sell me car insurance for my
  193. subway ride.</p>
  194. <p><b>It's not just ads</b></p>
  195. <p>That's a lot about profiling for ad targeting, which obviously doesn't work,
  196. if anyone would just stop and look at it. But there are way too many people
  197. incentivized to believe otherwise. Meanwhile, if you care about your
  198. privacy, all that matters is they're still collecting your personal
  199. information whether it works or not.</p>
  200. <p>What about content recommendation algorithms though? Do those work?</p>
  201. <p>Obviously not. I mean, have you tried them. Seriously.</p>
  202. <p>That's not quite fair. There are a few things that work. <a href="https://www.theserverside.com/feature/How-Pandora-built-a-better-recommendation-engine">Pandora's
  203. music
  204. recommendations</a>
  205. are surprisingly good, but they are doing it in a very non-obvious way. The
  206. obvious way is to take the playlist of all the songs your users listen to,
  207. blast it all into an ML training dataset, and then use that to produce a new
  208. playlist for new users based on... uh... their... profile? Well, they
  209. don't have a profile yet because they just joined. Perhaps based on the
  210. first few songs they select manually? Maybe, but they probably started
  211. with either a really popular song, which tells you nothing, or a really
  212. obscure song to test the thoroughness of your library, which tells you less
  213. than nothing.</p>
  214. <p>(I'm pretty sure this is how Mixcloud works. After each mix, it tries to
  215. find the "most similar" mix to continue with. Usually this is someone
  216. else's upload of the exact same mix. Then the "most similar" mix to that
  217. one is the first one, so it does that. Great job, machine learning, keep it
  218. up.)</p>
  219. <p>That leads us to the "random song followed by thumbs up/down" system that
  220. everyone uses. But everyone sucks, except Pandora. Why? Apparently
  221. because Pandora spent a lot of time hand-coding a bunch of music
  222. characteristics and writing a "real algorithm" (as opposed to ML) that tries
  223. to generate playlists based on the right combinations of those
  224. characteristics.</p>
  225. <p>In that sense, Pandora isn't pure ML. It often converges on a playlist
  226. you'll like within one or two thumbs up/down operations, because you're
  227. navigating through a multidimensional interconnected network of songs that
  228. people encoded the hard way, not a massive matrix of mediocre playlists
  229. scraped from average people who put no effort into generating those
  230. playlists in the first place. Pandora is bad at a lot of things (especially
  231. "availability in Canada") but their music recommendations are top notch.</p>
  232. <p>Just one catch. If Pandora can figure out a good playlist based
  233. on a starter song and one or two thumbs up/down clicks, then... I guess it's
  234. not profiling you. They didn't need your personal information either.</p>
  235. <p><b>Netflix</b></p>
  236. <p>While we're here, I just want to rant about Netflix, which is an odd case
  237. of starting off with a really good recommendation algorithm
  238. and then making it worse on purpose.</p>
  239. <p>Once upon a time, there was the <a href="https://journals.sagepub.com/doi/full/10.1177/1461444814538646">Netflix
  240. prize</a>,
  241. which granted $1 million to the best team that could predict people's movie
  242. ratings, based on their past ratings, with better accuracy than Netflix
  243. could themselves. (This not-so-shockingly resulted in a <a href="https://www.cs.cornell.edu/~shmat/netflix-faq.html">privacy
  244. fiasco</a> when it turned
  245. out you could de-anonymize the data set that they publicly released, oops.
  246. Well, that's what you get when you long-term store people's personal
  247. information in a database.)</p>
  248. <p>Netflix believed their business depended on a good
  249. recommendation algorithm. It was already pretty good: I remember using
  250. Netflix around 10 years ago and getting several recommendations for things I
  251. would never have discovered, but which I turned out to like.
  252. That hasn't happened to me on Netflix in a long, long time.</p>
  253. <p>As the story goes, once upon a time Netflix was a DVD-by-mail service.
  254. DVD-by-mail is really slow, so it was absolutely essential that at least one
  255. of this week's DVDs was good enough to entertain you for
  256. your Friday night movie. Too many Fridays with only bad movies, and
  257. you'd surely unsubscribe. A good recommendation system was key. (I guess
  258. there was also some interesting math around trying to make sure to rent out
  259. as much of the inventory as possible each week, since having a zillion
  260. copies of the most recent blockbuster, which would be popular this month and
  261. then die out next month, was not really viable.)</p>
  262. <p>Eventually though, Netflix moved online, and the cost of a bad
  263. recommendation was much less: just stop watching and switch to a new movie.
  264. Moreover, it was perfectly fine if everyone watched the same blockbuster.
  265. In fact, it was better, because they could cache it at your ISP and caches
  266. always work better if people are boring and average.</p>
  267. <p>Worse, as the story goes, Netflix noticed a pattern: the more hours people
  268. watch, the less likely they are to cancel. (This makes sense: the more
  269. hours you spend on Netflix, the more you feel like you "need" it.) And with
  270. new people trying the service at a fixed or proportional rate, higher retention
  271. translates directly to faster growth.</p>
  272. <p>When I heard this was also when I learned the word
  273. "<a href="https://en.wikipedia.org/wiki/Satisficing">satisficing</a>," which
  274. essentially means searching through sludge not for the best option, but for
  275. a good enough option. Nowadays Netflix isn't about finding the best movie,
  276. it's about satisficing. If it has the choice between an award-winning movie
  277. that you 80% might like or 20% might hate, and a mainstream movie that's 0%
  278. special but you 99% won't hate, it will recommend the second one every time.
  279. Outliers are bad for business.</p>
  280. <p>The thing is, you don't need a risky, privacy-invading profile to recommend
  281. a mainstream movie. Mainstream movies are specially designed to be
  282. inoffensive to just about everyone. My Netflix
  283. recommendations screen is no longer "Recommended for you," it's "New
  284. Releases," and then "Trending Now," and "Watch it again."</p>
  285. <p>As promised, Netflix paid out their $1 million prize to buy the winning
  286. recommendation algorithm, which was even better than their old one. But
  287. <a href="https://medium.com/netflix-techblog/netflix-recommendations-beyond-the-5-stars-part-1-55838468f429">they didn't use it</a>, they threw it away.</p>
  288. <p>Some very expensive A/B testers determined that this is what makes me watch
  289. the most hours of mindless TV. Their revenues keep going up. And they
  290. don't even need to invade my privacy to do it.</p>
  291. <p>Who am I to say they're wrong?</p>