Credit card company data mining makes us all instances of a type

The New York Times has recently published one of their in-depth, riveting descriptions of how credit card companies use everything they can learn about us. Any detail can be meaningful: what time of day you buy things, or the quality of the objects you choose.

The way credit collectors use psychology reminds me of CIA interogators (without the physical aspects of pressure). In fact, they're probably more effective than CIA interogators because they stick to the basic insight that kindness elicits more cooperation than threats.

So who gave them permission to use our purchase information against us? What law could possibly address this kind of power play?

There's another disturbing aspect to the data mining: it treats us all as examples of a pattern rather than as individuals. Almost eleven years I wrote an article criticizing this trend. The New York Times article shows how much we've lost from what we consider essential to our identity--our individuality.

Update

This article drew six comments in a few hours--thoughtful and valid comments, which have made me set down attitudes into words. Now we can look put the attitudes under a light and see what makes sense, or doesn't, to readers.

The article contained two levels of criticism: a criticism of data mining to build up composite pictures of individuals, and a criticism of the use of data accumulated from routine transactions to manipulate those individuals.

Building up a composite picture

Of course, a company that reaches out and does any marketing has to target people. Someone who bought the O'Reilly book Even Faster Web Sites (sorry about the plug) might appreciate a notification about our upcoming Velocity conference, which was founded by the book's author and covers the same topics. Someone who bought a book on a totally different subject wouldn't want or respond to the notification. O'Reilly does this kind of targeting, like most companies, and until everybody participates in truly frictionless information exchanges, companies will have to continue doing it.

Aggregated information is useful too. Organizations that mine public data for evidence of health epidemics can identify likely sites and investigate them further. The data mining is understood to provide an approximation of the truth.

Where I see a problem is when the increasing quantity of constant information refinement shades over into a qualitative change. There's a difference between a campaign targeted to 500 likely customers and a campaign targeted to one.

At some point the composite portrait starts to look so much like a person that corporate decision makers can begin to believe it is the person. The portrait becomes like a replicant, or like the statues that came to life in myths from Pygmalion to Pinocchio.

Joseph Weizenbaum, creator of the classic Eliza program, was shocked to see that people treated his "doctor" program like a human interviewer. There were plenty of computer programs that prompted the user with questions and gave varied responses based on the answers, but none had imitated a person so realistically.

Nowadays, nobody would be drawn in by Eliza. And perhaps companies and customers alike will get used to composite portraits. Perhaps the companies will send their composite to each of us and we can update it to make it more accurate. That will be a very different world, though.

Now we can turn to the next level, manipulation.

Manipulation

I've read numerous accounts in biographies and articles about interrogations, and talked to a couple people who have undergone interrogations. I haven't been on either side of an interrogation, but I've been deposed for a court case. All these situations remind me vividly of the exchanges reported in the New York Times article.

In these exchanges, a well-armed caller is laying, like a silkscreen, a composite over the real person and trying to manipulate the result. It's not exactly a case of asymmetric knowledge (because at least in theory, a customer could also learn a lot about a company and use that knowledge to manipulate it). It's more insidious: an employee carrying out a precise initiative on behalf of a company--a machine in the service of a goal--approaching the targeted customer in an informal manner that brings out a natural, human, empathetic reaction in customer.

Interrogation always takes place in the context of an open or implied threat--there would be no reason for making the contact otherwise--but as I mentioned in the article, the interrogation goes best when the threat is raised only rarely and strategically. A feigned sympathy and heart-to-heart engagement is the path to the most desired outcome.

In a sense, now, the employee has become the replicant. He is using a careful counterfeit of human responses to induce the behavior he or she is paid to induce. This is ethical when dealing with a criminal, although even then US law limits (based on the Fourth Amendment) the gathering of relevant information by the interrogator beforehand. I question how ethical it is in a business situation, especially when exploiting information given by the customer for entirely different purposes.