Skip to main content

Total Government Awareness, Software and Ethics

Posted by n_alex on August 20, 2003 at 11:02 AM PDT

Several years ago I was working on a system for modelling and simulating social pressure. The idea was that actions have repercussions and that I could build dynamic ecosystems staffed with characters of differing social power. The pressure system would stimulate social interaction between criminals, the wealthy corporate elite, impoverished citizens and various authorities in a cyberpunk RPG setting.

I was discussing my ideas with a professor and friend of mine over lunch one day, when he posed a difficut question to me. "What do you think would happen if a totalitarian government got ahold of and modified your software, Norm?" We discussed it for some time, and I abandoned my ideas for the project because of its potential social repercussions.

My caution at the time was inspired by fear. After a few years have passed, I think the fear was justified.

Earlier this summer the Computing Culture Group at MIT's Media Lab announced a project titled "Open Government Information Awareness" project. It is an effort to turn the Defense Advanced Research Project Agency (DARPA) inspired "Total Information Awareness" project on its heels and point it back at the US government. In this information-inspired age the idea of a "Civilian Information Agency" which conducts espionage on its own government is a chapter right out a Neal Stephenson novel.

The Computing Culture Group is doing with technology something which, in my mind, definitely needs to be done. They're using it in an open manner to address social and ethical concerns in their community. This project has nothing to do with entertainment or the enterprise. It is "Not Economically Viable". It generates no wealth, per se. But well-being extends so far beyond wealth.

Especially when it comes to the well-being of a society, or a group of societies, for that matter. Some argue that a person's economic freedom will always lead them to greater liberty. That's a fine argument, supported by many scholars and successful people throughout the centuries. But economic freedom is a symptom of, not the reason for this liberty.

There's an underlying pattern here. The concentration of social power has an inverse relationship to liberty. What does that mean? It means that social power--the ability to effect social change--has a natural tendency to concentrate and grow at the expense of freedom. That assertion deserves its own expanded essay, but this is just a blog entry, so I'll get to the point.

Whether it concentrates as totalitarian authority in the hands of a government, as capital (social power) in the hands of private industry, as money in the control of a private monopolistic financier class or as societal dominance by institutions of religion, the effects are always the same. Poverty, disease and warfare.

Capital, money and authority (whether the authority be mentally or physically coercive) are all forms of social power--they enable people to effect social change. Social power works to the detriment of liberty when it is concentrated into the hands of a few people.

So, what does this have to do with Open Source software or the Total Government Awareness project?

Because knowledge is also social power. The dissemination of knowledge hinders the concentration of social power. It enables a larger number of people to effect change in their society. It helps them recognize patterns of deception and corruption in their governments, their businesses, their institutions of faith. Because knowledge is social power, it too tends to concentrate and aggregate.

Google News is a perfect example of this. So is the Total Government Awareness project. They have created "Cathedral" style software pinnacles which are vulnerable to corruption. Google has already assisted the Chinese government in filtering its search results to prevent Chinese citizens from reaching certain information resources on the internet.

The question or study of ethics is not usually well recieved in any field. Throughout the twentieth century people have reviled and mistrusted any use of the term ethics. It has taken on a coercive role and the suspicion of many free thinkers.

Geoffrey Galt Harpham in his essay "Ethics", writes a particulary chilling explanation of the return of ethics to the world of literary criticism; one which prompts one to reconsider the role of ethics in software:

"virtually all the leading voices of the Theoretical Era (an era conspicuous for its deification of 'leading voices') organized their critiques of humanism as exposés of ethics, revalations of the transgressive, rebellious, or subversive energies that ethics had efffectively masked and suppressed. And virtually all joined [Jaques] Derrida in seeing ethics as a combination of mastery and delusion. For Jameson, the focus of ethics on the individual confronting a moment of moral choice obscured the deep currents of collective, historical life; for Jaques Lacan, the Kantian ethic of rational self-legislation masked the kernel of desire that ultimately aligns ethics with masochistic enjoyment, and Kant with de Sade; for Paul de Man, ethics had 'nothing to do with the will (thwarted or free) of a subject,' and was best considered a 'language aporia,' a 'discursive mode among others' . . . for Michel Foucault, ethics was a typical humanistic soap bubble whose reality was the relentlessly productive force of various discursive and disciplinary regimes. For all of them, the truth of ethics was announced by Nietzsche: 'a mere fabrication for purposes of gulling: at best, an artistic fiction; at worse, an outrageous imposture' . . .

"Then something happened.

On or about December 1, 1987, the nature of literary theory changed. When the New York Times reported the discovery of a large number of articles written by the youthful Paul de Man in a Belgian collaborationist newspaper in 1941-42, virtually everything about literary theory and criticism as practiced in the United States underwent a transformation as violent and radical as that which had been wraught a generation earlier by the advent of 'theory' itself. . . . Deconstructionists, who had been attempting to outlast and outpublish the feminist, Marxist, and historicist scholars who had been attacking them for their principled indifference to history, the material base, justice and value, suffered their heaviest blow since the death of de Man himself in 1983. Deconstruction's dominance had discouraged any ethical evaluation of the author; but now that that dominance was rapidly proving to be delusory, the repressed--ethics, which had been repressed, ironically enough, because it was seen as an agent of repression--was returning in force. "

The point of this experpt is that ethics (and their absence) do play a role in the work of academics and private sector software engineers. Whether we write software for the academy or for a corporation, we must eventually confront the repercussions of our actions and the effects that they will have on our community. Trying to deny this responsibility is, in Harpham's words "delusory".

Just because we are engaged in theoretical pursuits in software engineering or because we are "trying to earn a buck" doesn't mean we are immune to consideration for ethics and principles.

Yet the Open Source Initiative has repeatedly asserted the contrary. Earlier this summer, Lawrence E. Rosen, the general counsel for the OSI responded to a letter of mine inquiring about the potential for a non-military Open Source license that "Software is not viewed by the open source community as a valid way to address political or moral issues." It seems not everyone agrees with them and that I'm not alone in my belief that software is subject to the judgement of ethics just as surely as literature is.

The Computing Culture Group at MIT has stepped out of the "software should be politically neutral because ethics play no role in science" myth which is actively propagated by the Open Source Initiative. They have taken an ethical stand and are working, in their own way and of their own free will, to oppose the concentration of social power in their society.

As software developers, we have a responsibility to ourselves and to one another not only to write software for our economic progress, but also for social and ethical progress. The Computing Culture Group recognizes this, and hopefully their work will inspire software developers around the world to reconsider the conflict of interests they have as both scientists and as citizens of a free society.

    Sources Cited

  • Harpham, Geoffrey Galt. "Ethics". Selected from "Critical Terms for Literary Study" Lentricchia , Frank and McLaughlin, Thomas (editors). University of Chicago Press. 1995. (388-389).
  • MIT Open Government Information Awareness Project.
Related Topics >>