For those of us who have read Paul Graham's submarine essay, should the last paragraph be a giveaway? The "AI theoretician's" quote seems to have nothing to do with the rest of the article.
> [The] PR industry, lurking like a huge, quiet submarine beneath the news. Of the stories you read in traditional media that aren't about politics, crimes, or disasters, more than half probably come from PR firms.
Frankly, I'm kinda scared about a negative-trust future.
Compared to the 90's, not only are the financial pressures for scams/fraud/astroturfing rather extreme, but the cost of running lots of complicated lies is dropping like a stone.
If you're talking about the quote being a giveaway that the article is PR, I'm not following you. The point of the last paragraph is a warning that outsourcing ethical decisions to an AI is likely to result in decisions that one might not actually make and find morally dubious.
And prioritizing Consequentialism in AI, especially with weapons, is a dangerous bargain. "How do you make decisions when the consequences are unclear?" Since when are the full consequences _ever_ clear?
