Towards an Information Operations Kill Chain

Cyberattacks don’t magically happen; they involve a series of steps. And far from being helpless, defenders can disrupt the attack at any of those steps. This framing has led to something called the “cybersecurity kill chain“: a way of thinking about cyber defense in terms of disrupting the attacker’s process.

On a similar note, it’s time to conceptualize the “information operations kill chain.” Information attacks against democracies, whether they’re attempts to polarize political processes or to increase mistrust in social institutions, also involve a series of steps. And enumerating those steps will clarify possibilities for defense.

I first heard of this concept from Anthony Soules, a former National Security Agency (NSA) employee who now leads cybersecurity strategy for Amgen. He used the steps from the 1980s Russian “Operation Infektion,” designed to spread the rumor that the U.S. created the HIV virus as part of a weapons research program. A 2018 New York Times opinion video series on the operation described the Russian disinformation playbook in a series of seven “commandments,” or steps. The information landscape has changed since 1980, and information operations have changed as well. I have updated, and added to, those steps to bring them into the present day:

  • Step 1: Find the cracks in the fabric of society­—the social, demographic, economic and ethnic divisions.
  • Step 2: Seed distortion by creating alternative narratives. In the 1980s, this was a single “big lie,” but today it is more about many contradictory alternative truths­—a “firehose of falsehood“—­that distorts the political debate.
  • Step 3: Wrap those narratives around kernels of truth. A core of fact helps the falsities spread.
  • Step 4: (This step is new.) Build audiences, either by directly controlling a platform (like RT) or by cultivating relationships with people who will be receptive to those narratives.
  • Step 5: Conceal your hand; make it seem as if the stories came from somewhere else.
  • Step 6: Cultivate “useful idiots” who believe and amplify the narratives. Encourage them to take positions even more extreme than they would otherwise.
  • Step 7: Deny involvement, even if the truth is obvious.
  • Step 8: Play the long game. Strive for long-term impact over immediate impact.

These attacks have been so effective in part because, as victims, we weren’t aware of how they worked. Identifying these steps makes it possible to conceptualize­ and develop­ countermeasures designed to disrupt information operations. The result is the information operations kill chain:

  • Step 1: Find the cracks. There will always be open disagreements in a democratic society, but one defense is to shore up the institutions that make that society possible. Elsewhere I have written about the “common political knowledge” necessary for democracies to function. We need to strengthen that shared knowledge, thereby making it harder to exploit the inevitable cracks. We need to make it unacceptable­—or at least costly—­for domestic actors to use these same disinformation techniques in their own rhetoric and political maneuvering, and to highlight and encourage cooperation when politicians honestly work across party lines. We need to become reflexively suspicious of information that makes us angry at our fellow citizens. We cannot entirely fix the cracks, as they emerge from the diversity that makes democracies strong; but we can make them harder to exploit.
  • Step 2: Seed distortion. We need to teach better digital literacy. This alone cannot solve the problem, as much sharing of fake news is about social signaling, and those who share it care more about how it demonstrates their core beliefs than whether or not it is true. Still, it is part of the solution.
  • Step 3: Wrap the narratives around kernels of truth. Defenses involve exposing the untruths and distortions, but this is also complicated to put into practice. Psychologists have demonstrated that an inadvertent effect of debunking a piece of fake news is to amplify the message of that debunked story. Hence, it is essential to replace the fake news with accurate narratives that counter the propaganda. That kernel of truth is part of a larger true narrative. We need to ensure that the true narrative is legitimized and promoted.
  • Step 4: Build audiences. This is where social media companies have made all the difference. By allowing groups of like-minded people to find and talk to each other, these companies have given propagandists the ability to find audiences who are receptive to their messages. Here, the defenses center around making disinformation efforts less effective. Social media companies need to detect and delete accounts belonging to propagandists and bots and groups run by those propagandists.
  • Step 5: Conceal your hand. Here the answer is attribution, attribution, attribution. The quicker we can publicly attribute information operations, the more effectively we can defend against them. This will require efforts by both the social media platforms and the intelligence community, not just to detect information operations and expose them but also to be able to attribute attacks. Social media companies need to be more transparent about how their algorithms work and make source publications more obvious for online articles. Even small measures like the Honest Ads Act, requiring transparency in online political ads, will help. Where companies lack business incentives to do this, regulation will be the only answer.
  • Step 6: Cultivate useful idiots. We can mitigate the influence of people who disseminate harmful information, even if they are unaware they are amplifying deliberate propaganda. This does not mean that the government needs to regulate speech; corporate platforms already employ a variety of systems to amplify and diminish particular speakers and messages. Additionally, the antidote to the ignorant people who repeat and amplify propaganda messages is other influencers who respond with the truth­—in the words of one report, we must “make the truth louder.” Of course, there will always be true believers for whom no amount of fact-checking or counter speech will convince; this is not intended for them. Focus instead on persuading the persuadable.
  • Step 7: Deny everything. When attack attribution relies on secret evidence, it is easy for the attacker to deny involvement. Public attribution of information attacks must be accompanied by convincing evidence. This will be difficult when attribution involves classified intelligence information, but there is no alternative. Trusting the government without evidence, as the NSA’s Rob Joyce recommended in a 2016 talk, is not enough. Governments will have to disclose.
  • Step 8: Play the long game. Counterattacks can disrupt the attacker’s ability to maintain information operations, as U.S. Cyber Command did during the 2018 midterm elections. The NSA’s new policy of “persistent engagement” (see the article by, and interview with, U.S. Cyber Command Commander’s Gen. Paul Nakasone here) is a strategy to achieve this. Defenders can play the long game, too. We need to better encourage people to think for the long term: beyond the next election cycle or quarterly earnings report.

Permeating all of this is the importance of deterrence. Yes, we need to adjust our theories of deterrence to the realities of the information age and the democratization of attackers. If we can mitigate the effectiveness of information operations, if we can publicly attribute—if we can respond either diplomatically or otherwise­—we can deter these attacks from nation-states. But Russian interference in the 2016 presidential election shows not just that such actions are possible but also that they’re surprisingly inexpensive to run. As these tactics continue to be democratized, more people will attempt them. Deterring them will require a different theory.

None of these defensive actions is sufficient on its own. In this way, the information operations kill chain differs significantly from the more traditional cybersecurity kill chain. The latter defends against a series of steps taken sequentially by the attacker against a single target­—a network or an organization—and disrupting any one of those steps disrupts the entire attack. The information operations kill chain is fuzzier. Steps overlap. They can be conducted out of order. It’s a patchwork that can span multiple social media sites and news channels. It requires, as Henry Farrell and I have postulated, thinking of democracy itself as an information system. Disrupting an information operation will require more than disrupting one step at one time. The parallel isn’t perfect, but it’s a taxonomy by which to consider the range of possible defenses.

This information operations kill chain is a work in progress. If anyone has any other ideas for disrupting different steps of the information operations kill chain, please comment below. I will update this in a future essay.

This essay previously appeared on Lawfare.com.

EDITED TO ADD (10/10): I have updated the kill chain. (Blog link here.) Please use the updated version.

Posted on April 26, 2019 at 6:09 AM • 54 Comments

Comments

Sok Puppette • April 26, 2019 8:09 AM

The ability to speak anonymously is extremely important if you don’t want your discourse to degenerate into groupthink. In a democracy, groupthink tends to turn into irrational, tyrranical, or even downright genocidal policy decisions. That’s probably the most important failure mode of democracy, even if there are others.

Draconian “attribution” necessarily destroys anonymity and is therefore a non-starter.

If people can’t properly weight anonymous material, then that means that this whole democracy thing is just plain not gonna work. Trying to fix it with attribution will just break it in a different way.

At most you could try to do something about deliberate misattribution by impersonation, but such misattribution isn’t really necessary for these “information operations”, so that won’t solve the problem for you.

Jason • April 26, 2019 8:19 AM

@Sok Puppette

It’s a sliding scale. Anonymous Sources must provide a higher degree of “verifiable data” in their reporting.

AR • April 26, 2019 8:28 AM

“We need to become reflexively suspicious of information that makes us angry at our fellow citizens.”

Neto • April 26, 2019 8:44 AM

Hi,

Bruce, I’ve followed you and this blog for years now and I really appreciate everything you do but can’t stop but feel that, Everytime it comes to subjects relating the elections you have a very focused view from a perspective that buys a certain set of axioms about “fake news” and “propaganda” that come from a clearly Democrat party side.

I fear there’s an underlying “us vs them” hidden in there. As if propaganda only came from external foes and the republican/conservative side. As if “reputable news outlets” hadn’t used their own type of propaganda for ages.

Fake news is not new, it was just a coined term to attack politically. Arguments around truth always happen in politics and thinking it’s a binary decision that one can easily make is disingenuous and plays into the “good and bad guys” narrative.

I also see a tendency to remove agency from people as if, if they choose something after someone claims they’ve been exposed to propaganda then that means they’ve been fooled.

If you step outside the US and look at politics around the world, this problem is not new and not particular to one political party nor something you can really solve with a mindset of being “in the right”.

I’m more of a fan of decentralized security concepts than talking in regards of teams/tribes because the latter doesn’t translate to different context and involves buying into the axioms set by the tribe/team.

Anyway, this is probably a senseless rant but I just wanted to give my perspective from an empathic but earnest point of view.

Will • April 26, 2019 8:47 AM

“We cannot entirely fix the cracks, as they emerge from the diversity that makes democracies strong…”

Surely this must be wrong. Diversity drives tribalism instead of democracy. I would say it’s the learned willingness to bridge the gap, despite diversity, that makes a democracy strong, but this trait has to be taught.

Humdee • April 26, 2019 9:52 AM

We seemed to have strayed from technological security to propoganda studies. Both are legitimate topics but i have to admit that i have more respect for @bruce’s comments on the former than the latter. @neto raises some legitimste concerns; i have another. #2 seems to me an invitation to myopia and group think. Bubbles and selection bias is a greater threat than discord by a thousand cuts.

mark • April 26, 2019 11:17 AM

Reading the steps… that is a literal description of the development of Faux “News”, and their game plan. They are about as anti-democratic as you can get, and not be covert.

Harmless Drudge • April 26, 2019 12:02 PM

Apropos of neto’s comment implying a pro-Democrat party bias on Bruce’s part:

This is not my impression at all, nor I think would it be the impression of anyone living outside the US. Propaganda is real and it is used by political parties on all sides of elections, and not just in the US, as the recent investigations into the malfeasance surrounding the Brexit referendum in the UK have shown (to say nothing of the actions of Cambridge Analytical further afield).

We face a global crisis of democratic legitimacy because of what technology and money can enable. It’s a considerably more important issue than partisanship in the GOP v Democrat battle in the US. See Carole Cadawalladr’s recent TED talk and read her work. See also the book “Moneyland” by Oliver Bullough.

Quibbling about party bias in these circumstances is like complaining about the menu on the Titanic after it has hit the iceberg. It would be far more useful to focus on what can be done to protect democracy and democratic institutions from abuse by any bad actors.

Ross Snider • April 26, 2019 1:18 PM

Yes please.

I mean, this is maybe phrased a little self-important: “defend democracy”.

Between visibility, information operations treaties, and an active information operations kill chain and defense, hopefully we can get the geopolitical competition between rival nations to cool down a little, so common people around the world can live more securely in their respective countries.

I’m use the KGB already have a concept of an information operations kill chain (to defend against US operations on their society), but hopefully whatever conversation and improvements LawFare can make will improve their defenses, Israels’ defenses, UK’s defenses, China’s defenses, etc.

vas pup • April 26, 2019 2:03 PM

That is kind of related to the subject:

Exposure to violence does not change the ability to learn who is likely to do harm, but it does damage the ability to place trust in ‘good people,’ psychologists report.
https://www.sciencedaily.com/releases/2019/04/190426075442.htm

” Baskin-Sommers, her Yale colleague and co-senior author Molly Crockett, and graduate students Jenifer Siegel and Suzanne Estrada evaluated 119 males incarcerated in Connecticut prisons, some of whom scored high on exposure to violence. Participants learned about two strangers who faced a moral dilemma: whether to inflict painful electric shocks on another person in exchange for money. While the “good” stranger mostly refused to shock another person for money, the “bad” stranger tended to maximize their profits despite the painful consequences. The participants were asked to predict the strangers’ choices, and later had to decide how much trust to place in the good versus the bad stranger.

The team found that participants with higher exposure to violence effectively learned that the good stranger made fewer harmful choices than the bad stranger. However, when deciding whom to trust, they trusted the good stranger less than participants who had a lower exposure to violence.

“In other words, exposure to violence disrupted the ability to place trust in the ‘right’ person,” said Siegel, an Oxford doctoral student and first author of the paper. “We also saw that this disruption led to a greater number of disciplinary infractions within the prison setting.”

Crockett said the findings suggest that exposure to violence changes the way people use information they’ve learned to make healthy social decisions.

“Social flourishing depends on learning who is likely to be helpful vs. harmful, and then using that information to decide who to befriend versus avoid,” she said. “Our research suggests exposure to violence impairs this crucial aspect of social functioning.”

Clive Robinson • April 26, 2019 3:29 PM

@ Bruce,

Here the answer is attribution, attribution, attribution. The quicker we can publicly attribute information operations, the more effectively we can defend against them.

No you can not with current types of attribution that become public.

Because they are way to easy to shot full of holes and in the process themselves become as much “fake news” as anything else.

Most people understand the rider on the CIA motto of “all others we verify”. As I’ve noted before attribution is hard very hard and it is easy to “false flag” due to the nature and weaknesses of the technology involved.

As I’ve pointed out you should only publically attribute when you have verification that will stand up in a court of law.

Last century this used to be the case as was seen during various “crisis situations” during the cold war and slightly later time periods. This century any old nonsense that can be seen with a little thought as being “political target motivated” from various US administrations of both animals has been the norm. People are starting to see “it was XXX” from the US Administration, where XXX is China / Iran / North Korea / Russia / latest political whipping boy as being nothing but empty propaganda or “Fake News”. Much as in “The Boy Who Cried Wolf” real alarms are just going ignored.

So “Stump up or Shut up” with real evidence would be sage advice if you don’t want people thinking you are pulling their chain.

Look at it this way, it would be a bunch of fairly dim whitted individuals who did not ask themselves “What are other countries doing?” both in terms of attribution and in terms of cyber-espionage. After all if the Dutch, Israeli, UK and other Western Nations can come up with “end point intelligence” amounting to HumInt that in effect would be Court admissible you have to ask why the US IC/SigInt entities are apparently not delivering? Also you have to think “If they are doing that to Russia, what are they doing to other nations both friend and foe?”. There is only a limited set of conclusions they can come to and the application of Occam’s Razor gives up the fact that the various US Administrations are playing the “politically targeted motivation” game, and thus are being dishonest with the American people to the point of treating them like idiots.

As for MSM yes Russia has RT as a mouth piece but then America has Fox / CNN / etc, but then most countries have politically biased MSM. My advise is “watch them all and then sift out the truth”. That is that so many are biased in their reporting to the point of being dishonest, but usually all the stories have a germ of truth in them. The more news sources you watch especially the ones you don’t like, the faster you are going to learn to split the truth from oppinion and bias.

The realy funny thing is in the UK, the BBC had a reputation for impartiality and honest reporting. This did not suit a certain UK political party that was perpetually claiming the BBC was biased… Thus having sown the false perception they forced through legislation for supposadly “balanced reporting”. Therefor the BBC is forced to get “opposing voices” for each subject. This actually magnifies the “loony fringe” and “Fake News” asspects, and gives them a “public soap box” that nobody in their right mind would normally do, thus they gain “credability”. All of which suits the political party concerned and it’s like minded “loony / radical off shoots”. It was this legislation that gave and still does give credability to those who have actually been shown to have broken the law during the run up to the Brexit vote and subsequently, which is why Europe is scratching it’s head and the process stagers from one calamity to another. The moral being “Take care in what you wish for” when it comes to dealing with “Fake News by regulation”.

In the US with the likes of Facebook you are already seeing the suppresion of quite legitimate political discussion that shall we say is not popular in certain corners, whilst undesirable corpor-political bias gets a free run…

That is all of what you suggest as solutions is very easy to subtly abuse to give quite strong censorship…

name.withheld.for.obvious.reasons • April 26, 2019 5:57 PM

@ Bruce, et al

I hate to do this here, my PGP key-space (key and address mapping) is currently unavailable and entity validation has failed, but I had briefly outlined the far-a-field concept that I’d coined in 2001. The title of the white paper, “Knowfare, Hybrid Asymmetrical Information Warfare”, covered a formal treatment of a structure hypothesis that would find a home in the government’s unfounded seizure of private property (NSA states that there are no barriers to the scope of extent to their claim that “ALL ELECTRONIC” data is within their purview). This includes offline hard drives or other storage devices. Thus, in this case, there is no rational basis to assert any personal, private party, can exist. Truly a pre-emptive action against being secure…in papers and effects.

I keep harping on the issue that the government has “secretly” embraced an “IT’S ALL OURS”. And power, the ability to engage in the act of war, has objectively breached the walls of the castle that is congress. Congress, the sole originator of such matters is absent and only two conditions allow the Executive to Act; Invasion or Insurrection.

James Madison would be absolutely repulsed by the structure of the Executive and the specific usurpation of constraints disallowing Conquest and Adventure. We now have a government of kings. If you haven’t referenced Presidential Policy Directive 20, I suggest you do so. Leaked copies were made available in 2013. Two major take aways:

1.) The U.S. is on a unilateral pre-emptive (meaning illegal) war footing, and specific policies (which cannot have a basis in law as the two conditions have not been met) direct the DoD to penetrate, subvert, plant, and trespass on property not under their control.

2.) Power has devolved, the ability to the assistant/deputy security of any of the agencies of jurisdiction (not limited to the 17 organizations under the IC umbrella) can initiate and act using war powers without prior notice.

We have a system of social order, that as a society, claims no need to provide for order or civil consideration. The United States as an occupied territory of the MIC, cannot expect anything resembling those things that law, structure, or operational harmony (i.e. that the US is a collection of individuals as citizens, and is the sovereign) striving for a common goal of “a more perfect union”.

Petre Peter • April 26, 2019 7:01 PM

Counter attacks are an important deterrent tactic to scare incoming attackers but they can be a drain on the resources if the tools for defending are different than the tools for attacking. Persistent engagement makes sense because in cyberspace, the same tool is used for attack and for defense which lowers the costs of counter attacks.

Clive Robinson • April 26, 2019 10:03 PM

@ Name.Witheld…,

(NSA states that there are no barriers to the scope of extent to their claim that “ALL ELECTRONIC” data is within their purview). This includes offline hard drives or other storage devices.

That sounds about right, though you don’t say if they put any jurisdictional limitations on that… I’m guessing they did not so to quote Buzz Lightyear “To infinity and beyond” 😉

As a Russian who (studied law and) is married to a friend explained Russia has a series of laws that would in effect mean that it would be legal for any appointed Russian to lawfully execute any member of the NSA…

Put simply Russia passed a law in 2006 that legalises extra-judicial extra-teritorial execution of “exyreanists” on the nod of the Russian President. It has minimal requirments one of which is to slander the President another depending on how you translate it is to threaten to violate or actually violate Russian sovereign rights. This law has been mentioned a few times in the Western press,

http://news.bbc.co.uk/2/hi/europe/6188658.stm

Likewise they have a law that in effect defines illegal entry into a Russian computer system which is very broadly defined as violating Russian sovereign rights…

And laws I’ve not seen mentioned in the press that are kind of equivalent to “prescribed” terrorist / extreamist / etc organisations. I’m not too sure on the details but apparently it’s the equivalent of “guilt by association” / “conspiracy” it was designed to supposadly deal with “terrorism” but is written in much broader language. So in effect if you are a member of an organisation/entity or can be shown to be associated with an organisation/entity that has other members that have threatened to violate or violated Russian sovereign rights then you are as guilty as they are… Appart from the 2006 law on executions, many countries have similar laws or legal procedures.

Oh then going back further you have the UK RIPA legislation that basically claims jurisdictional rights to files/data that are reachable from any network connected to or passing through UK territorial waters. The “network” is not exactly defined, in essence it’s potentially any method of communication. So hypothetically a person carrying any kind of data medium is communicating it, thus forms a part of a network, their mobile phone with camera likewise. Potentially even a satellite footprint passing over UK territorial waters at some point can also pass over some other country in the world at some other point in time, which makes another network. Back when RIPA was a draft a group of us sat down to work out if there were any real limitations on it and the conclusion was “probably not”…

Impossibly Stupid • April 26, 2019 10:37 PM

Step 1: Find the cracks. … We need to become reflexively suspicious of information that makes us angry at our fellow citizens.

Do we, though? Maybe the right defense here is to be honest with each other about the significance of our division. I mean, shouldn’t we be a bit upset when our “fellow citizens” engage in bad behaviors? Sometimes the cracks are real.

The scientist in me says that instead of expending a lot of energy futilely trying to hold things together, we should explore what happens when you use those natural cracks to split things apart. Quarantine anti-vax populations. Brexit away! Let Texas secede. In whole or in part, we need to be more experimental when it comes to how we manage the cracks.

Step 4: Build audiences. … Social media companies need to detect and delete accounts belonging to propagandists and bots and groups run by those propagandists.

I think a better defense is to abandon social media itself. You don’t need those companies to mediate your genuine relationships.

Step 5: Conceal your hand. Here the answer is attribution, attribution, attribution.

Said with some irony, of course, since this blog allows anonymous commenting. It is a double-edged sword, though, because strong attribution can have dangerous consequences when paired with false attribution. And there’s the flip side, too, like when people who post something offensive to social media try to claim their account got hacked.

Step 6: Cultivate useful idiots. … Focus instead on persuading the persuadable.

Another double-edged sword. Perhaps they’d be less idiotic (or at least less useful) if they weren’t so easy to persuade in the first place. The root defense for a lot of these steps seems to be better educating the public so that they have the tools they need to behave like responsible members of a civil society. If they still can’t do that, well, let those cracks form and let their ice floe float away.

Pris Jognera • April 27, 2019 10:37 AM

All these techniques have always existed, have always been used. The internet mostly amplifies them. A lot of internet activity is sloppy inflated gas-bag emptiness with a appearance of substance, analogous to everyone using a 5th generation jet fighter for daily commuting. A bicycle or a simple car is more practical. Forget computing and the internet except when it is the reasonable and proportionate tool. Of course, this will mean reforming one’s habits.

chuck • April 27, 2019 3:22 PM

It is also very useful to have a large media outlet and a highly compromised con man politician as the useful idiot to work with.

k15 • April 28, 2019 6:35 AM

Broken record here:
When everything and everyone around you seems designed to get your “this is fake” alarm bells ringing, who do you go to for help, without either affirming your assigned role as Loon or becoming the useful idiot for some larger scheme?

Geoffrey Nicoletti • April 28, 2019 9:32 AM

Even under the term “footprinting” one plans “successful” attacks so this is a terrific way of disrupting a successful attack. I once thought of shutting down the Internet (to protect so many sites) before all-out war (which still may be necessary) but one is, at the same time, protecting the enemy already in our pipes. We have to get the adversary not only out of our face but inside the body…the infrastructure. Great article.

Clive Robinson • April 28, 2019 10:32 AM

@ K15,

When everything and everyone around you seems designed to get your “this is malign but also fake” alarm bells ringing, who do you go to for help

Realisticaly you can not everybody has biases. The only way is be “self reliant” and as I commented above,

    As for MSM yes Russia has RT as a mouth piece but then America has Fox / CNN / etc, but then most countries have politically biased MSM. My advise is “watch them all and then sift out the truth”. That is that so many are biased in their reporting to the point of being dishonest, but usually all the stories have a germ of truth in them. The more news sources you watch especially the ones you don’t like, the faster you are going to learn to split the truth from oppinion and bias.

As @Alyer Babtu, also noted above,

    the “hobbies of rich men” as Chesterton says (about 100 years ago). Since these interests are often in opposition, one has to read all the news, to try to sift out the fake. Learning to read the newspaper is a fundamental requirement for political freedom

Troutwaxer • April 28, 2019 11:24 AM

@ K15

You have to know what you really want for your country and the people who live there. Otherwise, you react to everything and are easy to manipulate. (Bruce not bringing this up is a weakness in an otherwise excellent article.)

gordo • April 28, 2019 8:37 PM

Generalizing heavily, given that the medium is universal, more and more everywhere, more and more critical, always on, intractably accessible and subject to breach anywhere at a moment’s notice, it’s only natural that as the various actors traverse this space they’ve come to recognize each other more clearly, what each is doing and their capabilities. Everyone operating in this space wants to know what their adversaries are doing and that won’t change, nor should it.

Norms develop out of relationships or how we treat each other and how we want to be treated. Despite the disparate political orientations that separate us, whether domestic or international, certain core interests or sectors that allow for the basic day-to-day functioning and outlay of common goods, regardless the society, should be respected.

Spheres or networks of influence will not go away. They are a feature of life as we know it. That some societies are more open than others is nothing new. That most societies want to protect themselves from undue foreign influence is nothing new. The creators of an instantaneous global communications medium now view themselves as vulnerable to attack by way of technological systems that they themselves produced. I suspect that, except for the speed of its adoption, in the annals of technology history this kind of situation is nothing new.

That said, I view the threat of foreign, malign information operations against open societies as relatively low-hanging fruit. Employing timely signals intelligence data-sharing agreements between governments and the larger communications platforms, as well as developing robust electioneering communications payment authentication systems would go a long way toward successfully combating threats from foreign actors.

On the sowing of confusion and discord from otherwise legitimate domestic actors, that problem is much deeper and is a story for another day.

Mark • April 29, 2019 12:15 AM

Bruce, as others have pointed out, your commentary is pretty biased based on your obvious political beliefs. Since realising this, I’ve questioned what I’ve learnt from you in the past.

Posting articles to Soros’ far-left Open Society makes me question your agenda. The very same organisation released a hit piece report on the so-called “alternative influence network” that lied and linked people (Tim Pool, for example) to people to whom they had never spoken. It was complete propaganda.

Linking to CNN and The Guardian about so-called “fake news” is laughable. For example, the gender pay gap has been debunked since Thomas Sowell in the ’80s, yet the Guardian never stops bleating on about it repeatedly. Your kill chain explains the Left’s narrative over truth approach in this example perfectly.

Likewise, reports from universities (notoriously far-left, especially the Humanities) aren’t going to cut the mustard.

Listen to some Ben Shapiro, then get back to us.

Clive Robinson • April 29, 2019 4:04 AM

@ gordo,

On the sowing of confusion and discord from otherwise legitimate domestic actors, that problem is much deeper and is a story for another day.

It’s also the story that most worries me…

As you know I’ve warned that attribution with IP networks as we currently have them is hard for various reasons. Well two further thoughts arise from this,

1, Nearly all our news now gets transported on IP networks.

2, Attributing a story had untill IP networks came to the fore been the hardest thing to do, which was why investigative journalists rarely wrote new pieces as the process takes time.

Now building newspaper style stories is easy due to the wealth of information which is why we have seen so much “clickbait” as proprieters push for “eyes on site” figutes. Quickly attribution got replaced by “whats trending”. But are such stories even true, well trending is a statistical measure of popularity and has nothing to do with accuracy of details which is what attribution is. Thus the traditional type of well researchrd and attributed news has to a certain extent died out in MSM… And what has replaced it is very very much more easy to get wrong due to the difficulty of both types of attribution.

But there have been other changes with “advitorials” and “paid for content”, pushing traditional news out of many newspapers. For exactly the same reason legislators just love “pre-written legislation from lobbyists” because it’s effectively “no work” other than “cut-n-paste”.

In the environment that has resulted it has become way way to easy for “news” to become “cut-n-paste” thus about as reliable as myths…

So is it surprising that anyone and almost everyone with sufficient resources is generating what would fall in the “fake news” catagory? And worse use it just as nearly every government has since the 1800’s for propaganda…

Back during WWII George Orwell worked for the British Broadcasting Corporation and saw first hand censoring and other types of “fact manipulation” done for supposadly “moral purposes”. However he realised where it eas going and in 1948 wrote a book with a fictional main plot set in the future, but the supporting story threads were where the real message was about how we were being manipulated and where he saw it as going. Unfortunatly he appears to have been right in oh so many of his predictions, I guess we are seeing the rest come true with time.

One thing George Orwell made clear was the “Invention/creation of a Distant enemy” that was used as a distraction. Thus keeping the home nation on a permanent “war footing” thus justifing “censorship” and all the other properganda for “moral purposes”.

What Orwell missed but later authors picked upon was the idea of “a state within a state” that was effectively the unseen hand behind the throne manipulating a pupet leader and government.

Thus as well written stories of fiction become the realities of life (9/11 being just one). As people who seek power use the stories as templates for their actions, it is unsuprising to find the likes of Cambridge Analytica poping up. We saw during the Obama Administration the significant rise of Silicon Valley in the political power game.

Thus we have to consider the use of “false flag” operations to hide the sources of “usefull narative” feed to “usefull idiots”. As I’ve pointed out with the “US four horsemrn of the cyber-apocalypse” it is very much “unattributed narative”. Common sense and the application of a little logic should give most cause to doubt the “It’s Russia”, “It’s China” narative, these are just the rabid reworking of the 1950’s “Reds scares”, done to keep the US population in line as they were in the past just as Orwell warned.

But what of the other two, Iran and North Korea?

Well sometimes you need “fake enemies, that are real”. That is you need a real rather than fictitious target, you can actually “bomb back to the stone ages” for the purposes of keeping other nations in line. They are not real enemies in that neither the citizens nor the governments of those countries have any intention of committing acts of agression against the US unless threatened. However certain people in the US see them in their “regional setting” and building hostility towards them and provoking them is the equivalent of “putting the pot on to boil”. They are deliberatly destabalising the region for their own reasons.

Standing back a few paces quickly tells you why certain people in the US are doing this, and both Iran and North Korea have likewise realised and thus taken steps that have actually made it clear to the rest of the world that it is US generated propaganda.

But even US politicians have warned against some of these “certain people” within their midst by talking about the MIC and War Hawks.

Thus people clearly have to ask where these unattributed potentially “false flag” stories have actually arisen from and why. That is who are the people behind them steadily gaining power against not just the US citizen but the US Government as well, and toward what end…

justinacolmena • April 29, 2019 10:42 AM

Anthony Soules, a former National Security Agency (NSA) employee who now leads cybersecurity strategy for Amgen.

That’s too much security clearance to go drugging and bioteching.

Step 8: Play the long game. … We need to better encourage people to think for the long term: beyond the next election cycle or quarterly earnings report.

The truth will always out in the long term. Meanwhile, it’s rather like breathing. You’ve got to take the next breath to stay alive before you start thinking too many breathing cycles into the future. People get ahead of themselves.

gordo • April 29, 2019 9:00 PM

@Bruce Schneier,

Yes, we need to adjust our theories of deterrence to the realities of the information age and the democratization of attackers.

As information operations go, there is definitely something to be said regarding the attack surface of the media environment or the state of the fourth estate, at the very least, for purposes of situational awareness.

excerpts from the book
What Orwell Didn’t Know
Propaganda and the New Face of American Politics
Edited by András Szántó
Introduction by Orville Schell
PublicAffairs, 2007, paperback

Welcome to the Infotainment Freak Show
By Martin Kaplan

In the year 1984, reflecting on the book 1984, sociologist Neil Postman gave a series of lectures at the University of Southern California Annenberg School for Communication in which he raised the question: Were Americans doomed to inhabit George Orwell’s authoritarian dystopia? His answer: No. We were veering instead toward the hedonic world depicted in Aldous Huxley’s Brave New World and were in imminent danger of amusing ourselves to death.

The pathology Postman diagnosed derived from what he termed the epistemology of entertainment, which was on the verge of displacing the epistemology of the Enlightenment. Entertainment substitutes juxtaposition for order, storytelling for truth telling, graphics for texts, sensation for reason, spectacle for seriousness, combat for discourse, play for purpose, sizzle for steak. “Our priests and presidents,” he wrote, “our surgeons and lawyers, our educators and newscasters need worry less about satisfying the demands of their discipline than the demands of good showmanship. Had Irving Berlin changed one word in the title of his celebrated song, he would have been as prophetic, albeit more terse, as Aldous Huxley. He need only have written, There’s No Business But Show Business.”

Today, the clock strikes thirteen every hour in America. That startling digit is not evidence that Big Brother rules, but rather that entertainment reigns. In contemporary America, and arguably in most industrial democracies, the imperative of practically every domain of human existence is to grab and hold the attention of audiences. In politics, it is now more important for a candidate to have big name recognition, and the money to buy big media, than it is to have big ideas. In news, it is absolutely essential to have high ratings, but it is only optional to have high accuracy. These days, a university can get by with average scholarship, but without a strong brand identity n the educational marketplace, it will surely perish. A museum at fails to mount blockbuster traveling shows, complete with a killer gift shop, is on a fast track to losing revenues, patrons, and public subsidies. To attract tourists, cities now turn to starchitects, whose glitzy creations rival Hollywood and Babel. Commerce, once about goods and services, is today about experiences and aspirations; every store promises a little bit of Disneyland. Even our interior lives are played out through the tropes of entertainment. It is now normal to experience clothing as wardrobe, furniture as set decoration, other people as characters, conversation as dialogue, and events as plot points in the narratives of our life stories, which come complete with voice-overs and flashbacks.

Postman’s jeremiad clearly failed to stem entertainment’s tide, just as Orwell’s fables failed to vanquish Totalitarianism. Our present moment in the history of consciousness is widely known as postmodernity – “pomo,” in a fun shorthand fitting for the age of show business. In the first part of the twentieth century, Karl Popper said that philosophy’s task was to demarcate between what is scientific and what is not, and he linked the project of science to the robustness of open societies. But by the twilight of that century, pomo intellectuals were declaring the Popperian project dead, and their doppelgangers in popular culture have since been dancing on its grave. By now we all know the postmodern mantra: Objective knowledge is a mirage. There is no such thing as epistemology without a knowing subject. Science is no longer a privileged realm, designed to weigh the truth of competing claims, but is yet another act in the pomo circus, where knowledge based on rigorous trial and error is on the same footing as all other tribal belief systems. All reality is socially constructed. Everything, even truth, is politics. Politics, once conceived as the craft of decisionmaking, has been remade as the art of attention getting. Communication, rather than striving to convey truth and meaning, now prizes informing an audience less than having an audience. The public interest has been redefined as what the public is interested in; the public sphere has become a theater, where citizenship is a performance.

Orwell famously worried about the divorce of public discourse, including journalism, from truth, but he did not anticipate its remarriage to entertainment. Journalism, whose practitioners in high-end newspapers and early broadcast outlets once saw their task as recording the truth, now largely functions to hold up a mirror for us to see our fun-house reflection. The old journalistic paradigm posited a search for accuracy, objectivity, and fairness. The new pomo paradigm declares those goals misbegotten; they are inherently unachievable. because everyone possesses and advocates for his or her own truth, the job of the reporter is not to adjudicate among competing assertions, but to assemble them into a de facto collage. The fairness of contemporary journalism resembles not the fairness of a judge and jury weighing evidence within a framework of rules, but rather the fairground of the carnival midway, where barkers out on behalf of their wares.

Whereas accuracy can never be achieved, “balance”-the new lowest common denominator and deceptive battle cry – is an easy goal; all you need to do is open up your airwaves and column inches to everyone. Better yet, open the public square to combatants, in polarized pairs. There’s no surer way to attract an audience than a bear fight, and no dispute is too nuanced not to be reducible by modern journalism to he-says versus she-says. Instead of trying to tell us what’s true, journalism now prides itself on finding two sides to every story, no matter how feeble one side may be. There’s no grand narrative making sense the progression of current events; there are just dueling narratives, competing story lines, alternate and equally plausible ways to connect the dots. It’s as though a generation of journalists has been weaned on Jacques Derrida and deconstruction, and their only recourse in the Dada zoo is pastiche.

In such a carnivalesque media ecology, people are patsies for propagandists. Even if a near majority of authorities holds one view about reality (on the causes of global warming, say, or the validity of evolution, the risks of mercury pollution, the existence of WMD in Iraq, the success of abstinence-only as a sex education strategy), a television “news” booker is always delighted to invite a fringe spokesperson onto a program as “balance,” to convey the notion that these are all wide-open questions. Journalists have recused themselves from ranking the legitimacy of “experts,” so corporate and political front groups with lofty names (Discovery Institute, Heritage Foundation, National Center for Policy Analysis, National Association of Scholars, et al. have the same opportunity to inject themselves into public debate as research institutions that still cling to old-fashioned standards of evidence and accuracy. These “think tanks” manufacture debate. That’s what they do: Their aim is to create the illusion of controversy, even when the facts are indisputable, because they know how enslaved contemporary journalism is by the tyranny of false equivalence. What’s more, the louder you are, the more outrageous your claim, the less civil your discourse, the farther you stand from common ground, the more welcome you will be as a guest and a source. It is not necessary to be right; to make the sale, it is only necessary to get attention.

The old paradigm depended on a hierarchy of gatekeepers. Within journalistic institutions, reporters were vetted by editors, copy editors, and executives; judgments were tempered by the time span of the daily news cycle. Among journalistic outlets, print publications set the gold standard, and national newspapers set the standard among them. But now, the brand names of old-guard institutions mean little to mass audiences. The Internet has made everyone a reporter, videographer, and distributor of content. This may well be a good thing in itself, but the Internet is also putting out to pasture the professionals who presided over the accumulation, fairness, and accuracy of the news.

The need to hold audiences round the clock has put a premium not on the information journalists see as important for citizens to know, but instead on the “content” that corporate owners bet will mesmerize consumers. The turning point probably came in the mid-1980s, when CBS News discovered that 60 Minutes could be a reliable cash cow. Today, nearly universally, news is programmed not as a public service but as a profit center. Its messages are designed to appeal to humans’ hardwiring. In Johan Huizinga’s formulation, we are not Homo sapiens, the creature who knows; we are Homo ludens, the creature who plays. Like it or not, our species is a sucker for novelty; sex, fear, pictures, motion, noise, scatology; fin; we can’t stop our eyes from turning toward celebrity and spectacle. Journalism’s job today is not to find and deliver us the worthwhile hidden in the muck, but rather to stream the muck at us, 24/7, and to sell our captive eyeballs to advertisers.

When Ted Turner launched CNN in 1980, there were high hopes for the broad diffusion of news. The results of the twenty-five-year experiment in round-the-clock cable “news,” which now includes Fox and MSNBC, are now in. Here’s what cable news is really good at: trapped miners, Michael Jackson, runaway brides, missing blondes, Christmas Eve murders, Princess Di, Paris Hilton, hurricanes, tsunamis, disinformation, whiz-bang graphics, scary theme music, polls, gotcha, HeadOn ads, “Thanks for having me,” people who begin every answer to antagonistic questions with “Look,” people who say, “I didn’t interrupt you when you were talking,” and anchors who say, “We’ll have to leave it there.” Here’s what cable news is not so good at insight, context, depth, reflection, proportion, perspective, relevance, humility, information, analysis, news.

There is nothing that pomo news likes more than Armageddon. When the Hezbollah kidnapping of Israeli soldiers in 2006 escalated into a war between Israel and Lebanon, it was covered as a tinderbox about to explode, a downward-spiraling crisis, the tipping point of a regional conflagration, the start of World War III. We were told it could trigger an oil crisis worse than the one in the late 1970s, resulting in gas lines and rationing and $20-per-gallon at the pump. It could spark worldwide inflation, recession, depression. It was the most dangerous moment since (fill in the blank). To be sure, it was a dangerous situation. But much of what the news media delivered was in fact crisis porn, fed to an insatiable audience, and itself a likely cause of the escalation of the crisis.

Journalism, especially television journalism, has tremendous ability to control the tone of what it covers. The quantity, the music, the graphics, the word choices can all be dialed up or down. The notion that professional news judgment-a reliable journalistic rule book – is what really drives the nature and kind of coverage is hopelessly quaint. The truth is that a missing white woman can easily be turned by the media machine into a global red-alert, and a holocaust in Africa can be marginalized as a sidebar story. When it comes to holding audiences’ attention, the only thing better than suspense is suspense about carnage, and the only thing better than suspense about carnage is suspense about the apocalypse. Terrorists, especially stateless terrorists, depend on the media’s addiction to fear and crisis. They have gamed the media system; they bank on getting their message amplified. This is not to diminish the legitimate news value of the horrors they perpetrate. But it’s also true that attempts to cool things off, reduce tensions, and back off from the brink are at odds with the sexy Nielsens that accompany realtime coverage of the end of the world. It is chilling that the arbiter we used to rely on for the facts is itself now a stakeholder with a vested interest in imagining the worst.

The firing, in the spring of 2007, of radio host Don Imus was depicted as a victory for civil discourse. In reality, the most important thing to know about Don Imus is that he was a financial gusher for Viacom and General Electric until his advertisers began peeling away. The only reason that shock jocks are on the air in the first place is that people pay attention to them. We lend our ears to Imus and his ilk because outrageousness amuses us. A merely curmudgeonly cowboy would not have pulled big numbers, and neither the political class nor the punditocracy would have returned his bookers’ calls. What made the powerful kiss Imus’s ring, and what made people tune in, was how bad-boy – how rude, disrespectful, licking-the-razor – he was. Clearly, large audiences liked to gasp at what he got away with, and CBS and NBC were champs at spinning those oh-my-Gods into ka-chings.

The same could still be said of the envelope-pushing by Rush Limbaugh, Ann Coulter, Glenn Beck, Sean Hannity, Michael Savage, and the dozens of other acts in the infotainment freak show. Their effect may be to debase discourse, inflame prejudice, sow ignorance, exculpate criminality, incite rancor, ruin reputations, and stoke the right-wing base-but their effect is not their job. Their job is to make money-for the-media conglomerates that employ them. We may revile them for being demagogues, but we are chumps if we ignore how relentlessly the companies that employ them monetize their noxious shtick. Those corporations are not in the news business, nor the public interest business, nor the patriotism business. They’re in the profit business.

l Imagine if the audience’s appetite for outrage extended to the atrophy of American democracy. Imagine if media bosses believed that we were insatiable for information about the apparent attempt to rig the 2008 election by politicizing the Justice Department and prosecuting phony voter fraud. Imagine if the same kind of blanket coverage that’s currently conferred on loopy astronauts, bratty rehaboholics, and slandered basketball teams were afforded instead to the assault on civil liberties and democratic processes now underway in America. Would we watch it the same Pavlovian way we watch tits, twits, and tornadoes?

Media executives think not. They don’t believe the jury is still out on that one. They don’t think that we’re addicted to junk news and shock jocks because it’s the only diet they’ve offered us; they think the market for civically useful information is simply saturated. And they don’t think that way because they’re just tools of the vast right-wing conspiracy (though some, like Fox, have made that their market niche) or because it serves their economic self-interest (though the tax cuts and wealth transfers whose consequences they’ve declined to cover have benefited them handsomely). No, they air what they air, and cover what they cover, as a capitalist service to us. Us, in the form of our mutual funds, our pension funds, our IRAs and 401(k)s, our collective American existence as Wall Street. Entertainment is exquisitely sensitive to demand. As long as we demand quarterly growth in profits more aggressively than we demand real news, the clowns will always get more airtime than the fifth column of hacks who have penetrated the halls of Justice.

Surely this knack for pandering to our taste for sensationalism is not why the news business is the only for-profit enterprise to be protected by the Constitution. Nor is it likely that the modern journalistic project, defined as the imperative to obtain market share, will do its part to deliver the educated citizenry that Thomas Jefferson said democracy depends on. Perhaps the Internet, by exponentially increasing the number of channels through which we can receive news, and by enabling new knowledge-networks that can be mobilized for information gathering, will prove to be a powerful antidote to the brave new world of entertainment über alles. But it is just as possible that the need to monetize cyberspace will have the same consequences for online information that it has had in the old media. On the Internet, no one knows if you’re Big Brother.

http://www.thirdworldtraveler.com/George_Orwell/What_Orwell_Didn%27t_Know.html
[italicization in the above are as found in the paperback]

Dennis • April 30, 2019 4:44 AM

@gordo, “On the Internet, no one knows if you’re Big Brother.”

The art of propaganda and “information warfare” predates World War I & II. This goes way back to ancient times. I believe the art had been honed and perfected by its perpetrators thru thousand years of practicality. If you want a first hand example of this, pick up any book of history as history was written by the victors.

The internet only goes so far as “reputation” goes. It does nothing to undermine deep rooted information confinement and the set of commonly accepted “logics” of modern times. Much of this has been trained and if one looks beyond the most basic fundamnetal knowledge block somewhere lays the truth.

Clive Robinson • April 30, 2019 8:02 AM

@ justinacolmena,

I don’t understand a single word of that marijuana-infused drivel.

It was not “marijuana infused” and most certainly not “drivel”.

It’s what various people have been saying here for quite some time.

News as it used to be, is for many dull as it has no “Glitzy” that the fakeness of stage managed “Reality TV” and sponsor driven “Internet Channels”.

That is people in the English speaking West have become shallow, very shallow and addicted to “Five Second brain fizz” from “Glitzy”. The motto of our times is “Bring me no dull, just Glitsy Glamour, nipple slips and moronic behaviour at which to scoff”. Thus each piece of Glitsy has to out do it’s self… The result is actually banality lit by the strobescope of special effects.

With such banality it is easy to generate and propagate “Fake Outrage” that is the core of “Fake News” in turn each “outrage” has to be “out done” in the otherwise vacuous echo chamber of fanboi-ism”. Thus you get polarization and the likes of Alt-Right, Britbart and the earlier Tea-Party and their moronic Atlas Shrugged quotes on placards toted by their little children. Each channel has to endlessly out do it’s self to “keep the faithfull” thus they go past polarisation to extreamism fairly rapidly.

Do you understand the above words?

If not I’ve a shortish reading list you can get upto speed on, including Sun Tzu, Machiavelli, Orwell and one or two others.

Bill • April 30, 2019 2:46 PM

Does anyone remember the era before everything was called “fake news”? I fear we can’t discuss here the “why” and “when” this change happened, due to it being political… but I miss the “good old days”…

gordo • April 30, 2019 10:33 PM

@ Bill,

“the good old days”…

I see the current fake-news era beginning in the 1980’s with the rise of boutique cable news networks, deregulatory frameworks and privatization. Each were in their own way reactions to the unrest of the 1960s and ’70s, e.g., civil rights, environmentalism, the Vietnam War, and the Church Committee. The “good old days”, despite its imperfections, allowed for “truth to be spoken to power”. That has been replaced by the daily din of the 24-hour “news” cycle and its competing narratives. There is, it seems, no longer any room for arbiters, only arbitrage.

@ Dennis,

beyond the most basic fundamnetal knowledge block somewhere lays the truth

Yes, histories as written and taught by victors tend to gloss over their purported failings and embellish their supposed virtues. The outcome of such reputational logic, whether held by a nation or personally, as history has so many times shown, is encapsulated in ancient truth:

Pride goeth before destruction, and an haughty spirit before a fall.

For such, it is as Julius Caesar wrote:

Experience is the teacher of all things.

Dennis • May 2, 2019 5:30 AM

@Clive Robinson @justincoma,

“Thus you get polarization and the likes of Alt-Right, Britbart and the earlier Tea-Party and their moronic Atlas Shrugged quotes on placards toted by their little children.”

And this is what drove the establishment so mad that they could be beaten by such pediatric display of politics. The placards seem moronic yet crisp to the point.

@gordo wrote, “Experience is the teacher of all things.”

This is certainly true in most times, including modern times. The propensity for the population to achieve awareness is proportional to the speed of information. In the distant past, information travels slow, often via horsebacks and words of mouth. Present days, information travels thru airwaves and landlines. Accordingly, the rise in spread of information is coupled with speed of gathering intelligence. A propagandist can more carefully and immediately gather the effect of his propaganda. This helps in many ways to evolve a propagandaist system.

gordo • May 2, 2019 6:38 PM

@ Dennis,

Accordingly, the rise in spread of information is coupled with speed of gathering intelligence. A propagandist can more carefully and immediately gather the effect of his propaganda. This helps in many ways to evolve a propagandaist system.

Yes, the scale, precision and efficacy of the real-time feedback loops of Modern ad networks provide a ready-made infrastructure for the application of propaganda.

Like the tobacco companies of the last century, who knew the dangers of their products long before they were admitted to and revealed to the general public, the ad platforms have more than a notion of the extent to which their systems can be, are and have been used for propaganda. Their studied experience in these matters is relevant to public discourse and policy.

Outraging, nudging, hardening and casting adrift the sentiments of the citizenry to make merchandise of them for political gain warps a society into conflicting states of polarization, paralysis and pathos. The longer this cancer spreads the less chance there is to stop it.

Nils • May 3, 2019 3:40 AM

I think Transparency is very important, and this is also something where government should and can do a lot more. The erosion of trust in institutions isn’t just a problem of foreign propaganda, it’s that many institutions have proven themselves to be less than trustworthy. If the government is lying or distorting the truth that’s exactly where those kernels of truths come from. “The end justifies the means” should not be modus operandi for the government or media.

Wesley Parish • May 4, 2019 6:44 AM

Well, nothing in this information operations kill chain article is of any surprise to me.

We saw it happen in the build-up to the 2003 invasion of Iraq; we saw it again in the intervention in Libya; we’re seeing it in the current build-up to a war with Iran (I don’t expect Bayt al-Saud (the House of Saud, the Saudi royal family) to survive Saudi outrage at the expected outcome of that projected war with Iran.)

We know quite well how the Roman Republic was beaten down, until it was merely an Empire with garnishings of Republic. And the maintenance of the patrician (wealthy bludgers) class was an essential part of that failing.

cui bono? is a useful check box for dealing with information/psychological warfare. “Who benefits?” Along with habeas corpus – “You must have the body – where is the evidence?”

gordo • May 7, 2019 11:40 PM

@ Bruce Schneier,

But Russian interference in the 2016 presidential election shows not just that such actions are possible but also that they’re surprisingly inexpensive to run. As these tactics continue to be democratized, more people will attempt them. Deterring them will require a different theory.

Let’s not fool ourselves. As Gareth Porter put it a couple of days ago in Truthout:

The [Mueller] report reaffirms the generally accepted view that the Internet Research Agency (IRA), the private St. Petersburg company owned by a businessman close to Putin, had a deep impact on pubic opinion through large-scale social media campaigns using false American personae. It repeats an estimate from Facebook that posts from IRA “trolls” may have reached as many as 126 million Americans – a figure The New York Times trumpeted as evidence of a Russian political coup in influencing the 2016 election by comparing it with the number of people who voted (139 million).

But that claim is a grotesque exaggeration of the IRA’s actual influence on the election. The original source of that Facebook statistic, Facebook General Counsel Colin Stretch, explained in his testimony that the figure was a calculation of how many people could potentially have gotten at least one IRA post in their Facebook feed over more than two years. Stretch further testified that the average Facebook user in the United States is served roughly 220 stories in News Feed each day. Facebook calculated that over the two-year period from 2015 to 2017, the Facebook posts from the IRA represented “about four-thousandths of one percent” of the content in News Feed, or approximately 1 out of 23,000 pieces of content” for those who did get some IRA feeds.

In fact, the troll farm’s influence was minute and its output was very crude and unsophisticated compared with that of the campaign of highly targeted social media ads carried out by the Trump campaign’s digital operation, “Project Alamo”. That unprecedented social media campaign was able to target individuals with ads based on information about their values and interest gleaned from a vast array of data sources. And Trump’s staff used the targeting to ensure that voting for Clinton among young liberal white voters and Blacks would be reduced.

That massive, data-driven campaign was complemented, moreover, by a huge new right-wing media system, led by Breitbart, that drove media coverage and mobilized tens of millions of pro-Trump voters with hyper-partisan stories – often “fake news – that helped consolidate Trump’s base.

Thus, Facebook users who were getting IRA content in their newsfeeds were overwhelmingly influenced by “Project Alamo” and the Breitbart-led media system – not by the Russian troll farm.

https://truthout.org/articles/mueller-stoked-trump-russia-alarmism-despite-finding-no-collusion/

To that (and though, disappointingly and for lack of a better phrase, ‘one’s mileage may vary‘), I would add an observation made last month by Noam Chomsky, in a pre-Mueller-Report interview on Democracy Now!, where he characterizes campaign funding as a “massive interference in elections”:

And, in fact, there’s no interference in elections that begins to compare with campaign funding. Remember that campaign funding alone gives you a very high prediction of electoral outcome. It’s, again, Tom Ferguson’s major work which has shown this very persuasively. That’s massive interference in elections. Anything the Russians might have done is going to be, you know, peanuts in comparison. As far as Trump collusion with the Russians, that was never going to amount to anything more than minor corruption, maybe building a Trump hotel in Red Square or something like that, but nothing of any significance.

https://www.democracynow.org/2019/4/18/chomsky_by_focusing_on_russia_democrats

One effect of the Mueller Report might be that it has rekindled an argument over the common political knowledge held by the American electorate regarding the extent of foreign influence in its elections. While I see nothing wrong with minimizing outside, foreign influence, keeping its scale and impact in perspective is a good place to start.

As you wrote, above:

Step 3: Wrap those narratives around kernels of truth. A core of fact helps the falsities spread.

[. . .]

Step 3: Wrap the narratives around kernels of truth. Defenses involve exposing the untruths and distortions, but this is also complicated to put into practice. Psychologists have demonstrated that an inadvertent effect of debunking a piece of fake news is to amplify the message of that debunked story. Hence, it is essential to replace the fake news with accurate narratives that counter the propaganda. That kernel of truth is part of a larger true narrative. We need to ensure that the true narrative is legitimized and promoted.

MarkH • May 8, 2019 2:11 AM

@gordo:

The part of Bruce’s argument you quoted makes the case that the attack is cheap, which is a fact regardless of its effect. The low cost will serve as an incentive to (a) repeat, (b) refine, and (c) scale up such attacks.

The truthout excerpt you kindly provided addresses only the identified Russian social media operation, which Mueller seemingly concluded was done without involvement by the Trump campaign.

The electoral effect of illegally accessing campaign data and publishing it was likely far greater. With respect to that operation, the Mueller report says only that it failed to find sufficient evidence to sustain charges of criminal conspiracy. It does NOT conclude that no such conspiracy took place.

Facebook’s average — perhaps self-serving — means little without considering the effects of targeting. The election outcome was tipped by 0.07% of all the persons who voted. It’s worthwhile to consider how many of THEM (and demographically similar folks) might have seen enough IRA material to influence their vote.

In any case, what we know of 2016 represents something like version 0.1 of this attack technique. We can be sure that greatly improved methods are ready for deployment.

P.S. Chomsky’s “minor corruption” was reportedly hoped by Trump to net him several hundred million dollars.

MarkH • May 8, 2019 3:54 AM

More about that Facebook statistic …

Colin Stretch, a lawyer for Facebook, testified:

Our best estimate is that approximately 126 million people may have been served one of their stories at some point during the two-year period. This equals about four-thousandths of 1% (0.004%) of content in news feed, or approximately one out of 23,000 pieces of content.

Remembering the Facebook was at great pains to minimize the US electoral impact of any foreign activity on their platform, let’s think about what Stretch said.

Confusingly, he speaks of people who “may have been served one of [the Internet Research Agency] stories.” Surely, there must have been people who were served more than one! Did he mean exactly one? At least one? Who knows?

Then he says “this equals …” as if the percentage could somehow be computed from the first statistic, though I don’t see any way that could be done. These seem to be essentially independent figures.

Anyway, let’s take the 1:23000 newsfeed number as a proportion of IRA stories.

There have been many reports that the Facebook algorithms are pretty accurately specific for general political leanings, so that a very small proportion of those ads would have reached people who wanted Clinton to win anyway. I’ll estimate from this that for the rest of the population, it would have been more like 1:15000.

Newsfeeds contain lots of stuff. Even during the election season, the great majority of stories would not be likely to strongly influence a voter’s presidential election decision. Many of them are news other than politics per se, or coverage of politics that doesn’t paint an especially stark picture of the desirability/undesirability of any candidate. If only one in fifteen newsfeed items were likely to cast favor/disfavor on a presidential candidate, then IRA content would now make up 1:1000 for people not already committed to Clinton.

I haven’t looked at a time breakdown, but it seems that Facebook arbitrarily chose 24 months (because it made their guilt seem smaller). But if IRA used any common sense, then they presumably would have devoted a lot of their attention to the last few weeks before the election, a particularly important time for voters who hadn’t yet committed. If IRA budgeted 20% of its messaging during that run-up to the general election, then IRA content would rise to about 1:300 of articles likely to favor/disfavor a candidate during that electorally important period, for people not already committed to Clinton.

Finally, considering targeting (not of the message contents, which were admittedly crude, but rather of audience selection). This could take into account factors which predict whether the Facebook user is likely to vote in the election, gender, age, geographic region, and more subtle kinds of information to select for the kind of anxious, dissatisfied voters who were trying to decide which candidate they preferred. If this targeting were reasonably effective, IRA posts might have made up (during that last few weeks) 1:50 of newsfeed articles, or perhaps even a higher proportion than that, of articles likely to influence electoral choice for persuadable voters in the weeks before the election.

Is it a “grotesque exaggeration” to suppose that many thousands of votes might have been influenced by these messages?

Probably the IRA didn’t make a significant effect. But it isn’t crazy to consider the possibility that it might have, or that such tactics might be many times more effective in a future election.

Clive Robinson • May 8, 2019 4:52 AM

@ gordo,

Noam Chomsky, in a pre-Mueller-Report interview on Democracy Now!, where he characterizes campaign funding as a “massive interference in elections”

It is and it’s something I might have also mentioned a few times over the years 😉

But what many don’t know is who we can thank for the unlimited spending?

Well it’s John Bolton poping up like the Devil’s advocate yet again. Some years ago there was a court case which John Bolton was one of the lawyer’s making argument for unlimited spending. He waxed lyrical about how in America financial largesse was directly equivalent to free speech…

Something wiser heads know is “complete baloney”, as it’s actually an argument that only those with sufficient money to spread by largesse that are entitled to have any voice in how thr country is run. Thus it’s an argument of actual oppression of the majority by a small minority. Something that subsequent history has shown to be both true and a very bad thing for society.

So it also runs “true to character” on John Bolton, who’s aim is to oppress as many people as possible prefereably by squandering the lives of as many US citizens outside his chosen pay masters as possible. In what is best described as “Orwellian Conflict” that would enrich not just the MIC but presumably John Bolton as well, just like any other bottom feeding shyster.

Mike • May 8, 2019 6:11 AM

@Clive Robinson,

This isn’t only about “campaign funding” from wealthy individuals, non-profits, and NGO’s. Additionally, these funds were channeled to the various politico organizations that serve to “mobilize” crowds and trickle more money down to these people to get things done that adhere to its agenda.

The fact that we learn of no “fake news” prior to breitbart is another complete fallacy. Most of the gesturing being talked about relating to social networks is just another example of applying censorship known as “taboo system” in other circles.

gordo • May 8, 2019 11:24 PM

@ MarkH,

Probably the IRA didn’t make a significant effect. But it isn’t crazy to consider the possibility that it might have, or that such tactics might be many times more effective in a future election.

Thank you for your thoughtful post. It’s too early in the season for the weed wacker, so I won’t/don’t want to get into that. We could go back and forth for a good long while, to no end, and each of us could come up with reasonable “what if’s” leaving us with nothing more than “narratives wrapped around kernels of truth”. In a word: conjecture.

These “kernels of truth” do not exist in a vacuum. When evaluating the various narratives these kernels have been wrapped in, sometimes the best we can do is locate them in the bigger picture.

An example of this is Chomsky’s throw-away line about Trump wanting to have a hotel in Moscow. In the context of political campaign funding corruption, it’s “a drop in the ocean”.

Porter does the same with the IRA. In the bigger picture, it’s “a drop in the ocean” of political campaign digital information operations.

As we begin “fighting the last war”, e.g., DDoSing and blackholing rogue disinformation operations, identifying coordinated inauthentic behavior, and passing legislation like the Honest Ads Act, my guess, looking at the bigger picture, is that the latter will be the hardest to enforce because “That’s where the money is.”

MarkH • May 9, 2019 2:56 AM

@gordo:

I pay acute attention to “intellectual labor saving devices” which are logically invalid, and therefore frequently lead to false conclusions.

One such heap of rubbish masquerading as reason is what is now often called “whataboutism.”

An example goes like this: it doesn’t matter that I’m burgling your home, because the banks are stealing from everybody. Well, that’s wrong. The burglary does matter.

All systems of popular election are necessarily imperfect, and in general the potential for powerful actors to distort outcomes is very grave. In the U.S., intentionally weak regulation makes this problem particularly severe.

I can acknowledge that this is a difficult and dangerous problem, and at the same time insist that covert intervention by hostile sovereigns is a distinct problem of enormous import.

To say that we should ignore the burglary because banks are unfair is nonsense. To say that there was no “collusion” because the Special Counsel failed to find enough evidence to prove criminal conspiracy is completely invalid. To cite a statistic concocted by Facebook in a transparent and dishonest attempt at self-exoneration, as proof that IRA was ineffectual — which is what the truthout author seems to have done — is utterly irresponsible.

In recent years, many crucial elections outside of Russia have had outcomes shocking to a huge proportion of the population, which happened to be exactly as Putin desired.

Brexit and Trump are the most famous examples, but Ukraine was a pioneer when Putin’s despised stooge Yanukovich won the presidency there, with the help of Paul Manafort.

These Kremlin-friendly results have flowed both from luck, and the fruits of extensive Russian political operations … in what proportion, is very difficult to ascertain.

Anyone who wishes to, is free to fantasize that Kremlin electoral intervention is negligible. Pay no attention, to that man behind the curtain!

Clive Robinson • May 9, 2019 12:47 PM

@ Mike,

The fact that we learn of no “fake news” prior to breitbart is another complete fallacy.

Yes and no, “fake news” as a form of propaganda has been around at least as long as “Chinese Whispers”. What has changed is the medium it is dispersed with.

Mad as it might sound the Internet can be considered like “Chinese Whispers” with “10MW of sound power” reaching around the globe.

Whilst anybody can “tune in” very very few do so.

Whilst “Preaching to the Choir” “keeps the faithfull happy” the echo effect makes things not just more polarized it makes things more extream. To use the old “super highway” analogy people generally don’t jump into fast moving traffic, the tend to use on ramps.

Which means if an echo chamber or it’s choir want’s to survive it needs to grow for various attrition reasons at the very least. Which means they need a “feeder mechanism” to in effect “groom the new acolytes into the new faithfull”.

It is these softly softly “on ramps” that make or break echo chambers and groom the potentials to the point they are now at the very least “useful idiots” acting almost as on ramps themselves, before the “Director of the choir” basically fleeces them of some resource be it, time, money or ultimately their vote.

These tactics are little different in style to the processes that have been used by cults for decades if not centuries. Again the difference is the medium and it’s reach.

In most cases the easiest way for a cult to recruit is to promise a journey to a destination the potential member desires, the trick is to make the journey last long enough to achive the objective of the director / leader well before the members desired outcome occurs…

The interesting thing with elections is that once the members have voted, it does not matter what the outcome is, because the director leader has another election to build up to. If the desired outcome was not achived the leader can blaim the voters and thus the members for not striving hard enough, if however the desired outcome is achived the leader can exhort the membership to try harder because there is always the “enemy in the wings”. So either way the manipulation continues endlessly…

How you actually break the cycle is remove the “on ramp” and wait for attrition to run it’s course. In all the bruhar I see people wanting to bring down the church the choir meet in, that is a pointless corse of action because all it will cause at best is the choir to meet somewhere else, or worse fracture and creat many more new choirs.

gordo • May 9, 2019 5:12 PM

@ MarkH,

a statistic concocted by Facebook

Thus, the 126 million number is meaningless.

MarkH • May 10, 2019 12:32 AM

@gordo:

“Thus, the 126 million number is meaningless.”

NO!!! But I thank you, for a fine example of defective reasoning.

I don’t have independent data. I think it likely that both numbers — neither of which can apparently be derived from the other — are reasonably accurate. The Facebook lawyer probably didn’t want to lie in testimony, which would be a very serious offense.

Instead, he presumably told some truth, in a manner calculated to be misleading and deceptive.

I don’t see any logical connection between (a) 1:23000 is intentionally misleading, and (b) the 126 million number is meaningless.

gordo • May 10, 2019 7:10 PM

@ MarkH

By concealing the details of Facebook’s operational analysis of the IRA ad campaign in their reporting, the NYT and Office of Special Counsel have distorted the meaning of the 126 million number.

By omission they’ve created an alternate narrative that may leave many in their audiences believing that over 90% of the people who voted in the 2016 U.S. Presidential election were influenced by IRA ads.

Why would the NYT and Office of Special Counsel do that?

Regarding “a statistic concocted by Facebook”, since you seem to have walked yourself back from that as applying to the 126 million number, my comment is moot.

MarkH • May 11, 2019 3:16 PM

@gordo:

I imagined it would be clear from context, that the number I referred to as “concocted” was the proportion of newsfeed items. In the future, I’ll try to be more clear in my writing.

As to 126,000,000 … this is from Facebook’s own lawyer. However the numbers were derived, Facebook’s obvious incentive was to minimize their role in a ghastly conspiracy. Perhaps for one reason or another, they overstated it.

The report from the Office of Special Counsel, which by design hews closely and conservatively to reliable evidence, says that Russia’s social media operations to covertly influence the 2016 election “reached tens of millions of U.S. persons.”

Mike • May 12, 2019 2:50 AM

@Clive Robinson wrote, “How you actually break the cycle is remove the “on ramp” and wait for attrition to run it’s course. In all the bruhar I see people wanting to bring down the church the choir meet in, that is a pointless corse of action because all it will cause at best is the choir to meet somewhere else, or worse fracture and creat many more new choirs.”

Don’t bring down the venue is certainly in the right direction of thinking, but it doesn’t operate this way in reality. For example, what happens to a ponzi scheme when discovered? It will be brought down by the feds and people will get arrested. However, the scheme will continue in another way shape or form as the choir will rise again and be fleeced. This is not to say that don’t bust the exposed scheme.

The US of A is adapt at doing this type of things because we (they?) are proponents of regime changes. When was the last time we the citizens get to vote on a foreign regime change perpetrated by US of A ? The vicious cycle continues not only in its domain but its works like a ripple flowing and far reaching with no recourse.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.