Ethical Problems in Computer Security

Tadayoshi Kohno, Yasemin Acar, and Wulf Loh wrote excellent paper on ethical thinking within the computer security community: “Ethical Frameworks and Computer Security Trolley Problems: Foundations for Conversation“:

Abstract: The computer security research community regularly tackles ethical questions. The field of ethics / moral philosophy has for centuries considered what it means to be “morally good” or at least “morally allowed / acceptable.” Among philosophy’s contributions are (1) frameworks for evaluating the morality of actions—including the well-established consequentialist and deontological frameworks—and (2) scenarios (like trolley problems) featuring moral dilemmas that can facilitate discussion about and intellectual inquiry into different perspectives on moral reasoning and decision-making. In a classic trolley problem, consequentialist and deontological analyses may render different opinions. In this research, we explicitly make and explore connections between moral questions in computer security research and ethics / moral philosophy through the creation and analysis of trolley problem-like computer security-themed moral dilemmas and, in doing so, we seek to contribute to conversations among security researchers about the morality of security research-related decisions. We explicitly do not seek to define what is morally right or wrong, nor do we argue for one framework over another. Indeed, the consequentialist and deontological frameworks that we center, in addition to coming to different conclusions for our scenarios, have significant limitations. Instead, by offering our scenarios and by comparing two different approaches to ethics, we strive to contribute to how the computer security research field considers and converses about ethical questions, especially when there are different perspectives on what is morally right or acceptable. Our vision is for this work to be broadly useful to the computer security community, including to researchers as they embark on (or choose not to embark on), conduct, and write about their research, to program committees as they evaluate submissions, and to educators as they teach about computer security and ethics.

The paper will be presented at USENIX Security.

Tags: academic papers, computer security, ethics

Posted on June 21, 2023 at 1:54 PM • 21 Comments

Comments

You don't say... • June 21, 2023 2:09 PM

At the heart of EVERY “ethical problem” are corrupt humans. It is not okay to be optimistic and naive by claiming that “most humans are okay”. Reality is very different unless one lives in a bubble of lies and ignorance. To remedy this is practically impossible as it would require either complete transparency for e v e r y o n e – no exceptions whatsoever, or complete privacy and anonymity for e v e r y o n e. UTOPIA.

Vesselin Bontchev • June 21, 2023 2:39 PM

No solutions but for your amusement, here is a paper of mine about “Contemporary Cyber-Security Trends and Related Ethical Dilemmas”, presented at an ethics conference 4 years ago:

Part 1
Part 2

There was no ChatGPT back then, so AI-related stuff is not covered.

David Leppik • June 21, 2023 2:48 PM

The problem with hypothetical “Trolley Problems” is that they don’t account for how real people make decisions: using unreliable information. In real life, even if a situation appears as clean-cut as the Trolley Problem, there’s a good chance it isn’t. Harming one person to save two people is morally dubious because the harm is a clear, deliberate, and avoidable outcome; the positive impact is more dubious.

Similarly, immoral behavior is often justified by a “ticking bomb” situation. In that scenario, if you don’t succumb to torture or immoral threats, you won’t find a bomb that’s about to go off, causing much greater harm. The problem is: there’s always a ticking bomb. Or, at least, there’s always someone who’s convinced that there’s a ticking bomb. You can’t have an ethical police force if it breaks the rules of ethics for a perceived greater good.

What’s more, bad actors can manipulate a person who is willing to make ethical compromises for the greater good. By spreading misinformation or making threats, they can create a ticking bomb or trolley situation—that’s the whole point of hostage-taking. Inflexibility is a powerful bargaining tool.

Computer security is riddled with all of these issues, since threats are easy to make (e.g. ransomware) and data is easy to falsify.

Winter • June 21, 2023 4:08 PM

@you don’t say

It is not okay to be optimistic and naive by claiming that “most humans are okay”.

If most humans were not OK, there would be no society, no trade, and no justice at all. Alligators do not have a social life.

At the heart of EVERY “ethical problem” are corrupt humans.

Where are the corrupt humans in the trolley problem?

vas pup • June 21, 2023 5:12 PM

As I see it with ANY choice we made in our life it is not pure good option versus pure bad option as we taught.

It is almost always one choice out of two or more options with less evil.

That is universal in any field. How can you really set up the criteria to compare options for less evil? E.g.Freedom fighters versus rebels/terrorists? Goals? Means? War for independence of US states?

Unfortunately, violence is often part. What is the point to have sentences of more than one life in prison? Combine sentences up to 100 or more years?
That just absolutely unreasonable and stupid.

If I have clear answer I’ll be somewhere very high on food chain.

Anonymous • June 21, 2023 5:29 PM

This is a good paper, and I am glad that you selected it.

increase the airgaps • June 21, 2023 6:07 PM

What happens if I get my entire life’s work stolen and and my freedoms ignored and ejected by unknowns seeking nonexistent contraband within my possession, meanwhile a local community college quietly teaches network pennetration testing yet goes unmentioned as a pool of suspects and their alleged decades long list of bomb threats goes unmentioned as well?

How is anybody supposed to get any traction towards stability?

I’m invited to use the latest trendy tools for peaceful playtime uses, and yet when I do I get attacked like I’m digital vermin.
Meanwhile so many hoards of suspicious hostile and their damaging effects show up so much it’s got me thinking that:

“Black Operations must mean embarrassing activities that were supposed to be impossible and forbidden and unlikely, yet happened anyway. This is a specific difference instead of requested or selected covert missions to accomplish some kind of noble goal; that must be something else entirely”.

I shouldn’t have to be learning these things.
Usually, there’s nobody trustworthy to tell by the time the info is stacking up inside me as PTSD ironies.

Clive Robinson • June 21, 2023 11:43 PM

@ vas pup,

Re : You might make office 😉

“If I have clear answer I’ll be somewhere very high on food chain.”

Yup, and as far removed from ethical dilemma as is possible (they have special assistants to give as a minimum “arns length” ignorance).

A few years ago in the UK we had the “Cash for Questions” scandle where “elected representatives” called “Members of Parliament”(MP) were openly asking for the then equivalent of $7,000-10,000 to ask a single questions in Parliament…

And after a couple of sting operations that got televised it became blatently obvious there was a problem. As with spotting a cockroach, you know when you look more carefully there will be hundreds of them hiding out of sight and trying to stay in the dark.

Well it rapidly turned into an “expenses scandle” as well and the more people looked the more there was found. Most Members of Parliament who got caught got away with it by claiming it was a missunderstanding of the rules by their assistants etc and “paying it back”[1].

But others could not as “their boots were overflowing”. But few MP’s had any real sanctions against them.

But a very well known journalist did some simple maths and concluded that a UK MP was four times more likely to become a convicted criminal than an average UK citizen…

Which begs the question that with well under a thousand MP’s and four times the national average found to be criminals, just what percentage were basically crooks that had managed to avoid being charged, taken to court and convicted?

If we were to apply the usual rules for the average citizen that MP’s had avoided… It works out that MP’s were at that time, over 50times more likely to be crooks than average citizens…

And the UK was untill then supposed to be a shining example of honest democracy at work. A shining beacon of hope and probity in a world of sleaze, corruption and bribery…

[1] There was one, where it was probably an honest mistake. I can’t remember all the details but briefly it involved a small packet of biscuits purchased instead of a “meal” when out from the office on MP business. Apparently a slap up five course at an expensive London restaurant costing hundreds would be OK for an MP to put on expenses… But a quickly grabbed packet of biscuits[2] from a shop payed for with “pocket change” was not…

[2] Biscuits occupy an odd position in food stuffs in the UK as far as VAT is concerned. Fresh food and food for making a meal from are, exempt, as were cakes even if coated in chocolate back last century, but sweets such as chocolate were not. Biscuits were in some strange place in between, when they have chocolate on them they are not exempt from VAT. There was a famous case brought by the then “UK Customs and Excise” against the manufacturer of Jaffa Cakes in the early 1990’s… The tribunal went on for an extended period of time, but it was argued that though round, small, and biscuit like, they were in fact a cake and thus exempt… Thus this strange status for biscuits gets carried into company and other expenses. I’m not sure what the rules are these days because I don’t eat them any more, but at one time if you had biscuits with a cup of tea or other “cafe style” beverage –but not hot chocolate–, they were allowable otherwise not.

flowers and first party sets • June 22, 2023 12:22 AM

strangely related to cayenne allergies, jews, electronics, korea, and developing industrys TRUE history from the 80s, found in wwwsk. K=kim.

really.

Ted • June 22, 2023 2:21 AM

The three primary scenarios in the paper each offer a piquant dilemma.

Scenario B asks us to think about how we might handle stolen data. Here a company’s AI hiring system – with its sensitive applicant data and suspected biases – has been leaked.

Two different ethical frameworks (consequentialist and deontological) lead to opposite conclusions on whether or not to pursue research on the data.

These frameworks – and others, including the oft-referenced Menlo Report – are laid forth as structured approaches for evaluating ethical decision-making.

On a side note, if anyone has any ethics-related computer security experiences and outcomes they’d like to share, the researchers are working to create a repository:

https://securityethics.cs.washington.edu

Anonymous • June 22, 2023 9:38 AM

When I weigh out all the comments and what Bruce has said.. to me it strikes me as quite obvious that “cyber security” is a bully pimp game of sorts…justlike all war is too… i.e. keep exaggerating real threats, keep enough perceived threat to come out there , and you got a great business model.

The millions of dollars the blog owner has “earned” and earned reputation to be secure in his travels, etc… is a consequence.. regardless of how much downplay silver tongue trickery goes on with blasting consequentialism…

there are some brave souls/people who get themselves killed really quickly for such activism, be at my Uncle Sam’s “castor bean” program or many other similar programs such as the domestic targeted individual program by the council and foreign relations etc etc

To the supposed security specialists or those claiming to ne on they have long-term careers and setting up several Managed IT firms and Consulting with the government which I know Bruce does a lot of and several others Krebs Etc… and then advocate for policy changes to mitigate the appearance of privacy rights…and so forth *(which is not a bad thing) but at the end of the day …fear mongering and exploitation in an environment of todays zeitgeist… where other people are motivated almost exclusively by double speak, you cannot blame a blind man for being blind and you cannot blame a confirmation bias individual for falling as confirmation bias and you cannot blame “idiot security Specialists” out there perpetuating the problem so that they can , however falsely,experience their personality expression and sense of contribution to humanity and altruism and ethic.. when in reality, it is egotism… not that anybody really needs that.. and off topic here….more so imho.. God doesn’t need any favors from anybody any…

to those security specialist that enjoy playing games with relativistic semantics and downplaying consequentialism may they enjoy the consequences of their own actions… why not offer side of ranch dressing in a prophylactic everybody too?

Pigeon • June 22, 2023 9:41 AM

Test

Keith Douglas • June 23, 2023 9:38 AM

Great to see this – I am going to be giving a workshop on ethics in our profession to my coworkers soon and this will help.

vas pup • June 23, 2023 3:48 PM

@Clive
Thank you for responding to my post.

I have my own moral task to discuss:
Two close friends climbing mounting being attached to each other with rope.
One fall down. Now the second have two options: cut the rope and save himself – no chance both could survive, or fall down and die together.

When option 1 selected how second could live thereafter. Looks like moral choice should incorporate time dimension as well.

Clive Robinson • June 23, 2023 10:37 PM

@ vas pup,

“I have my own moral task to discuss”

Mostly people climbing together are well equiped/prepared and importantly friends. Which in my experience several decades ago ment not only that they trust each other, they react instinctively.

As I’ve mentioned in the past I did go bouncing down Glen coe’s devil’s staircase, but that was “walking” not “climbing” so equipment was way different and we were not roped together, and my companions quickly scrambled after me in an organised fashion.

Yes I’ve had incidents climbing, I was one of the first people in the UK to regularly “freeform” on sandstone which you can not use normal climbing kit on (the sticky shoes they have these days are worlds ahead of the plimsoles I used as are the specialised gloves).

I’ve never had to think ethically or morally whilst up a mountain, because of “instinct and training”. Basically it just kicks in and if it goes wrong… Well you’d done your best. I’ve never had slow drag or similar issues because I’m generally “the largest lump moving on the mountain” and I dug in. The problem these days is politicians via the police have made life less safe for climbers in the UK. To many “scuppers” treat climbing axes/hammers as the equivalent of swords or machetes, thus you can not travel with them “on your bergan” as you used to do (as far as I’m aware there never has been an injury let alone a death caused maliciously by a climber in the UK with one).

As I’ve indicated in the past I don’t believe in “acts of deities” and have said there is no such thing as an accident, only too little information/knowledge or time to respond.

At human scale the level of randomness in the movment of inanimate objects is too small to have any real difference to an objects movments.

As we actually know, as we have the technology deployed to do it, you can actually shoot bullets out of the sky with other bullets… Likewise bombs and mortar rounds which is what the technology was first designed for. Even missiles on ballistic paths, but it’s not so easy with all of them not just due to the higher speed causing lack of time, but also some can manover over less than predictable paths that you don’t have time to correct for.

JonKnowsNothing • June 24, 2023 10:03 AM

@ vas pup, @Clive, All

re: Climber’s Dilemma

This situation happens on high altitude climbs, as well as some smaller climbs or wilderness adventures.

The easiest one to envision are the Death Zone climbs such as Everest. Each climber knows (or should understand) that at that point of the climb, it is a Point of No Return.

The photo op – step back, step off the side of the mountain
The climber overcome by altitude sickness, unable to move on their own is left on the side of the path (1)
The climber who is dying from altitude and exposure, has their oxygen tank taken and given to another climber

Anecdote stories of these choices are some that most of us are unwilling to undertake unless we have no other options; then we take whatever comes first. In theory if you are a well trained and conditioned for such extremes, (police, fire, first responders) you can do all the Triage in your head and act accordingly.

One aspect of COVID-19 is that all of these happened every day 2019-2022. No oxygen, no treatment, limited beds, denied care. Doctors and health professionals all over the globe faced these conditions. The morgue trucks are reminders of what took place.

Can you determine if you made the “best choice”?

You will never know

===

1) The Sherpas do their best to bring down the dead as well as the trash from hundreds of oxygen tanks discarded on the mountain. It can take a while, because the conditions are so extreme that any additional efforts puts the Sherpa’s life at risk too. So, they drag these items down the mountain a few feet at a time over the season.

The other aspect that’s not much considered, is that the climbing path and schedule of attempts is fully booked. There are hundreds of climbers attempting the top wanting to experience the very short time they can be there and still get back to base camp alive. The path has a long queue of climbers and they all have to pass by what’s left by the side of the path.

They may not have made the direct choice themselves, but each of them made a choice.

Phillip • June 24, 2023 2:46 PM

Counterfactual thinking may be mature enough to improve the cause of ethics research. Specifically, Constructor Theory.

An introductory book, “The Science of Can and Can’t” (2022, Chiara Marletto), seems accessible enough. I caution against believing any short synopsis of the topic may really inform – were you to even excuse my own characterization of the subject as transcendent, or what have you?

Clive Robinson • June 24, 2023 9:31 PM

@ JonKnowsNothing, vas pup,

Re : “This is the mission” types.

“Each climber knows (or should understand) that at that point of the climb, it is a Point of No Return.”

To me climbing was a hobby, fun, ok some personal challenge but always a “group activity”. Where the “group always came first” as aby good leader should know. It was never “a suicide mission”[1] of personal ambition.

Call them “driven types” or “goal oriented” or similar but the first thing to note is “they don’t have friends” they have at best “mission associates” and they actually only care about their own wants.

My advice from experience is “turn and walk away” because they will dump you as soon as they can nolonger use you. Or worse they will have planned from the very begining to “throw you under the bus” when it comes along as part of a pile to lift them up.

It’s part of the “strong man” nonsense and for those that have not noticed, it’s currently playing out at just beyond the East of Europe[2].

Remember the statment attributed to Paton About “Dumb 13ast4rds”?

Well it was better said back in the early 1880’s… Back in the 1980’s a photographer friend and survivor of the Falkland’s campaign made a poster and put the words,

blockquote>“The muster-roll of the dead may be a monument of governmental incapacity as well as a certificate of patriotism and courage. It is always glorious for the other man to die for his country, at least the lone survivor will tell you so. But the fact that a soldier and his life have been needlessly squandered is calculated to throw much doubt on the subject. It is for all to know, ‘A civilized nation cannot afford to throw away a single life’.”

blockquote>

Put over a picture of Falkland Sound taken when it was at it’s worst. I have a copy framed in my home, to remind me, not just of times past, but how to ensure there is a future for others.

[1] Yes there are times when “suicide missions” become necessary, but they are almost always because of someone being “incompetent” in managing an earlier set of events. Sadly if the incompetents are in your societal grouping / tribe they want to use you to make them “look good” to someone higher up the food chain… I saw some of this crap back years ago, there is nothing brave or noble about such “I’m a hero” types they are the very essence of “users and abusers”, and almost always the first to claim “we are the good guys”. In any given situation they almost always make the situation worse, because they chose not see beyond their own wants. So making it worse is part of their goal, such that they think you will make them look more heroic when you somehow solve the mess they created for them.

[2] The “incompetents” for this were the UK and US Governments. That negotiated a “demilitarization” of the Ukraine with respect to Russia. In return the UK/US promised to come to the defence of the Ukraine should Russia invade… As history shows both the UK and US repeatedly failed to honour it’s commitments. So the inevitable happened as centuries of history said it would. There is a stange belief that taking the toys away from squabbling children will stop them squabbling, practical evidence suggests that it won’t and hostilities will be at best be minimally deferred.

Clive Robinson • June 24, 2023 11:03 PM

@ Phillip,

I assume by “Constructor Theory” you are refering to the more recent work of “The theoretical Father of Quantum Computing” and proponent of the many-worlds interpretation of quantum mechanics David Deutsch?

That trys to explain “Why things don’t happen” rather than the normal science theories of “Why things happen”. Which tries to unravel at a more basic level what we currently cover up with “statisticaly improbable”[1].

Interestingly it also gets around the “Many Worlds” and other issues that have ariseb with Quantum Computing[2].

Both “New Scientist” and “Scientific America” have reasonabley readable lay person overviews. Which can be said in more space than this thread alows (hence my brief overview[3] that may appear incomprehensible ;-).

https://www.scientificamerican.com/article/a-meta-law-to-rule-them-all-physicists-devise-a-theory-of-everything/

[1] You pour “a splash” of milk in your tea and it spreads through the tea untill it reaches a near uniform density in it. There is no reason to assume it is impossible that the milk could not reform into the original “splash” we just never see it. Currently in “above the quantum divide” in physics, and in Information Theory, we talk about statistical behaviours in thermodynamics like Brownian Motion to explain that the probability of the splash of milk reforming is unlikely to occure through the life time of the universe[2].

[2] Hence “entropy” or “Moving from a coherant / organised state to an incoherent / disorganised state” That is near uniform in distribution which in physical terms minimizes the ability to extract work. But in terms of “information” increases it’s utility as it increases the number of possible states[3].

[3] The information growth of incoherent / disorganised physical objects, can be viewed in one of two basic ways.

Firstly by the number of states which is what we normally do hence the log2 nature of H which Claude Shannon originated (back around WWII on his thinking about cipher systems).

But secondly by the relations between the states of physical objects in terms of the combination relationships and their vector values which is the meta-information or “data shadows” view. Which gives a vastly increased potentially infinite information capacity[4] (which is one of the interesting things with both Quantum Computing and Neural Networks).

[4] Think of the non tangible distances information between tangible physical objects can encode information in theory infinately, like “reals between natural numbers”. But whilst they can be measured by physical means, the measurments are usually not fundementaly physical. That is they are arbitary, and expressed as ratios or fractions thus infinately scalable. Which amoungst many other things gives rise to the interesting observation of why 1/3 is more accurate than 0.333.. recuring when represented in a displayable form based around decimal presentation. That spills over into how we process, store and communicate information in computers and in part why we may never be able to get useful information from quantum computers or fundementaly explain how information is stored in large but finite neural networks.

Phillip • June 25, 2023 11:32 AM

@Clive Robinson: Thank you for the SA reference. Yes, the copy property is elucidated near the front of Marletto’s book. Though light on physics, much of it goes over what seems like (for me at least) a novel approach to sentential logic. It may sort of work like this: suppose one has fallen short of figuring out something less than unsurmountable. Then, one can sort of elastically stretch the scope of one’s own introspection by playing with absurdities (i.e., what can or cannot be literally true). Call it edge cases, if one desires. Though not really. It is more like “free association” used in psychology, coupled with testing whether an absurd sentence might be diagrammatically pieced apart with other, less ridiculous, statements. Yes, one can dismiss this as entertainment for a bored mind, though it seemed to bear fruit much more often than one might expect. And when something did prove impossible, knowing it enabled my focus to be redirected, or switched, with dispatch, i.e., without continuing by wasting “compute cycles”. Moreover, when something proved impossible, it might take a little much effort to reframe it. What is an experiment which might not continue endlessly? A kind of experiment we can prefer. To “give it a rest” might best be giving it a real rest. Extra mile kind of rest.

name.withheld.for.obvious.reasons • June 27, 2023 7:31 PM

What does ethics have to do with security and/or computing?

Without creating a parametric or an analog equivalent of real decisions in meatspace, my favorite example is EULA’s. A masters class in subterfuge and doublespeak and down right hypocritical demands in a contract masquerading as warranty instrument, which it is not.

I would suggest that a less subjective measure in the realm of ethics should start a little lower in the social or commercial chain. More and more, individuals are commodified within the context of a buyer/seller/owner/leaser relationship down to simple transactions such as the purchase of a coin battery.

Schneier on Security

Ethical Problems in Computer Security

Comments

Leave a comment Cancel reply