How I evolve consequence scanning for better ethical decision making

This post is for teams committed to building stuff that does good. Let me know how you go with the approach.

A lot of folks in and around this space (‘tech ethics’) are familiar with the work Dot Everyone published a few years ago on Consequence Scanning. The approach has become fairly popular, likely for good reason (my claim here is that, for the most part, few teams engaged in practices like this early and often. The result has been something like plenty of intended consequences that are far worse than some may have hoped, along with plenty of unintended consequences that, with more deliberate effort, could have been better mitigated. By formally adding consequence scanning these teams are better places to do more good and less bad. Grossly oversimplified, but roll with it).

With that said, one could argue that risk and business continuity professionals have been doing something very similar for quite some time (forecasting or ‘simulating’ possible outcomes - both intended and unintended - in an attempt to enhance the likelihood of good and decrease the likelihood of bad). I learned this long ago from my Dad, who in a former life was pretty darn good at this stuff (keep that between us 😉).

So, let’s explore how I have evolved and use Consequence Scanning (opens a short video on LinkedIn that covers the basics) in the context of the systems for ethical decision making I design and help implement.

It’s important to note upfront that Consequence Scanning exists in the broader context of the system (inputs, throughputs and outputs). It’s both a tool and process that is called upon to support certain decision-making workflows (information about it exists in the Knowledgebase and Toolkit. Some of the outputs feature on the Key Decision Log etc.). That isn’t to say it can’t be one of the only things you do, or doesn’t have independent value (if you’re very early in the process). I believe it does. This is more about accurately situating this work for me.

To quickly re-cap, decision making processes within an ethical decision making system can be: 

1. Proactive as part of doing something new or making a change (this is most likely the context within which Consequence Scanning will prominently feature)

2. Reactive as part of a triggering event that requires a more formal process to be undertaken

3. Proactive as part everyday work (issue backlog grooming workflow etc.)

4. Retrospective or retroactive as part of a variety of potential workflows the system describes

The general sequence is (proactive example here):

1. Something new is identified. This can be a new initiative (i.e. we want to build X) or an ‘ethical issue’ (something identified that requires attention)

2. The parties responsible for the decision-making process review existing evidence in the form of the Key Decision Log (have we explored an issue like this in the past?) and Knowledgebase (what theories might help us?). This acts to both speed up, and ensure consistency of ethically oriented decisions

3. If there is clear precedent, the decision will be made (or, depending on structure, advice will be give, which might lead to a business decision), added to the Key Decision Log and used to inform future business activities

4. If significant uncertainty remains, with answers not found in existing resources, additional work is done

5. If the core team is party to the decision, the next step is likely an internal process using a decision-making template (Consequence Scanning can be one of them). This template will draw on fit for purpose decision making tools (defined in the Knowledgebase and Toolkit)

5.1 A tool will be selected to support the decision. If decision clarity emerges from this structured process (say Consequence Scanning combined with a principles assessment), the decision is logged in the Key Decision Log. Evidence of the process accompanies the decision and anything new and/or useful is added to the Knowledgebase. 

5.2 If decision clarity is not reached, the issue is added to the Issue Backlog.

6. If the core team is not party to the decision, the issue might be flagged (based on weightings, consistent criteria etc.) for review by the core team as part of their everyday workflows (this might be a team comprised with specific expertise, including moral analysis and applied ethics in the relevant context)

7. As part of the core team’s everyday workflows, backlog grooming helps prioritise unresolved issues

8. Unresolved issues given priority can be flagged for Social Preferability Experiments (or other ‘augmenting’ workflows)

9. Social Preferability Experiments are then defined, which forms a clear picture of the requirements (work to be done, type of cohort that needs to be recruited for participation, pass/fail parameters etc.)

10. The Social Preferability Experiments are then conducted

11. The results are analysed 

11.1 If the result is a pass, the issue is resolved, the decision is logged and the data from the experiment is added to both the Key Decision Log and Knowledgebase

11.2 If the result is a fail, the issue circulates back through the same issue grooming process and will likely inform another experiment (unless this is expedited due to the issue remaining a priority. If this is the case, the learnings from the previous SPE can inform another SPE, which then follows the same process in terms of design, recruitment, running, analysis etc.)

12. The decision that’s been inclusively made and logged to the key decision log informs specific future actions

13. Then the cycle repeats, continuously, forever…

This is a super high level overview of how the system for ethical decision making might operate. All further details aside, let’s now focus on Consequence Scanning.

To start with, you want to recruit the right ‘team’ to engage in this process. This will depend on the context of course and there are different ways to do this. Ideally the more diverse skills and life experiences the better (important note: This can also be done with folks outside of your business. You can run this same approach with a representative sample of your customers for instance).

You’ll also want to ensure that the framing is clear. Know why you are there, what your focus is etc.

Once you’re in the room (can be co-located, virtual or hybrid), the basic process (outside of actual facilitation, which is SUPER important and something I’ll cover in another post):

1. Map the consequences based on their 'strength' (magnitude, effect size etc.) along the spectrum (from -10 to 10. The size of the circle can also help visually represent this)

2. Describe the consequence in a more qualitative sense (after all, everything is consequential, so details matter)

3. Specify whether it is intended or unintended

4. Assign a likelihood to each consequence (score from 1 - 10)

5. Do the math and viola! You have a Net Consequence Score (you'll also have a really wonderful visual representation of the work)

From here you can work to identify amplification and mitigation tactics (which could be during the same session, or another. Practical constraints will govern this). These can be added to a backlog, weighted based on certain values, prioritised and then acted upon via whatever workflow you're comfortable with.

The important stuff to note here, that doesn’t usually feature in consequence scanning, is:

1. The specific scoring system (both direction and strength of consequences)

2. The likelihood score

Taken together, these help give you a net consequence score. This net consequence score can be used to inform decisions about what is good and right. The score can feature prominently, when combined with other decision making practices (more robust moral analysis, an assessment of how a given project aligns to your purpose, values and principles etc.), in whether you choose to do something or not. And of course, if you do choose to do it, this will help you figure out how you want to do it.

None of this is perfect. But it is the type of process that, I’dd argue from direct experience, can be fun, a great learning opportunity and incredibly useful for all of those committed to building tech for the purpose of doing more good.

Happy scanning folks!