The Technocratic Oath

mdSilicon Valley moves fast. A major reason for the success of technology companies in the Valley is speed. The combination of talent, funding and experience found in the region gives many startups significant first-mover advantage as well as the ability to scale quickly.

And importantly, there is a pervasive obsession with speed. Companies launch “minimum viable products” that will “fail fast”. Facebook auspiciously encouraged staff to “move fast and break things”. The most important imperative is to get the product to market, receive feedback, fix bugs, and iterate.

This approach works great when you are launching, for example, software to link your phone to your car stereo.

But what about the major platforms that at current scale have major societal impact? What if “breaking things” represents major social disruption? It’s great if Facebook supports small businesses and enables photo sharing, but what if it also facilitates election tampering, hate speech, or in the case of Myanmar, genocide?

As Tom Wheeler describes in his latest book From Gutenberg to Google: The History of our Future, the greatest impact of new technologies generally isn’t their direct influence, but secondary and tertiary influences. Gutenberg knew that the printing press would facilitate communications, but didn’t know that it would facilitate the Protestant Reformation.

So how should technology companies, with their growing societal influence, think about their responsibilities?

Actually we know the answer: Primum non nocere.

Every medical student learns the Hippocratic Oath, an important part of which invokes “First, do no harm”. From day one students are taught that innovation in medicine needs to be coupled with responsibility for the well-being of the patient.

Thank goodness it is such. Could we imagine, for example, a major pharmaceutical company launching a new cancer drug that only worried about a “minimum viable product”? Aspired to “fail fast”? Sought to “move fast and break things”? It’s unthinkable. If a new ointment cures your rash but makes your skin fall off as a side effect, that isn’t good. Physicians understand that.

As such, medicine has both a culture and approach of considering secondary and tertiary effects of innovation at every step. Products are tested rigorously for side effects. The government spends billions monitoring drug safety. If problems are discovered, drugs or products are immediately recalled.

It is now apparent that technology innovation needs to be equally cautious. Sometimes the stakes are low: if my phone doesn’t link effectively to my car stereo, it’s not the end of the world. But when companies reach the scale and influence where they can facilitate crime, abet violence, or undermine democracy, the secondary and tertiary impact of their products absolutely need to be considered at every stage. This impact needs to be monitored. And just as with a rogue drug, dangerous technology innovations need to be recalled.

It’s time for the technology sector to take a page from medicine and agree to a “Technocratic Oath”.

What should it be?

For starters, it should include the words first penned twenty five centuries ago: “First do no harm.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s