How many times you, as a software expert, saw some software or process and thought “damn, this can be done so much better”. Yes, a lot. But why, since large organizations spend a lot of money on IT? Is it because software is too complex, is it because of organizational issues, is it legacy software, or just the way things are?
And if what we see is so bad, isn’t common market sense saying that surely these companies will be disrupted and made obsolete by a new and better competitor? I won’t give a definitive answer why software is bad (I have already blamed developers, though I admit, developers are only a part of the problem). But I’ll try to reason about what it’s not getting replaced under market forces.
Peter Thiel in his book “From Zero to One” says that a a new product has to be at least 10 times better in order to be disruptive. Everything below that is just gradual improvement over the status quo. Google was 10 times better than AltaVista, Lycos and Yahoo as a way to find information on the web. Skype was 10 times better than whatever was out there before that. But most of the things that we see as broken don’t get fixed because there’s no 10x improvement that would kill them. Let’s take a loot at a few examples.
Banking. Banking is horrible. Online banking UX is usually on par with a legacy ERP, bank transfers rely on a patchwork of national specifics and ages old international standards like SWIFT. I can’t really think of anyone who really likes their online banking. Mobile banking is even worse, as, of course, it’s tedious to copy-paste IBANs on a phone. Banks are, in many cases, still running COBOL on mainframes, and it’s really hard and expensive to get something changed or fixed. Why they are still around? Because the experience of the online banking is of marginal importance to the bank’s bottom line.
Yes, Revolut and TransferWise are cool tech billion-dollar-valuation startups that are “disrupting” banking. But not really. They are just fancier banks. Yes, finally you can do something on your mobile and you can be onboarded without going to a physical office and support is better and the UI is sleek, and the UX make sense. Do banks care? Not much. Because these startups are just small banks that are currently losing money in order to get to more people and make them spend small amounts online. Nothing of that is a 10x improvement in the value provided. It surely is much better technically, but once you have to transfer money elsewhere, you hit SWIFT or ideally SEPA. Once you want to pay in store, you hit Visa or MasterCard. Nothing of the legacy infrastructure is replaced, we just have better UX for, in many cases, sending money to those legacy banks.
Digital Identity. Anything online where it’s important to know who’s on the other end, requires digital identity. And yet we are nowhere near solving that problem. There are country-specific solutions where governments and/or consortia of banks issue some form of digital identity which others can trust. But it’s fragmented, with varying levels of security and integration isn’t necessarily easy. At the same time we have KYC companies that basically let you scan your customers passport or ID, optionally does a liveness detection (automated or manual) video conference and then check the person against databases of terrorists or sanctioned individuals. So for each service you have to do that again, slightly differently, and if something changes (e.g. an address or name change), there’s not really much you can do.
Ideally, digital identity is federated, using a common standard and easy to integrate, so that enrollment in sensitive online services that require some knowledge about a customer, is straightforward. The identification process gives just as much information as you need. Identity can be managed by multiple entities, government or private, that can vouch that an individual is indeed who they claim to be. The way you identify, with the current technology setting, would be with a key, securely stored in a mobile phone secure storage, using WebAuthn, SAML. There should be a way to re-issue they key in case a phone is lost, and then we’ll have an awesome, reusable digital identity to solve all online authentication and enrollment woes.
Alas, we are nowhere near that. Because what we currently have is working. It’s working terribly, expensively and with a lot of hacks and patches, and most importantly – users have to enrollment for every service separately, but from the point of view of each individual company, it’s easier to just get some service for passport verification and then issue username and password, with optional 2FA, and you are done. The system I described above is not 10 times better than the status quo. A blockchain-based self-sovereign identity, by the way, is also not 10 times better. And it has usability issues that even make it inferior to the status quo.
Privacy. Literally every day there’s a huge data breach or a huge privacy issue (mostly with Facebook). Facebook has been giving our data to 3rd parties, because why not. Companies are collecting whatever they can get ahold of, without much care of protecting it. Sensitive personal data is still stored unencrypted, unpseudonymized, in SQL databases with barely tracked admin access; applications still export bulks of sensitive data to excel sheets sent over email or published to public S3 buckets. Passwords are still stored in plaintext or unsalted, data is communicated over unencrypted connections.
We know how to fix all that; there are many best practices to address all of that. How many companies encrypt data at rest, which is the simplest and most obvious thing to do? How many encrypt data with separate keys to protect bulk exports? How many use pseudonymization when doing exports? Knowledge and best practices are out there but our software is still built as a CRUD application over a SQL datastore and the most important task is to get it done quickly, so let’s not bother with privacy protections.
And, from a business point of view, rightly so, unfortunately. Check the share price of a few recently breached companies. It drops for a few days and then bounces back up. And at the same time no privacy-preserving technology is 10 times better than what we have today, no company will be 10 times better if it protects personal data. So why bother?
Security. In the long run cybersecurity is very important. In the short term, it isn’t. Nothing bad will happen. The other day I inspected the 2FA application of a bank. It’s full of mistakes and yet, through multiple hoops and security-through-obscurity, it reduces the risk of something very bad happening. And if it happens, well, insurance will cover the losses. Data breaches happen not only because we don’t care about personal data, but also because we don’t care about security. Chief infomration security officers find it hard to convince boards that their role is needed, let alone that it needs budget. Security tools are a patchwork of barely working integrations. There was recently a series of rants from one infosec professional that rightly points out that our infosec tools are bad.
Are things improving? A little bit, mostly by introducing more secure protocols. Backward compatibility slows down the improvements, of course, but we’re getting there. But is there a security “fix” that makes things 10 times better, that changes the game, that cuts costs or risk so much? Nope.
These observations go for many more areas, both horizontal and vertical – social networks (where Facebook can’t even get sharing right, let alone data protection), public sector software is mostly still in the 90s, even online payments have not moved since PayPal at the beginning of the century – we still take our credit card from our wallets and type numbers and CVC codes, with all the associated fraud.
In many cases when there’s no visible improvement for years, some proactive governmental structure like the European Commission decides to step in and write some piece of legislation that tries to fix things. For banks, for example, there’s the PSD2 (2nd payment services directive), which mandates open banking – all data about a bank account should be accessible via APIs to third parties who can manage it, display it properly, analyze it. Wonderful in theory, a mess in practice so far – APIs barely work, there’s no standardization and anyone who wants to offer a service ontop of open banking APIs is suffering at the moment. SEPA, the single euro payment area standard, was introduced by a directive as well, more than a decade ago.
Regarding digital identity, there’s the eIDAS regulation which defines how governments should offer their eID cross border, so that, in theory, anyone can use any government or otherwise issued electronic identity to identify for any service across the EU. Things are not there yet at all, and the architecture is so complicated, I don’t think it will work as intended. The fragmented eID space will remain fragmented for all practical purposes.
For privacy there’s, of course, GDPR. And while it resulted in more people trying to take care of personal data, it lead to more documents being written than lines of code changed. Same for information security – the NIS directive tried to improve the security of providers of substantial services. It’s often unclear, though, what is a substantial service to begin with. And then it’s mostly organizational measures. Countries like the UK and Germany have good guidelines and frameworks to improve security but we are yet to see the results.
Why is solving problems properly so hard? Legacy software, legacy standard and legacy thinking certainly doesn’t help. But in many of these domains, solving problems properly does not bring sufficient business value. And we are not prepared to do things properly – while the knowledge on how to do things better exists, it’s not mainstream. And it’s not mainstream, because it’s not perceived as required.
We have half-assed our technology because it kinda works sufficiently to support the bottom line and to make users happy. We have muddled through the myriad of issues with the minimum effort required. Because even the maximum effort would not have been a sufficient improvement to change the game. Even the perfect re-imagined banking, digital identity, social network would not be significantly different for the end user. Sure, a little cheaper and a little more convenient, but not groundbreaking. Not to mention hidden horizontal aspects like security and privacy.
And we will continue that way, pushed by standards and regulations when nothing else helps, to a messy gradual improvement. But it won’t be worth the investment to do things right – why would you invest millions in quality software development when it will marginally affect your business? It’s only worth to get things to barely work.
Published on Java Code Geeks with permission by Bozhidar Bozhanov, partner at our JCG program. See the original article here: Solving Problems Properly Is Often Not Viable Opinions expressed by Java Code Geeks contributors are their own. |
