A pathway to a well regulated instant messaging market
For the last couple of years I have been consulting and doing contract work for various instant messaging companies. All of them use the Extensible Messaging and Presence Protocol (XMPP); Mostly with standard extensions on standard servers. There might be a custom extension here and there but nothing that couldn’t be written down as a standard. (I’m of course not implying that all companies use XMPP, but the ones that do might contract me because that’s my area of expertise.) Using XMPP works fine for those companies. Their products might not always be a success simply because it is difficult to launch an instant messenger in 2019 where even big companies like Google and Microsoft have failed numerous times. However some of them strive in their particular niche and I’m proud to have been working with them. From talking to other developers in the XMPP community I know that many of them have made similar experiences. While the open source side of XMPP admittedly looks pretty rough at times and can not unconditionally be recommended to end users, there are plenty of closed source solutions out there that don’t usually advertise themselves as being built upon XMPP. I’m saying all this to make one particular point: If you can look past some of the open source clients that are being developed by people in their spare time (and those make up what people usually perceive as the XMPP ecosystem) there are plenty of companies who have built successful services on top of the same protocol. However those services have one thing in common; None of them are federated and none of them make it particularly easy to connect with third party clients. From my experience this is mostly due to two reasons. The first reason – and that is pretty common with the more open companies; for example those who use and advertise their use of open source software – is quality control. They are afraid of people trying to connect with one of the bad open source clients (I’m sorry but I’m looking at you Pidgin) and have a bad user experience as a result of that.
However the most predominant reason on why companies don’t federate – despite the fact that they are using XMPP, and federating with other XMPP server would be as simple as flipping a switch – is that it doesn’t work with their business model. There are people who are better suited than I am to explain modern, venture-capital driven, business models but here is a rough outline: A VC funded business is all about rapid growth and making it to the next evaluation. Investors invest in 10 start ups; 9 of them won’t make it; so the remaining one not only has to make up for the loss of the other 9 but also return additional profit on top of that. Ironically a lot of those companies don’t even have an actual business model or a planned source of revenue outside new rounds of funding. They just vaguely point towards “Ads or whatever”. “Once we have enough market share we are going to sell ads or premium services; but for now let’s focus on growth” seems to be the common idea.
It is easy to understand why federation and interoperability goes against that business model. If your users can just move to a different server once you start sending advertisements or install a different app as soon you start showing ads in it, your vague promise of a future revenue stream falls apart. Aside from that, a truly federated ecosystem has no way of measuring its users so it is impossible to achieve the rapid growth investors demand simply because it is not quantifiable.
Those problems are not going to go away. In a perfect world XMPP could be a flawless protocol with standard libraries in every programming language or one could even imagine an alternate approach that puts JSON in a block chain to create the perfect instant messaging protocol and the market economics that disincentivise federation and interoperability would not go away.
So where do we go from here? Ask the open source scene and the answer is simple: Improve the existing open source clients. And yes more government support in form as subsidies, such as allowing open source developers to be non-profit/charitable organizations or even more direct forms of funding, is always welcome and probably not a bad idea independently of a discussion regarding instant messaging.Will it help to drive people away from WhatsApp? Probably not. Imagine a situation where all open source XMPP developers are funded by their respective governments and together they create a perfect set of instant messaging clients. Will that make people stop using WhatsApp and prefer federated XMPP? Probably not. It will just put the open source scene in a similar position that Google, Microsoft and other well funded companies have found themselves in before. A situation where – despite having good clients – it is impossible to get over the network effect that keeps everyone in the existing walled gardens. Admittedly a well funded, federated solution might actually have slightly better chances than the attempts made by Google and Microsoft because it promises to move people out of the walled garden and not just from one walled garden to the next. However the amount of people who care about that are marginal and not enough to actually do something about the network effect.
This brings us to regulation. If companies don’t have the incentive to make their instant messengers interoperable and allow for competition, we have to force them. In some circles – especially in tech – regulation has a bad reputation. I’m not the best person to explain the market economy to you and there are better resources out there; But I can say that a working market economy is never fully free and there are always laws and regulation that hold everything together. Just think of trademark and patent laws as one example. Without those the life-style company that puts fruit logos on overpriced smartphones would quickly be out of business.
So instead of explaining economics I want to focus on the technical aspects of a regulation of the instant messaging market.
Interoperability and standardization is nothing new in the tech scene and not at all limited to open source. Large companies sitting down and agreeing on a standard is something that happens on a regular basis. And I’m not going to bore you with DNS or TLSv1.3 but instead pick an example that even makes it into mainstream media when ever a new version of the standard comes out. Unicode or to be more specific: Emojis. Emoji is one of the standards where the big players can seemingly work together and create good and relatively fast results and even roll them out relatively quickly to their respective end users. (And as a side note: Unicode in parts is a weird, historically grown mess with some parts hacked together or reused for different purposes; but it still works. Success is not always about quality of the protocol.)
So everyone saying that standards can’t work and the idea of companies working together to create something that is interoperable is inherently flawed, can be proven wrong.
Note that I’m talking about ›large companies‹ not about governments creating the standard. For some people regulations in the tech industry might have a bad taste, because we don’t trust our government to have the technical competence to create a good standard. But the government doesn’t have to. It will just have to prescribe the constraints under which companies in the instant messaging market will have to create a standard for themselves. In practice those will very much look like the constraints a lot of existing standards organizations already have in place like: standards must be publicly available to anyone; any interested party can become a member of the organization; members elect a chair or a council of some sort; no single company should gain a majority in votes and so on. If you wanted to be lazy about it you could just write ‘Has to be an IETF standard’ into law and rely on the decades of experience the IETF has in creating standards and organizing interests of multiple actors. Note also that it doesn’t necessarily have to be XMPP. However with the required extensibility it would probably look a lot like XMPP and as a side effect also face similar challenges that are inherent to extensibility and a protocol that changes over time. The organizational structure behind the protocol extensions and the process that leads to new extensions would probably also look very similar to the process the XMPP Standards Foundation already has in place. But again if its use of XML and the fact that Pidgin is a bad unmaintained messenger is the one thing that keeps you from believing regulation is possible from a technical perspective I’m fine with doing JSON over HTTP instead. Just note that in 20 years from now we will be looking at JSON over HTTP like we are looking at XML over TCP now.
So why not XMPP? With Quicksy we even have a prototype implementation on how a regulated instant messaging market could look to the end user. With Quicksy a user downloads the app and is guided through creating an account with their phone number as an identifier. Quicksy will subsequently discover other Quicksy users based on their phone number. Pretty much exactly as it would happen with WhatsApp, Signal or WeChat. However a Quicksy user is not limited to talking to other Quicksy users, but instead can add individuals on other (XMPP based) services. Those indivduals are of course not automatically discovered but leave the user to enter a full address like name@different-provider.com; Analogous to that, users on different services can themselves establish communication to a Quicksy user, if they have their phone number and know that they happen to use Quicksy, simply by adding +1800123@quicksy.im as a contact. Note that doing so is entirely optional and not forced upon the user. If a user of different-provider.com decides they don’t like to talk to Quicksy users because they are not a fan of their privacy policy they don’t have to. But at least they have the possibility to contact a Quicksy user without being forced to download Quicksy themselves and create an account on that service.
If a law would regulate that kind of interoperability between the major instant messengers, I as a user could very quickly get into the convenient situation where I can pick my own messenger based on what suits me in form of privacy policy and user experience but with a little bit of effort (mainly manually adding contacts) can still talk to people who have made different choices. This will certainly increase competition and might even drive some users away from WhatsApp and WeChat. Right now we are in a situation where some users have installed multiple messengers. The privacy friendly one and WhatsApp. So WhatsApp is not actually losing a customer just because the privacy friendly one is gaining another. If all messengers were forced to interoperate WhatsApp would actually lose a customer in that case and in in the long term might change their mind on some of their privacy decisions.
To me this sounds like a worthy goal. A goal that can only be reached with regulation.