Using an MQTT Broker? Here’s Why You Need to Think About Scalability

MQTT broker scale parabolically as you add devices, so you should always keep this in mind as you scale any IoT platform that's based on MQTT.

There are several options for developing a message broker with full duplex functionalities and various supporting features. Some of these options are the use of a raw TCP socket, a raw UDP socket, AMQP and CoAP. Most of these alternatives have more limitations and complications than benefits, especially when compared to MQTT. This is where MQTT becomes the most appropriate, efficient and convenient choice, especially when building our own Internet of Things platform.

Remember that all of these protocols can coexist, and we could deploy them on the same cloud instance, if necessary. This means that, in the future, if you decide to use AMQP as well as MQTT, it’s possible to integrate some or all of them. More importantly, we can link these channels with an additional plugin program so that there’s a seamless communication from an application’s and device’s perspective.

Using an MQTT Broker

Fundamentally, MQTT is an asynchronous protocol and thus enables duplex communication with a lightweight burden on systems. It allows systems to run on low bandwidth and low power. Conversely, HTTP and similar protocols require relatively high bandwidth and power and are request-response in nature, which means that the client must always initiate communication.

In places where you want either party (server or client) to initiate communication, MQTT is the best choice. Moreover, if systems need to run on low data consumption, especially on batteries, for a long period, it’s prudent to use MQTT. If the device needs to send or receive data frequently and at random, then MQTT also makes sense because it reduces a significant HTTP overhead.

If bandwidth and power is not a concern, then HTTP may be a better choice. It could also be a better choice when data sending or receiving frequency isn’t high, which can block the resources sooner in the process.

Why and When to Use MQTT For IoT Messaging

In an application where live control or monitoring is required, MQTT is an obvious choice because it provides duplex, two-way communication abilities with the least amount of overhead.

You must be mindful of the fact that the workload of an MQTT-based the system can grow parabolically, which means that for each device added to the MQTT speaking network that has n devices in total, the load on the system becomes n squared n*n. The figure below explains this concept graphically.

MQTT-based platform loads increase by n-squared. For example, let’s assume an extreme scenario where there are two clients in which each subscribes to all possible topics. When a client publishes a message, the broker needs to receive a message and another client needs to receive the message too. This means one message sent could result in two transmissions. The same goes for the other client, making it four messages in total for a two-client system.

For a three-client system, this number becomes nine messages in total, (i.e. three messages per client). Simply having 10 devices connected means that the message broker should be capable of handling 10*10 (i.e. 100 messages, and so forth).

When the number of MQTT clients starts to grow, the load on the message broker, overall system and platform will grow almost exponentially.

Always keep this in mind as you scale any IoT platform that’s based on MQTT in the later stages or add more devices to it.