Looking for a Benchmarking Framework for IoT platforms

Our counting of IoT platforms has exceeded 300, writes Saverio Romeo, the principal analyst at Beecham Research. There are now more than 300 solutions that identify themselves as IoT platform offering a variety of services, from device management to data management passing by API management.

It is difficult to make sense out of that landscape. It is difficult to benchmark those platforms through a solid set of indicators. Beecham Research has grouped those platforms based on their sets of services, target market, business models and customer base. But, still, the picture is not clear enough. Our existing classification and analysis can be empowered by benchmarking those platforms towards an architecture of reference. Those has taken us to explore a vast literature, academic and research-based, that has looked at the issue for some time with a rigour that is not necessarily market oriented.

Defining the middleware for Internet of Things

The starting point of this literature is providing a definition of the Internet of Things and, therefore, arguing the absolute necessary of a “middleware for Internet of Things”. The Internet of Things is a combination of the Internet and the things. The Internet is a global network infrastructure self-configuring, scalable and dynamic based on standard and interoperable protocols. The things are physical objects and virtual objects having identities, attributes and intelligent interfaces. The things are heterogeneous, growing out from an unstandardised environments, but all integrated into an information network[1]. The government of these plethora of devices require an intelligent layer that connect the things to the network and abstract applications from the things in order to enable the development of services. We want all this because we want to achieve at least three main objectives: flexibility (being able to deploy things in different contexts), usability (being able to make the user experience easy), and productivity (enabling service creation in order to improve efficiency, but also enabling new service development)[2].

Internet of Things architectures of reference

With this view in mind, the “middleware for Internet of Things” research community has moved in different directions reflecting different school of thoughts on the matter. The Industrial Internet community is working intensively on the matter. The most notable propositions in the area are: Reference Architecture Model Industrie 4.0 (RAMI 4.0) coming from the German manufacturing arena reflecting their effort into deploy the Industrie 4.0 initiative; Industrial Internet Reference Architecture (IIRA) from the Industrial Internet Consortium (IIC). There are then other initiatives which they try to have an holistic view – in terms of sector of applications – on the matter. The OneM2M is in this direction. It brings together ETSI members and other organisation members. A rich contribution to the debate has come from the EU FP7 (Framework Programme 7) project Internet of Things – Architecture (IoT-A)[3]. There are then a number of company-led initiatives such as AllSeen Alliance and Open Fog Computing Consortium.

Common building blocks for the architecture of reference

Those approaches present differences, but some common and essential building blocks for IoT platforms can be drawn.

• Data and connectivity management. Device discovery, authentication, management, and control
• Context-awareness. Collecting, managing and using contextual information (event processing for example)
• Scalability. Being able to enlarge the deployment of devices, agnostically from the type of devices (importance of drivers and APIs)
• Data analytics and visualisation tools. Being able to analyse different data sources (big versus low data, real-time versus batches, high velocity of data production versus low velocity)
• Interoperability. Being flexible enough to engage with other software solutions and exogenous data sources (IT data for example)
• Security. Ensuring the security, the privacy and the integrity of the data gathered
• Innovation enabler. Creating ecosystem of users and developers through application enablement suites

Defining a Benchmarking Framework for IoT Platforms

Combining this set of technological features with the market-oriented ones briefly discussed at the beginning of the article can define a solid benchmarking framework useful for the many stakeholders in the value chain. The IoT platform providers can assess the competitive edge of their solutions. IoT solution and technology providers can identify the right partners for their market strategy. The IoT adopters can assess vendors and the framework can guide them in the final selection.

We are aware that this benchmarking framework is not the final answer, but, it is a step towards a more informed assessment of IoT platforms and a tool to make the IoT platform landscape less cloudy to all the stakeholders.

 

[1] Bandyopadhyay,S. et al. 2011. Role of Middleware for Internet of Things: A Study. International Journal of Computer Science and Engineering Survey (IJCSES), Vol.2, No.3, August 2011.

[2] Ebert, C. et al. 2016. Reference Architectures for the Internet of Things. IEEE Software.

[3] The Internet of Things (IoT-A): http://www.iot-a.eu/public