EIT Digital begins work on Hadoop open source product and start-up to take innovation to market
EIT Digital the European open innovation organisation has launched a new Innovation Activity* called HopsWorks to work on a next generation Hadoop open-source software framework for distributed storage and processing of very large data sets.
Hadoop provides massive storage for all kinds of data, enormous processing power and the ability to handle a vast number of concurrent tasks or jobs. It also enables the storage of files that are bigger than what can be stored on one particular node or server.
Dr. Jim Dowling, senior researcher at the Swedish Institute of Computer Science (SICS Swedish ICT) and the leader of EIT Digital HopsWorks Innovation Activity, explained: “Hadoop is an open-source software framework for storing data and running applications on clusters of commodity hardware. Our product, dubbed ‘Hops’, will provide the first truly multi-tenant, elastic Hadoop distribution service with unified batch and streaming.
“Unlike existing offerings, Hops will allow organisations to host multiple sensitive datasets on the same cluster, providing dynamic role-based access control for both Hadoop Distributed File Systems – HDFS and Kafka distributed streaming platforms. The platform is unified in an intuitive user interface and provides first-class support for Spark-Streaming, Flink distributed big data analytics, and Kafka.”
The Hadoop ecosystem consists of computer clusters built from commodity hardware. All modules in Hadoop are designed with a fundamental assumption that hardware failures are a common occurrence and should be automatically handled by the system.
HopsWorks will launch Hops as a market-ready product and will distribute the framework via new client facing start-up focusing on the Internet of Things (IoT) and telecoms markets as well as sensitive Big Data owners. The new start-up is expected to gain critical mass and exposure through project partner Ericsson, a global leader in delivering ICT solutions, who will act 
as the reference user of the framework.
The project will also transfer skills in Big Data and streaming analytics to industry, train skilled workers, and will link with EIT Digital’s Data Science Masterschool Education Program and a free course on the edX online learning and MOOC service.
“We expect to be able to bootstrap the Hops ecosystem, and give the startup the escape velocity required to raise the seed capital needed for building a successful business. We estimate our startup to have substantial revenue growth in 2017 and beyond”, Dr. Dowling says.
Comment on this article below or via Twitter @IoTGN