Data Rerouting from TTN and Storage

Introduction

Since The Things Network (TTN) does not offer permanent data storage services (can only store for 7 days), in this tutorial section, the focus will be on how to transfer time series data from TTN to a permanent time series database on a virtual machine on KENET servers. Kenya Education Network (KENET,), is Kenya's National Research and Education Network. It serves education and research institutions in Kenya by providing internet connectivity, cloud services (processing and storage) and research grants.

Originally in Project Muringato, the data from the sensor nodes was being re-routed from TNN to an InfluxDB database on the Google Cloud Platform (GCP) as indicated under “Project Muringato ''. InfluxDB is an open-source database for storing time series data. This data can come from logging and monitoring systems or from IOT (Internet of Things) devices. The difference between the KENET and the GCP setup is, on GCP a custom machine with a pre-installed InfluxDB can be deployed but on KENET a user has to set up the database.

Data Rerouting from TNN

To transfer data from TTN to an InfluxDB database on a KENET server, a Data API (MQTT) was utilized. As an MQTT broker, The Things Stack exposes an MQTT to work with streaming events (data from the sensor nodes). Established in 1999, the MQTT is a machine to machine, lightweight, publish/subscribe connectivity protocol for massage queuing service. It is mainly designed for connections with remote locations that have devices with resource constraints such as power or limited network bandwidth such as IoT (Internet of Things).

On The Things stack, every application TTS automatically exposes an MQTT endpoint. In order to connect to the MQTT server, an API key that functions as a password is needed. Figure 1 shows the MQTT page on TTN.

Figure 1: MQTT Page on The Things Stack

Data Storage on the InfluxDB-KENET Machine

Data storage was done by continuously running an MQTT-Python script on the KENET machine. The script would enable the subscription to the Project Muringato applications on TTN, connect to the application uplinks from the sensor nodes, and then publish the data on the Influx database.

• To set up the InfluxDB-KENET machine, the steps on (This Link) can be utilized. (Setting up the KENET-INFLUXDB Virtual Machine)
• Figure 2 shows the data transfer process.

Figure 2: Data transfer from TTN to a visualization platform

Figure 3 and 4 shows the utilization of the TTN API key and the Python MQTT library on the MQTT-Python script.

Figure 3: TTN API key on the MQTT-Python script.

Figure 4: MQTT Python library on the MQTT-Python script.

To visualize the data from the test sensor node shown on Figure 5, we will run (Python Code on a notebook) on Google Collab. ((Link))

Figure 5: Sensor node at the sensor-node testing station in DeKUT.