Use Python to control and data collection of IoT devices

Use Python for IoT device control and data collection

The Internet of Things (Internet of Things) is a hot topic in the current technology field, connecting various physical devices through the Internet, enabling them to exchange data and remote control. As an efficient and easy-to-learn programming language, Python has become one of the preferred languages ​​for developing IoT applications. This article will explore how to use Python to control and data collection of IoT devices and provide corresponding code examples.

1. Overview of IoT Architecture

A typical IoT system consists of the following parts:

• Sensors and actuators: Hardware components for data collection and device control.
• Communication network: The network responsible for data transmission, including Wi-Fi, Bluetooth, Zigbee, etc.
• Data processing and storage: A server or cloud platform used to process and store collected data.
• User Interface: The interface for users to interact with the system, such as mobile apps or web applications.

Under this architecture, Python can be used in multiple levels, including device control, data collection, data processing, and user interface development.

2. Use Python to control IoT devices

To show how to control IoT devices using Python, let’s take controlling a simple LED light as an example. We will use the Raspberry Pi as the hardware platform and control the switch of the LED lights through the GPIO (General Input and Output) interface.

Hardware preparation

• A Raspberry Pi
• An LED light
• A resistor
• Several DuPont Lines

Connect the circuit

Connect the positive electrode (long pin) of the LED lamp to the GPIO17 pin of the Raspberry Pi through a resistor, and the negative electrode (short pin) to the GND pin of the Raspberry Pi.

Control code

We will use Pythonlibrary to control GPIO pins. First, install the library:

pip install 

Then, write the following Python code:

import  as GPIO
import time

# Set GPIO mode to BCM()

# Set GPIO17 to output mode(17, )

try:
    while True:
        # Turn on LED        (17, )
        (1)
        # Turn off LED        (17, )
        (1)
except KeyboardInterrupt:
    pass
finally:
    # Clean up GPIO status    ()

This code sets the GPIO17 pin to output mode and controls the switch of the LED light by switching between high and low levels to achieve the effect of flashing once per second.

3. Use Python for data collection

In IoT applications, data collection is another key link. We can use various sensors (such as temperature sensors, humidity sensors, etc.) to collect environmental data and upload this data to a server or cloud platform for processing and storage. Here, we take the DHT11 temperature and humidity sensor as an example to demonstrate how to use Python to collect environmental data.

Hardware preparation

• A Raspberry Pi
• A DHT11 temperature and humidity sensor
• Several DuPont Lines

Connect the circuit

Connect the VCC pin of the DHT11 to the 3.3V pin of the Raspberry Pi, the GND pin to the GND pin of the Raspberry Pi, and the data pin to the GPIO4 pin of the Raspberry Pi.

Data collection code

We will useAdafruit_DHTLibrary to read data from DHT11 sensor. First, install the library:

pip install Adafruit_DHT

Then, write the following Python code:

import Adafruit_DHT
import time

# Set the sensor type to DHT11sensor = Adafruit_DHT.DHT11
# Set the sensor pin to GPIO4pin = 4

while True:
    # Read temperature and humidity data    humidity, temperature = Adafruit_DHT.read_retry(sensor, pin)
    
    if humidity is not None and temperature is not None:
        print(f'Temperature: {temperature:.1f}°C  Humidity: {humidity:.1f}%')
    else:
        print('Failed to get reading. Try again!')

    # Read every 2 seconds    (2)

The code passesAdafruit_DHT.read_retry()The function reads temperature and humidity data from the DHT11 sensor and outputs the data to the console. Read data every 2 seconds.

4. Data upload and processing

The collected data usually needs to be uploaded to the server or cloud platform for further processing and analysis. We can use the HTTP protocol to send data to a RESTful API. Here we assume that there is already an API endpoint that can receive POST requests.

Upload data code

We will userequestsThe library sends data to the API endpoint. First, install the library:

pip install requests

Then, write the following Python code:

import Adafruit_DHT
import time
import requests

# Set the sensor type to DHT11sensor = Adafruit_DHT.DHT11
# Set the sensor pin to GPIO4pin = 4

# API Endpoint URLurl = '/api/data'

while True:
    # Read temperature and humidity data    humidity, temperature = Adafruit_DHT.read_retry(sensor, pin)
    
    if humidity is not None and temperature is not None:
        # Prepare data        data = {
            'temperature': temperature,
            'humidity': humidity
        }
        # Send a POST request        response = (url, json=data)
        
        if response.status_code == 200:
            print('Data uploaded successfully')
        else:
            print('Failed to upload data')
    else:
        print('Failed to get reading. Try again!')

    # Read every 2 seconds    (2)

After successfully reading the temperature and humidity data, the code sends the data to the specified API endpoint via a POST request. If the upload is successful, the console will output "Data uploaded successfully".

5. Data storage and analysis

After collecting and uploading the data, the next step is to store this data for subsequent analysis. Common storage options include relational databases (such as MySQL, PostgreSQL) and NoSQL databases (such as MongoDB). Here we will use MongoDB to store temperature and humidity data and show how to perform simple data analysis.

Install MongoDB and related libraries

First, make sure you have installed MongoDB server and installed Python's MongoDB client librarypymongo：

pip install pymongo

Connect to MongoDB and store data

Write the following Python code to store the collected temperature and humidity data into MongoDB:

import Adafruit_DHT
import time
import requests
from pymongo import MongoClient

# Set the sensor type to DHT11sensor = Adafruit_DHT.DHT11
# Set the sensor pin to GPIO4pin = 4

# Connect to MongoDB serverclient = MongoClient('mongodb://localhost:27017/')
# Select a databasedb = client.iot_data
# Select a collection (equivalent to a table in a relational database)collection = db.sensor_data

while True:
    # Read temperature and humidity data    humidity, temperature = Adafruit_DHT.read_retry(sensor, pin)
    
    if humidity is not None and temperature is not None:
        # Prepare data        data = {
            'temperature': temperature,
            'humidity': humidity,
            'timestamp': ('%Y-%m-%d %H:%M:%S')
        }
        # Insert data into MongoDB        result = collection.insert_one(data)
        
        if :
            print('Data inserted with id:', result.inserted_id)
        else:
            print('Failed to insert data')
    else:
        print('Failed to get reading. Try again!')

    # Read every 2 seconds    (2)

The code connects to the local MongoDB server and inserts the temperature and humidity data intoiot_dataDatabasesensor_datain the collection. Each data record includes temperature, humidity, and timestamp.

Data Analysis

Once the data is stored in MongoDB, we can analyze it. Here is a simple example showing how to calculate the average temperature and humidity over a certain period of time.

from pymongo import MongoClient
from datetime import datetime, timedelta

# Connect to MongoDB serverclient = MongoClient('mongodb://localhost:27017/')
# Select a databasedb = client.iot_data
# Select a collectioncollection = db.sensor_data

# Set the time rangeend_time = ()
start_time = end_time - timedelta(hours=1)

# Query dataquery = {
    'timestamp': {
        '$gte': start_time.strftime('%Y-%m-%d %H:%M:%S'),
        '$lte': end_time.strftime('%Y-%m-%d %H:%M:%S')
    }
}
data = list((query))

# Calculate average temperature and humidityif data:
    avg_temperature = sum(d['temperature'] for d in data) / len(data)
    avg_humidity = sum(d['humidity'] for d in data) / len(data)
    print(f'Average Temperature: {avg_temperature:.2f}°C')
    print(f'Average Humidity: {avg_humidity:.2f}%')
else:
    print('No data found for the given time range.')

This code queries data over the past hour and calculates the average temperature and humidity. The results will be printed on the console.

6. Data visualization

Data visualization is an important means to understand and analyze data. We can use PythonmatplotlibLibrary to draw a chart of temperature and humidity over time.

Install matplotlib

First, installmatplotlibLibrary:

pip install matplotlib

Draw data charts

Write the following Python code to get data from MongoDB and draw a chart:

import  as plt
from pymongo import MongoClient
from datetime import datetime, timedelta

# Connect to MongoDB serverclient = MongoClient('mongodb://localhost:27017/')
# Select a databasedb = client.iot_data
# Select a collectioncollection = db.sensor_data

# Set the time rangeend_time = ()
start_time = end_time - timedelta(hours=1)

# Query dataquery = {
    'timestamp': {
        '$gte': start_time.strftime('%Y-%m-%d %H:%M:%S'),
        '$lte': end_time.strftime('%Y-%m-%d %H:%M:%S')
    }
}
data = list((query))

# Extract time, temperature and humidity datatimestamps = [(d['timestamp'], '%Y-%m-%d %H:%M:%S') for d in data]
temperatures = [d['temperature'] for d in data]
humidities = [d['humidity'] for d in data]

# Draw the temperature curve(figsize=(10, 5))
(timestamps, temperatures, label='Temperature (°C)', color='tab:red')
('Time')
('Temperature (°C)')
('Temperature Over Time')
()
(True)

# Show charts()

# Draw the humidity curve(figsize=(10, 5))
(timestamps, humidities, label='Humidity (%)', color='tab:blue')
('Time')
('Humidity (%)')
('Humidity Over Time')
()
(True)

# Show charts()

This code will take data from MongoDB for the past hour and plot the graph of temperature and humidity over time. usematplotlibofplotFunctions draw data curves and usefigureandshowFunction display chart.

7. Remote control and automation

An important function of the Internet of Things is the ability to remotely control devices and implement automation. We can remotely control the LED lights on the Raspberry Pi by writing server-side programs and client programs.

Create a simple web server

We can use PythonFlaskThe framework creates a simple web server that accepts control instructions from the client.

First, installFlask：

pip install Flask

Then, write the following server code:

from flask import Flask, request
import  as GPIO

app = Flask(__name__)

# Set GPIO mode to BCM()
# Set GPIO17 to output mode(17, )

@('/led', methods=['POST'])
def control_led():
    action = ('action')
    if action == 'on':
        (17, )
        return 'LED is ON', 200
    elif action == 'off':
        (17, )
        return 'LED is OFF', 200
    else:
        return 'Invalid action', 400

if __name__ == '__main__':
    (host='0.0.0.0', port=5000)

This code creates a Flask application, defines a/ledEndpoint, the switch of the LED light can be controlled through POST request.

Create client control code

Write the following client code to control the LED light by sending HTTP requests:

import requests

url = 'http://raspberrypi_ip:5000/led'

def turn_led_on():
    response = (url, json={'action': 'on'})
    print()

def turn_led_off():
    response = (url, json={'action': 'off'})
    print()

# Test control LED lightsturn_led_on()
(5)
turn_led_off()

Willraspberrypi_ipReplace with the actual IP address of the Raspberry Pi. Run this client code to remotely control the switch of the LED light.

8. Security and Extension

Security is an important issue in IoT applications. In order to ensure the security of the system, we need to take some measures, such as data encryption, authentication and authorization control.

Enable HTTPS

AvailableFlaskofFlask-TalismanExtensions to enable HTTPS, thereby encrypting data transfers.

pip install Flask-Talisman

Add the following to the server code:

from flask_talisman import Talisman

# Initialize Flask-TalismanTalisman(app)

Add authentication

AvailableFlask-HTTPAuthExtensions add simple authentication mechanisms.

pip install Flask-HTTPAuth

Add the following to the server code:

from flask_httpauth import HTTPBasicAuth

auth = HTTPBasicAuth()

# User authentication informationusers = {
    "admin": "password"
}

@auth.get_password
def get_pw(username):
    if username in users:
        return (username)
    return None

@('/led', methods=['POST'])
@auth.login_required
def control_led():
    action = ('action')
    if action == 'on':
        (17, )
        return 'LED is ON',

 200
    elif action == 'off':
        (17, )
        return 'LED is OFF', 200
    else:
        return 'Invalid action', 400

In this way, only users who provide the correct username and password can control the LED lights.

Expand to more devices

With a similar approach, the system can be extended to control more devices and collect more types of data. Just add the corresponding hardware and code to build a more functional IoT system.

9. Deployment and Operation and Maintenance

After successfully developing and testing IoT applications, the next step is deployment and operation and maintenance. Deployment involves transferring applications from development environment to production environment, while operations and maintenance are about ensuring that applications run smoothly in production environments. Here are some key steps and considerations.

Deployment environment preparation

When deploying IoT applications, you need to prepare your production environment first. For projects using Raspberry Pi, the following points can be considered:

• operating system: Make sure to run the latest version of Raspbian (now Raspberry Pi OS).
• Dependency management:usevirtualenvorpipenvTo manage Python environments and dependencies for better control of versions and isolation environments.
• Hardware monitoring: Regularly check the hardware status of the Raspberry Pi, including temperature, power supply voltage, etc. to ensure that the equipment operates within a safe range.

Deployment automation

To simplify the deployment process, automation tools can be used. For example, useFabricorAnsibleRemote deployment and management:

Deployment with Fabric

First, install Fabric:

pip install fabric

Then, write a deployment script：

from fabric import Connection

def deploy():
    host = "raspberrypi_ip"
    user = "pi"
    code_dir = "/home/pi/iot_project"

    conn = Connection(host=host, user=user)

    with (code_dir):
        ("git pull")
        ("pip install -r ")
        ("sudo systemctl restart iot_service")

if __name__ == "__main__":
    deploy()

Running this script can automatically pull the latest code, install dependencies, and restart the service.

Operation and maintenance and monitoring

In the operation and maintenance of IoT applications, monitoring the status and performance of the system is key. The following tools and techniques can be used to monitor and maintain the system:

Monitoring with Prometheus and Grafana

Prometheus is an open source monitoring system, and Grafana is an open source data visualization platform. The combination of the two can achieve powerful monitoring and alarm functions.

Install Prometheus and Grafana

Install Prometheus on a Raspberry Pi:

sudo apt-get update
sudo apt-get install prometheus

Install Grafana on a Raspberry Pi:

sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
wget -q -O - / | sudo apt-key add -
echo "deb /oss/deb stable main" | sudo tee /etc/apt//
sudo apt-get update
sudo apt-get install grafana

Configure Prometheus

Edit Prometheus configuration file/etc/prometheus/, add the target to monitor:

scrape_configs:
  - job_name: 'iot_devices'
    static_configs:
      - targets: ['localhost:9090']

Configure Grafana

Start Grafana:

sudo systemctl start grafana-server
sudo systemctl enable grafana-server

Then, access through the browserhttp://raspberrypi_ip:3000, configure data source and dashboard.

Backup and Restore

To prevent data loss, regular backups are necessary. AvailablersyncOr other backup tools to back up MongoDB data and other key files.

Backup data using rsync

Write a backup script：

#!/bin/bash

# Define backup directory and target directoryBACKUP_DIR="/home/pi/backups"
TARGET_DIR="/mnt/external_drive/backups"

# Create a backup directorymkdir -p $BACKUP_DIR

# Backup MongoDB datamongodump --out $BACKUP_DIR/mongodb_backup_$(date +%Y%m%d)

# Use rsync to synchronize to the target directoryrsync -av --delete $BACKUP_DIR $TARGET_DIR

Set up timed tasks to perform backups regularly:

crontab -e

Add the following line and perform the backup every day at 2 a.m.:

0 2 * * * /home/pi/

Update and upgrade

Keeping the system and software updated ensures security and functionality improvement. You can periodically update the system and Python packages using the following command:

sudo apt-get update && sudo apt-get upgrade -y
pip install --upgrade pip setuptools wheel
pip list --outdated | cut -d ' ' -f1 | xargs -n1 pip install -U

troubleshooting

During the operation and maintenance process, various problems may be encountered. Here are some common problems and their solutions:

• Network connection issues: Check the network connection to ensure that the Raspberry Pi can access the Internet normally.
• Device restart problem: Check the power supply to ensure that the Raspberry Pi is powered stably and avoid frequent restarts.
• Service startup issues: Check the service log, such asjournalctl -u iot_service, locate the cause of the problem and solve it.

10. Extension and Optimization

The expansion and optimization of IoT systems are the key to improving system performance and availability. Here are some common extensions and optimization methods.

Extended sensors and devices

To add system functionality, more sensors and devices can be added. For example, light sensors, air quality sensors, etc. can be added to realize monitoring of various environmental data by modifying the code and circuit connection.

Data processing and analysis optimization

Using more advanced data processing and analysis technologies can improve the intelligence level of the system. For example, machine learning algorithms can be used to predict and classify collected data to achieve intelligent control and alarms.

Prediction using scikit-learn

Installscikit-learn：

pip install scikit-learn

Write data prediction code:

from sklearn.linear_model import LinearRegression
from pymongo import MongoClient
import numpy as np

# Connect to MongoDBclient = MongoClient('mongodb://localhost:27017/')
db = client.iot_data
collection = db.sensor_data

# Get datadata = list(().sort('timestamp', -1).limit(100))

# Extract features and tagstemperatures = ([d['temperature'] for d in data]).reshape(-1, 1)
timestamps = ([i for i in range(len(data))]).reshape(-1, 1)

# Create and train a modelmodel = LinearRegression()
(timestamps, temperatures)

# Predict future temperaturefuture_timestamps = ([i for i in range(len(data), len(data) + 10)]).reshape(-1, 1)
predictions = (future_timestamps)

print('Predicted future temperatures:', predictions)

System performance optimization

In order to improve system performance, the following measures can be taken:

• Optimize code: Through code optimization and reconstruction, unnecessary computing and data transmission can be reduced and system efficiency can be improved.
• Using asynchronous programming: For I/O-intensive operations, an asynchronous programming model can be used (e.g.asyncio), improve concurrent processing capabilities.
• Hardware upgrade: Choose higher performance hardware devices according to your needs, such as upgrading the Raspberry Pi version or using other embedded development boards.

Distributed system architecture

For large-scale IoT systems, you can consider adopting a distributed system architecture to improve the scalability and reliability of the system. For example, multiple Raspberry Pi nodes are used to form a cluster to implement distributed processing of data and tasks through distributed databases and load balancers.

Cloud platform integration

Integrating the Internet of Things system with the cloud platform can leverage the powerful computing and storage capabilities of the cloud platform to achieve large-scale data processing and intelligent analysis. Common cloud platforms include AWS, Google Cloud, and Azure. By using IoT services on cloud platforms, such as AWS IoT Core, it is easy to manage and monitor a large number of IoT devices.

11. Example Project

In order to better understand how to apply the above technologies and methods, we can implement a practical IoT project. Here we take a smart home system as an example to show how to use Python to develop a smart home system with multiple sensors and devices.

Project requirements

• Environmental monitoring: Monitor the indoor environment through temperature and humidity sensors.
• Light control: Remote light control is realized through smart light bulbs.
• Access control system: Access control management is realized through RFID module.
• Data analysis and visualization: Collect data and analyze and visualize it.

System architecture

The system consists of multiple subsystems, each subsystem corresponding to a functional module. Each subsystem communicates through the MQTT protocol and summarizes the data to a central server for processing and analysis.

Hardware components

• Raspberry Pi: As the control center of the central server and each subsystem.
• DHT11 temperature and humidity sensor: used for environmental monitoring.
• Smart light bulb: used for lighting control.
• RFID module: used for access control management.

Software Components

• Flask: Used to develop web server and API interfaces.
• paho-mqtt: Used for MQTT communication.
• MongoDB: Used for data storage.
• **

Grafana**: for data visualization.

• scikit-learn: used for data analysis and prediction.

Environmental monitoring module

The code to implement the environmental monitoring function is as follows:

import Adafruit_DHT
import time
import  as mqtt

# MQTT configurationMQTT_BROKER = "broker_ip"
MQTT_PORT = 1883
MQTT_TOPIC = "home/environment"

# Set sensor type and pinssensor = Adafruit_DHT.DHT11
pin = 4

# MQTT client configurationclient = ()
(MQTT_BROKER, MQTT_PORT, 60)

while True:
    # Read temperature and humidity data    humidity, temperature = Adafruit_DHT.read_retry(sensor, pin)
    
    if humidity is not None and temperature is not None:
        # Publish data to MQTT        payload = f"{{'temperature': {temperature}, 'humidity': {humidity}}}"
        (MQTT_TOPIC, payload)
        print(f"Published: {payload}")
    else:
        print("Failed to get reading. Try again!")
    
    (2)

Light control module

The code to implement the lighting control function is as follows:

import  as mqtt
import  as GPIO

# MQTT configurationMQTT_BROKER = "broker_ip"
MQTT_PORT = 1883
MQTT_TOPIC = "home/light"

# Set GPIO mode and pins()
(17, )

# MQTT callback functiondef on_message(client, userdata, msg):
    if  == MQTT_TOPIC:
        action = ()
        if action == "on":
            (17, )
            print("Light ON")
        elif action == "off":
            (17, )
            print("Light OFF")

# MQTT client configurationclient = ()
(MQTT_BROKER, MQTT_PORT, 60)
(MQTT_TOPIC)
client.on_message = on_message

# Start the MQTT clientclient.loop_forever()

Access control system module

The code to implement the access control management function is as follows:

import  as GPIO
from mfrc522 import SimpleMFRC522
import  as mqtt

# MQTT configurationMQTT_BROKER = "broker_ip"
MQTT_PORT = 1883
MQTT_TOPIC = "home/access"

# Initialize the RFID readerreader = SimpleMFRC522()

# MQTT client configurationclient = ()
(MQTT_BROKER, MQTT_PORT, 60)

try:
    while True:
        print("Place your card to scan")
        id, text = ()
        print(f"Card ID: {id}, Text: {()}")
        
        # Publish access control data to MQTT        payload = f"{{'card_id': {id}, 'text': '{()}'}}"
        (MQTT_TOPIC, payload)
        print(f"Published: {payload}")

finally:
    ()

Data analysis and visualization

Data monitoring and visualization were used using the aforementioned Prometheus and Grafana, and data analysis and prediction were used using scikit-learn.

12. Summary and future prospects

Through the above examples, we can see that Python has a wide range of applications and powerful functions in the field of the Internet of Things. From hardware control, data collection and storage, to data analysis, visualization and remote control, Python provides a complete set of solutions to help developers quickly build and deploy IoT systems. In the future, with the continuous development of IoT technology, Python will continue to play an important role in this field, providing more innovative and efficient solutions for smart homes, industrial automation, smart cities, etc.

Summarize

Through this article, we discuss in detail how to use Python for IoT devices control and data collection, covering all aspects from hardware control, data storage and analysis, remote control and automation, to deployment and operation and maintenance, expansion and optimization. Here is a summary of the key points of each part:

1. Hardware control：

   • Use a Raspberry Pi and DHT11 temperature and humidity sensor.
   • Control LED lights to achieve basic hardware operation.

2. Data collection and storage：

   • Use the Adafruit_DHT library to read sensor data.
   • Create a web server using Flask and upload data through the REST API.
   • Use MongoDB to store data.

3. Data analysis and visualization：

   • Use MongoDB to conduct simple data query and statistics.
   • Use the matplotlib library to draw a graph of temperature and humidity changes.
   • Data prediction is performed using scikit-learn.

4. Remote control and automation：

   • Develop a web server using the Flask framework.
   • Use the MQTT protocol to realize communication and control between devices.

5. Deployment and operation and maintenance：

   • Use tools such as Fabric and Ansible to automate deployment.
   • System monitoring is performed using Prometheus and Grafana.
   • Data is backed up regularly through tools such as rsync.
   • Enable HTTPS with Flask-Talisman to ensure data transmission is secure.
   • Add HTTP Basic Authentication to implement simple authentication.

6. Extension and optimization：

   • Add more sensors and devices to expand system capabilities.
   • Improve system performance through code optimization and asynchronous programming.
   • Use distributed system architecture and cloud platforms to improve system scalability and reliability.

7. Example Project：

   • Develop a smart home system, including functional modules such as environmental monitoring, lighting control, access control management, etc.

Through these steps and examples, we can see the wide application and powerful capabilities of Python in the Internet of Things field. Python not only enables hardware control and data processing easily, but also supports efficient development, deployment and operation and maintenance through rich libraries and tools. With the continuous development of IoT technology, Python will continue to play an important role in smart homes, industrial automation, smart cities and other fields, providing developers with more innovative and efficient solutions.

The above is the detailed content of using Python for the control and data collection of IoT devices. For more information about Python device control and data collection, please follow my other related articles!