Foresta Inclusive Resources

Section 1: Introduction

Foresta-Open is a networked environmental sensing project that uses MQTT (Message Queuing
Telemetry Transport) to transmit real-time sensor data from embedded microcontrollers to an
Internet of Things prototyping platform called shiftr.io, which can then be subscribed to from
anywhere in the world to drive art installations or other projects. It is part of an ecological and
artistic research initiative exploring how environmental data can be experienced and visualized
in inclusive and interactive formats.

Why MQTT?

MQTT is a lightweight publish/subscribe protocol designed for low-power devices. It’s ideal for transmitting sensor data from remote devices, managing multiple sensor nodes, and keeping communication efficient and scalable.

This project includes two key repositories: 

  • EspMQTTClient: A modified version of Patrick Lapointe’s ESPMQTT Library tailored for the Foresta project, which runs on the ESP32 to read sensors and send data via MQTT 
  • Foresta-Open: A repository that houses all of the sketches required to run your own remote sensing project

1.1 Shiftr.io

Shiftr is a cloud-hosted MQTT broker that provides an interface for visualizing device connections and message pathways in real time. It supports MQTT standard functions, but keeps things lightweight for prototyping and education.

 

💡Note: shiftr.io has a basic free service, which works for 6 hrs a day only. For long term projects it is worth purchasing your own instance. 

1.2 Instances and Tokens

In shiftr, you connect to an MQTT broker instance. Access is controlled through a token, which includes the username, password (key), and instance URL. Tokens take the form:

protocol://user:password@instance.cloud.shiftr.io

1.3 Public Instance

For testing, shiftr provides a free public broker: 

mqtt://public:public@public.cloud.shiftr.io

Here public is the user, password, and instance name. This is useful for examples, but not secure.

1.4 Custom Instances

You can create your own broker instance in the shiftr dashboard. Each instance gets its own URL and supports multiple tokens with different permissions (read, write, or full access). 

Example token: 

mqtt://tester123:DrB0qlfo4EHJGT7W@tester123.cloud.shiftr.io

 

Where: 

user/instance = tester123

password (key) = DrB0qlfo4EHJGT7W

URL = tester123.cloud.shiftr.io

 

Use custom instances for projects or class work. Public is fine for demos, but tokens tied to your instance give you control and security. For this setup we will use the public instance of shiftr.io to test our systems. 

1.5 System Architecture

The ESP32 sketches read analog or digital sensor values and publish them using WiFi to shiftr.io. These values are formatted as MQTT messages under specific topics, which allows any client to subscribe and to receive real-time values. The Foresta-Open system publishes live sensor readings to customizable MQTT topics that can then be subscribed to for individualized projects.

 

At this point, you should have a clear sense of what MQTT is and how shiftr.io works as the broker that manages communication between your devices and clients. With this foundation in place, we can now turn to the practical steps of the Foresta-Open project. This guide will help you to: 

  • Build custom sensor shields for your ESP32 to collect environmental data
  • Program your ESP32 board 
  • Transmit data securely to an MQTT broker (shiftr.io)
  • Harvest the data using Process, Max8, Touch Designer, or another ESP

Section 2: Materials and Tools 

The following hardware and software components are needed to replicate or build the Foresta-Open MQTT system:

Hardware

Component  Description
ESP32 Firebeetle x 4 Microcontroller with built-in WiFi, BLE support and an onboard battery charger. 
Sensors  VOC, wind, lux, soil moisture, soil temperature, particulate, CO2, temperature and humidity 
USB-C/USB-micro cable For programming and power
Breadboard & jumpers For prototyping sensor connections

 

Software

Tool  Purpose
Arduino IDE Used to upload sketches to the ESP32
ESP32 Board Package Must be installed via the Board Manager
Arduino Sensor Libraries Includes EspMQTTClient and sensor-specific libraries.
Shiftr.io Account Acts as a cloud-based MQTT broker
Foresta-Open Repository  Visualizes incoming MQTT messages

 

Section 3: Setup

This section guides you through setting up the Arduino development environment and preparing your ESP32 boards for use with the Foresta system. By the end of this section, you’ll be ready to flash sketches to your ESP32 boards and begin testing communication between them.

3.1 Install Arduino IDE and ESP32 Board Definitions

  1. Download the Arduino IDE
  2. Open Preferences and paste the following under the Additional Board Manager URLs field:
    https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json
  3. Open the Boards Manager (Tools > Board > Board Manager), search for ESP32 by Espressif Systems and install it

 

New to Arduino? Follow this detailed walkthrough from Random Nerd Tutorials:

Installing the ESP32 Board in Arduino IDE (Windows instructions)

3.2 Install the Required Libraries  

In the Arduino IDE, go to the Library Manager (Tools > Manage Libraries) and install the following: 

Core Libraries

  • EspMQTTClient by Patrick Lapointe (this includes PubSubClient by Nick O’Leary and Wire.h). This project uses a modified version of Patrick Lapointe’s ESPMQTT Library. https://github.com/JaneTingley/EspMQTTClient replace Lapointe’s library with this one.

Sensor-Specific Libraries

 

💡Note: Each sketch includes its required libraries at the top. You only need to install the ones relevant to your use case.

3.3 Clone the Foresta Sketches

All ESP32 sketches used in this project are located in a single GitHub repository.

  1. Clone the repository using GitHub Desktop or the command line:
    git clone https://github.com/JaneTingley/Foresta-Open.git
  2. Navigate to: 
    Foresta-Open/Generic-ESPNow

This folder contains: 

  • 1 x Gateway sketch (MQTT-Gateway-WIND-Lux-Generic.ino)
  • 3 x Sensor sketches (EspNowMQTT-Air-Generic.ino,EspNowMQTT-Atmosphere-Generic.ino, EspNowMQTT-Soil-Generic.ino)

You will use one ESP32 for each of the sensor sketches, and one ESP32 for the gateway. For flashing instructions and communication workflow, continue to Section 4.

Section 4: Running the Sensor Sketches

This section walks you through uploading the Generic-ESPNow sketches to ESP32 boards. 

💡Note: there are two types of sensor sketches:

  1. Generic-ESPNOW sketches that send sensor data regularly. This can be defined in the sketch and can be sent as often as you wish. These sensors should be connected to a stable power supply, such as USB.
  2. Generic-DeepSleep Sketches which are optimized for low-power operation and support deep sleep, making them ideal for remote, battery-powered contexts.

4.1 System Overview

In the Foresta network, there are two types of ESP32 devices: 

  • The Gateway: Is connected to WiFi and receives ESP-NOW data, and publishes through MQTT
  • The Sensors: Reads environmental data and sends it via ESP-NOW to the gateway

This approach enables remote sensing that is beyond wifi access. The gateway is connected to WiFi (it needs ongoing power and WiFi access) and creates its own mesh network that has a range of about 300-500 meters. (The ESP is capable of high power mode to increase this range, but this has not been deployed yet.) It is possible to use a MiFi device to convert SIM data to WiFi. This has been tested and works well. However MiFi requires a robust solar/battery combination to keep it charged.

 

[Sensor ESP32]
    |
ESP-NOW
    ↓
[Gateway ESP32 (e.g., WIND-Lux Rare)]
    |
  Wi‑Fi → shiftr.io (MQTT)

 

    1. The Sensor runs an EspNowMQTT-* sketch
    2. It scans for and pairs with the Gateway via ESP‑NOW (based on matching SSID and channel)
    3. It sends payloads in the format topic=value or topic:=value (for retained messages)
    4. The Gateway receives these packets, connects to Wi-Fi, and publishes them to shiftr.io via MQTT

4.2 Sketch Overview 

All required sketches are found in the Foresta-Open/Generic-ESPNow folder and correspond to different environmental sensing modules:

Sensor Sketches

Sketch Name Purpose Required Libraries
EspNowMQTT-Air-Generic Detects airborne VOCs/ CO2, air temperature and humidity Sensirion I2C SGP41, Adafruit_Sensor, CCS811
EspNowMQTT-Atmosphere-Generic Measures Particulate Matter and Rain PMS3005 (modified version)
EspNowMQTT-Soil-Generic Measures soil water content Analog read only (no additional library)

 

Gateway Sketch

The gateway connects to WiFi, receives ESP-NOW messages from any sensor node and publishes them to shiftr.io via MQTT.

Gateway Sketch Role Required Libraries
MQTT-Gateway-WIND-Lux-Generic Gateway + optional wind/lux sensor readings EspMQTTClient (modified version)

 

Responsibilities:

  • Listens for ESP-NOW messages on a designated channel
  • Connects to WiFi using credentials
  • Publishes all received messages to MQTT under clean topic structure

4.3 Prerequisites 

Before uploading any sketches, ensure the following:

  • Arduino IDE is installed and configured (see Section 3)
  • Sensor libraries are installed
  • Two ESP32 boards are connected:
    • One for the sensor
    • One for the gateway
  • Your WiFi and shiftr.io credentials are ready

4.4 Gateway Node Setup

  1. Open MQTT-Gateway-WIND-Lux-Generic.ino in Arduino IDE
  2. Edit the following values at the top of the sketch:
    EspMQTTClient client(

      “wifi-SSID”, //SSID

      “wifi-password”, //wifi password

      “public.cloud.shiftr.io”, //Instance location, 

      “public”, // Username,

      “public”, // secret password,

      GATEWAY_NAME, // Client ID (do not change)

      1883  // default MQTT port (do not change)

    );

  3. Make sure your GATEWAY_NAME is unique and consistent across the gateway and sensor nodes
    #define GATEWAY_NAME “00Name-Gateway”
  4. Make sure your Topic-Names are also consistent:
    lightReading = analogRead(luxPin); // This is the light sensor
        client.publish(“Topic-Name/Light-ESP”, String(lightReading)); // this publishes Light data
        windReading = analogRead(windPin); // This is the wind sensor
        client.publish(“Topic-Name/Wind-ESP”, String(windReading)); //this publishes Wind data
  5. Select the correct board and port, then click Upload
  6. Open the Serial Monitor (baud: 115200) and verify:
    WiFi connected
    MQTT connected to shiftr.io as [client_id]
    Listening for ESP-NOW packets…

Once the gateway is running, it will automatically forward any messages it receives from ESP-NOW nodes to shiftr.io using the topic and value string received.

 

💡A note on upload speed: the ESP32 firebeetle is not capable of managing uploads automatically set to 921600. In order to upload, the speed must be set to 115200

💡A note on topic naming: The included gateway sketch already publishes sensor values to MQTT topics like Topic-Name/Wind-ESP and Topic-Name/Light-ESP. These do not need to be changed unless you are customizing your own MQTT structure later on.

4.5 General Workflow for Sensors (EspNowMQTT-*

Each sensor pod (e.g., Soil, Air, Atmosphere) is an ESP32 that collects environmental data and transmits it to the gateway via ESP-NOW. These steps walk you through preparing and flashing the sensor code.

  1. Follow the sensor diagrams in Section 5 to create the shields for the ESP
      • Build the Gateway first, as this is the only device that is connected to the Internet. Once it is built, test it to ensure that it functions properly. Once it does, then build each shield and test individually.
      • Use the wiring diagrams provided in the Schematic-Hubs pdf to connect your sensors to the ESP32 (these diagrams image three different shields. It is possible to put all sensors onto one shield, but we have not done this yet)
  2. Set the gateway name in the sensor sketches
      • Open the appropriate sensor sketch (ex: EspNowMQTT-Soil-Generic.ino)
      • Ensure that the GATEWAY_NAME matches the name of the gateway defined in the Gateway sketch. This ensures that the sensor can discover and communicate with the correct gateway over ESP-NOW.
  1. Upload the sketch
    • Connect your ESP32 via USB
    • Select your board and COM port, then Upload (set upload speed to 115200)
  1. Open the Serial Monitor (baud: 115200
    • Look for output like:
      Found gateway on channel X
      Paired with gateway
      Publish: Soil/Moisture=512
      Send Status: Success

Section 5: Building the Sensors 

Before uploading and deploying any sensor sketches you must build the ESP sketches. This section shows how to physically build each sensor module, using the schematics provided in the Foresta repository.

5.1 General Wiring Guidelines 

  • This system uses 3.3V
  • The sensors are toggled on/off using a mosfet. This is used to enable the deep-sleep mode and save power. In the ESP-NOW sketches, the mosfet is always on.

💡Note: double check the pin definitions at the beginning of each sketch (they may vary slightly between sketches)

5.2 Gateway Wiring

Sketch: MQTT-Gateway-WIND-Lux-Generic

Sensors: 

  • TEMT6000 Ambient Light Sensor
  • VH400 Soil Moisture (analog)
  • THERM200 Soil Temp (analog)

5.3 Soil Sensor Wiring

Sketch: EspNowMQTT-Soil-Generic

Sensors:

  • VH400 Soil Moisture (via MOSFET) 
  • THERM200 Soil Temp (via MOSFET)

5.4 Atmosphere Sensor Wiring (Particulate and Rain Detection)

Sketch: EspNowMQTT-Atmosphere-Rare.ino
Sensors

  • PMS5003 Particulate Sensor (TX/RX communication)
  • Rain Drop Detector

5.4 Air Sensor Wiring (VOC/CO₂)

Sketch: EspNowMQTT-Air-Rare.ino
Sensors:

  • Adafruit SCD-40
  • CCS811 VOC sensor

5.6 Notes on Assembly

  • Prototype with a solderless breadboard to test the sketches first
  • Jumper wires must be fully seated in the ESP32 header and breadboard
  • Keep sensor wires short to reduce noise

Section 6: Deep Sleep for Remote Sensing

Deep sleep mode allows your ESP32 sensor nodes to operate for months on a single battery charge. This section explains how to use deep sleep, how to adjust the sleep duration, and how to power the devices safely in remote environments. 

6.1 What is Deep Sleep?

Deep sleep is a special low-power mode where the ESP32 shuts down most internal functions, including the CPU, radio, and peripherals. Only the RTC (Real-Time Clock) and essential wake timers stay active. In Foresta-Open, the sensor nodes wake up briefly to power their sensors, collect data, and send data to a WiFi-connected gateway, before returning to deep sleep. This cycle can be tuned to optimize battery life, making it ideal for remote environmental sensing.

6.2 Where to Find Deep Sleep Sketches

Deep sleep versions of each sensor sketch are located in:

Foresta-Open/Generic-DeepSleep

These sketches are modified to include:

  • Deep sleep logic
  • Shorter setup/loop functions
  • A sleep timer that determines how long the board will sleep between transmissions

6.3 How it Works 

At the end of loop(), the following can be found: 

#define DEEP_SLEEP_SECONDS   59*60

#define AWAKE_SECONDS        60 

This code:

  • Tells the ESP32 to wake up after 60 seconds
  • Immediately enters deep sleep
  • After sleep, the board restarts from setup()

 

💡Note: Adjust the number 60000000 to control how often your sensor wakes up

For example:

  • 1 minute: 60000000
  • 15 minutes: 900000000
  • 1 hour: 3600000000

6.4 Hardware for Deep Sleep

In a field deployment, sensor nodes must operate independently from wall power. To enable long-term, low-power functionality, the hardware supports deep sleep with the following features:

Method Description
USB power Used for testing, debugging, or indoor installations
3.7v Lithium Ion 18650 Button-Top Battery Primary rechargeable option for remote deployments; built-in protection circuit prevents dangerous overcharging, over-discharging, and short circuits that can cause damage or fire
Solar + Battery  Can be used in combination with a charge controller for long-term outdoor setups

 

💡Note: Deep sleep only works effectively if your sensor is not draining power while sleeping. That’s why the hardware uses a MOSFET switch to cut power to the sensors while the ESP32 sleeps.

 

About MOSFETs

The sensor ground is routed through a MOSFET. The ESP32 Pin D2 controls the gate of the MOSFET, which switches the sensor power on and off.

6.5 Considerations for Testing

When working with deep sleep sketches during development:

  • Shorten the sleep time for faster testing:
    #define DEEP_SLEEP_SECONDS   10  // 10 seconds for test purposes
  • Use USB power and open the Serial Monitor at 115200 baud
  • Remember: Serial output ends when the ESP32 goes to sleep, so you’ll only see messages during the brief awake window
  • Use the bootCount variable (already included in most sketches) to confirm that the ESP is sleeping and waking up correctly:
    Serial.println(“Boot #” + String(bootCount));

6.6 When to Use Deep Sleep

Use deep sleep if:

  • Your sensor is deployed remotely, powered by battery
  • You only need periodic data, not real-time streaming
  • Your sensors can be safely powered down between reads

Avoid or disable deep sleep if:

  • You are testing/debugging and need continuous Serial output
  • Your sensor must stream values continuously (e.g., every second)

6.7 Summary of Deep Sleep Protocol 

Stage What Happens
Wake  ESP32 powers up, starts in setup()
Sensor On  MOSFET pin (e.g., D2) set to HIGH
Sensor Warmup Delay ensures sensor stabilizes
Read/Send  Data is collected and sent via ESP-NOW
Sensor Off MOSFET pin set to LOW
Sleep esp_deep_sleep_start() called
Repeat Wake on timer, process restarts 

 

Section 7: Harvesting the Data 

Now that your sensors are sending data to shiftr.io via the gateway, you’ll want to receive and use that data in your own creative or analytical applications. This section covers how to subscribe to your MQTT topics using popular platforms.

7.1 MQTT Topic Structure Recap

All sensor nodes send data using this format:

topic=value

The gateway receives this and publishes it to the MQTT broker using:

  • Topic: GATEWAY_NAME/POD_NAME/parameter
  • Value: numeric or string

These topic names are customizable in your sketches but should remain consistent across the network for easier mapping.

7.2 Processing  

In this step, you’ll learn how to receive real-time sensor data from your Foresta MQTT network directly into Processing, using the MQTT Client for Processing library by 256dpi.

Before you begin:

To install the MQTT library:

  1. Open Processing
  2. Go to Sketch > Import Library > Manage Libraries…
  3. Search for MQTT Client for Processing
  4. Click Install

To connect to the shiftr.io broker:

Here’s a minimal working example that connects to the public instance of shiftr.io, publishes a simple counter to the topic ForestaOpen/debug/counter, and listens for messages on the same topic. This example tests basic send-receive functionality, and can be modified later for your own sensor type and topic.

import mqtt.*;

MQTTClient client;
String latestMessage = “”;
int crashCounter = 0;        // Keeps a running count that increases every few seconds; reconnect cycles
int lastPublishTime = 0;
boolean connected = false;   // Track connection state manually
int lastReconnectAttempt = 0; // Track last reconnect attempt time

void setup() {
  size(400, 200);
  client = new MQTTClient(this);

  println(“Connecting to MQTT broker…”);
  client.connect(“mqtt://public:public@public.cloud.shiftr.io”);
  client.subscribe(“ForestaOpen/debug/counter”);
}

void draw() {
  background(255);
  textAlign(CENTER, CENTER);
  textSize(20);
  fill(0);
  text(“Counter: “ + crashCounter, width/2, height/230);
  text(“Last message: “ + latestMessage, width/2, height/2 + 20);

  // Every 3 seconds, publish the counter (only if connected)
  if (connected && millis() – lastPublishTime > 3000) {
    lastPublishTime = millis();
    crashCounter++;
    client.publish(“ForestaOpen/debug/counter”, str(crashCounter));
    println(“Published counter: “ + crashCounter);
  }

  // Try reconnecting if disconnected (every 5 seconds)
  if (!connected && millis() – lastReconnectAttempt > 5000) {
    lastReconnectAttempt = millis();
    println(“Connection lost — attempting to reconnect…”);
    try {
      client.connect(“mqtt://public:public@public.cloud.shiftr.io”);
      client.subscribe(“ForestaOpen/debug/counter”);
    } catch (Exception e) {
      println(“Reconnect failed: “ + e.getMessage());
    }
  }
}

// Called automatically when a message arrives
void messageReceived(String topic, byte[] payload) {
  latestMessage = new String(payload);
  println(topic + ” → “ + latestMessage);
}

// Called automatically when connected
void clientConnected() {
  println(“Connected to broker!”);
  connected = true;
}

// Called automatically when connection lost
void connectionLost() {
  println(“Connection lost!”);
  connected = false;
}

 

💡Note: If you have a personal shiftr.io account and don’t have your ESP node running yet, you can test this sketch by creating a private namespace and manually publishing values to your subscribed topic from the dashboard using the “Send Message” panel.


2022 © Jane Tingley. All rights reserved.