MQTT broker restart: retaining values with Mosquitto, EMQX, HiveMQ, persistent sessions, QoS

User question

How to make MQTT to send values after the MQTT broker has restarted

Artificial Intelligence Response

Direct answer to the question

• Publish every “state” message with the retain flag set and (preferably) QoS 1.
• Keep broker persistence enabled so the retained database survives a restart.
• Let clients reconnect automatically (fixed ClientID, cleanSession=false/cleanStart=false).
With these three measures the broker will send the last stored value to every client immediately after both sides are back online—no manual re-publishing is required.

---

Detailed problem analysis

1. What disappears during a broker restart?
   • RAM-resident sessions, inflight queues, subscriptions, and retained messages – unless persistence is enabled.
2. Protocol mechanisms that cure the problem
   a) Retained Message
   – One per topic; stored on disk if persistence is on.
   – Delivered as the very first packet after a successful SUBSCRIBE.
   b) Persistent Session (MQTT 3.1.1 cleanSession = false; MQTT 5 cleanStart = false, sessionExpiryInterval > 0)
   – Broker remembers un-acknowledged QoS 1/2 packets, subscriptions, and will resume them after restart.
   c) QoS 1/2 publish + client-side retransmission queue
   – Publisher keeps message until PUBACK/PUBCOMP; if the connection drops it pushes the packet again when the TCP link is back.
3. Failure matrix

Failure moment	Retain	Persistent session	Result
Broker dies before PUBACK	Irrelevant (retain not stored)	Client resends after reconnect, so value finally stored
Broker dies after PUBACK	Retain already stored on disk, survives	Nothing to redo
Subscriber reconnects	Broker immediately sends retained msg	Broker also drains queued QoS 1/2 messages

---

Current information and trends

• MQTT v5 adds sessionExpiryInterval (fine-grained control instead of boolean cleanSession).
• Message properties messageExpiryInterval prevent stale data buildup in retained DB.
• Modern brokers (EMQX 5.x, HiveMQ 4.x, Mosquitto 2.0) persist the store with LMDB or RocksDB for GB-scale datasets.
• Edge devices increasingly include on-device flash buffering (Azure IoT SDK, AWS Greengrass, Zephyr RTOS “settings” backend) so that a publisher can ride out multi-hour outages.

---

Supporting explanations and details

1. Retained publish examples

Mosquitto CLI

mosquitto_pub -t "plant/temperature" -m "23.8" -q 1 -r

Python / Paho

client.publish("plant/temperature", payload="23.8", qos=1, retain=True)

2. Enable persistence (Mosquitto)

   persistence true
   persistence_file mosquitto.db
   persistence_location /var/lib/mosquitto/
   autosave_interval 30 # s

3. Robust client skeleton (Paho)

   
   import paho.mqtt.client as mqtt
   client = mqtt.Client(client_id="sensor-001", clean_session=False)
   client.reconnect_delay_set(min_delay=1, max_delay=120)
   client.will_set("plant/status", "offline", qos=1, retain=True)

def on_connect(c,u,f,rc): if rc==0: c.publish("plant/status", "online", qos=1, retain=True)

client.connect("broker.lan", 1883, 60) client.loop_start()

4. Clearing a wrong retained value 

mosquitto_pub -t "plant/temperature" -r -n

---
### Ethical and legal aspects
• Retained data can unintentionally expose historical or private sensor values.
• In medical / safety-critical fields ensure compliance with IEC 62304, FDA CFR 820 (software recall of stale data).
• GDPR/CCPA may classify retained user data as *personal data*; provide a deletion mechanism.
---
### Practical guidelines
• Always give each device a **stable ClientID**; otherwise the broker cannot link the new connection with its old session.
• For high-rate telemetry consider a “state topic” (retained) plus a “live stream topic” (non-retained) to avoid disk bloat.
• Validate persistence by:
1) Publish retained value.
2) `systemctl restart mosquitto`.
3) `mosquitto_sub -t "plant/#"` – the value should pop up instantly.
• If your client library only buffers in RAM and the device might reboot, add a local flash queue (SQLite, JSON file).
---
### Possible disclaimers or additional notes
• Retained messages are **not** a historical queue—only the last value is kept. For full history use MQTT v5 Shared Subscriptions with an external database or time-series system (InfluxDB, TimescaleDB).
• QoS 2 adds latency; use it only when duplicate suppression is indispensable.
• Disk persistence can be disabled on embedded brokers (memory constrained); confirm the default.
---
### Suggestions for further research
• Evaluate MQTT-SN or Sparkplug-B if you run sensor networks with sleep cycles.
• Look at “bridging” two brokers with inter-broker queues for geographic redundancy.
• Explore NTS 1.1 (NATS JetStream) or Apache Pulsar for scenarios requiring replay of the entire message stream.
• Read the MQTT v5.0 specification sections 3.3 (PUBLISH) and 3.1.2.4 (Session Expiry Interval) for deeper protocol insight.
---
### Brief summary
Set the **retain flag**, keep **broker persistence** enabled, and make clients use **persistent sessions with auto-reconnect**. After a broker restart, the broker reloads the retained database from disk and, upon each client’s reconnection, instantly republishes the last stored value, achieving seamless state recovery with no lost data.

Disclaimer: The responses provided by artificial intelligence (language model) may be inaccurate and misleading. Elektroda is not responsible for the accuracy, reliability, or completeness of the presented information. All responses should be verified by the user.