Node Red Aedes MQTT Broker Persistenz in MongoDB

jokakilla · 28. Oktober 2020

Hi zusammen,
ich möchte bestimmte Messwerte an verschiedenen Stellen überwachen und habe dafür ESP8266 Module verteilt die die Messwerte per MQTT Protokoll an einen Node Red mit Aedes MQTT Broker übertragen. Das ganze funktioniert auch einwandfrei. Jedoch sind die Daten die man im Dashboard grafisch sieht nach dem Neustart des Containers weg.

Daher wollte ich die MQTT Daten persistieren. Aedes hat dafür wohl ein Modul um sich mit einer MongoDB zu verbinden.
Ich habe also per Docker-Compose noch einen Mongo Container gestartet der auch grundsätzlich von NodeRed Container erreichbar ist und Daten abspeichern kann. (Getestet mit node-red-mongodb mongodb-in Modul und Timestamp aus einem Test-Flow)

Leider scheint Aedes aber gar nicht zu versuchen etwas abzuspeichern.

mongo_aedes ist der Service name im docker-compose und dort ist auch die MongoDB erreichbar.
Zu dem Thema findet man leider immer nur sehr rudimentäre Videos/Tutorials. Hat jemand eine Idee woran es scheitern könnte?

snaxilian · 28. Oktober 2020

1. Was sagt das Log? (docker-compose logs im Verzeichnis aufrufen wo die docker-compose.yml liegt...)
2. Bist du dir wirklich sicher, dass der aedes-mqtt-broker keine Möglichkeit der Persistenz hat und du beispielsweise nur vergessen hast dem Container entsprechend ein Volume bereit zu stellen?
2.1. Falls nein hoffe ich, dass du an das Volume bei der mongodb gedacht hast sonst sind die Daten dort ebenfalls weg nach Neustart des Containers.

jokakilla · 28. Oktober 2020

Der NodeRed persistiert die Flows, Dashboard usw in seinem Daten Volume. Für den Mongo gibt es ein Volume für das Datenbankverzeichnis.
Docker-compose logs geht leider nicht aus der Console weil ich die Compose Datei über Portainer bei Stacks eingerichtet habe. Der Logs vom MongoDB container sieht unauffällig aus.
Ping vom NodeRed auf mongo_aedes löst auf die passende IP auf und wird beantwortet.

Die docker-compose.yml

Code:

version: '3.1'

volumes:
  mongo_data:
  node_red_data:
    external: true

services:
  nodered:
    image: nodered/node-red:latest
    ports:
      - 8883:8883
      - 1880:1880
    restart: always
    volumes:
      - node_red_data:/data
  mongo_aedes:
    image: mongo:latest
    restart: always
    environment:
      MONGO_INITDB_DATABASE: mqtt
    volumes:
      - mongo_data:/data/db

Die Container im entsprechenden Netz

MongoDB Log nach Containerstart:

Code:

{"t":{"$date":"2020-10-28T13:16:22.343+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"},
{"t":{"$date":"2020-10-28T13:16:22.347+00:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"},
{"t":{"$date":"2020-10-28T13:16:22.347+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."},
{"t":{"$date":"2020-10-28T13:16:22.348+00:00"},"s":"I",  "c":"STORAGE",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"68e5bc0f42d3"}},
{"t":{"$date":"2020-10-28T13:16:22.348+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.1","gitVersion":"ad91a93a5a31e175f5cbf8c69561e788bbc55ce1","openSSLVersion":"OpenSSL 1.1.1  11 Sep 2018","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu1804","distarch":"x86_64","target_arch":"x86_64"}}}},
{"t":{"$date":"2020-10-28T13:16:22.348+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"18.04"}}},
{"t":{"$date":"2020-10-28T13:16:22.348+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"}}}},
{"t":{"$date":"2020-10-28T13:16:22.349+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/data/db","storageEngine":"wiredTiger"}},
{"t":{"$date":"2020-10-28T13:16:22.349+00:00"},"s":"I",  "c":"STORAGE",  "id":22297,   "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]},
{"t":{"$date":"2020-10-28T13:16:22.349+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=3435M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}},
{"t":{"$date":"2020-10-28T13:16:22.974+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1603890982:974257][1:0x7fbdbe023a80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 1 through 2"}},
{"t":{"$date":"2020-10-28T13:16:23.056+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1603890983:56554][1:0x7fbdbe023a80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 2 through 2"}},
{"t":{"$date":"2020-10-28T13:16:23.133+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1603890983:132996][1:0x7fbdbe023a80], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Main recovery loop: starting at 1/96384 to 2/256"}},
{"t":{"$date":"2020-10-28T13:16:23.228+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1603890983:228916][1:0x7fbdbe023a80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 1 through 2"}},
{"t":{"$date":"2020-10-28T13:16:23.290+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1603890983:290923][1:0x7fbdbe023a80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 2 through 2"}},
{"t":{"$date":"2020-10-28T13:16:23.337+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1603890983:337295][1:0x7fbdbe023a80], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global recovery timestamp: (0, 0)"}},
{"t":{"$date":"2020-10-28T13:16:23.337+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1603890983:337345][1:0x7fbdbe023a80], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global oldest timestamp: (0, 0)"}},
{"t":{"$date":"2020-10-28T13:16:23.345+00:00"},"s":"I",  "c":"STORAGE",  "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":996}},
{"t":{"$date":"2020-10-28T13:16:23.345+00:00"},"s":"I",  "c":"RECOVERY", "id":23987,   "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}},
{"t":{"$date":"2020-10-28T13:16:23.348+00:00"},"s":"I",  "c":"STORAGE",  "id":22262,   "ctx":"initandlisten","msg":"Timestamp monitor starting"},
{"t":{"$date":"2020-10-28T13:16:23.350+00:00"},"s":"W",  "c":"CONTROL",  "id":22120,   "ctx":"initandlisten","msg":"Access control is not enabled for the database. Read and write access to data and configuration is unrestricted","tags":["startupWarnings"]},
{"t":{"$date":"2020-10-28T13:16:23.350+00:00"},"s":"W",  "c":"CONTROL",  "id":22178,   "ctx":"initandlisten","msg":"/sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'","tags":["startupWarnings"]},
{"t":{"$date":"2020-10-28T13:16:23.354+00:00"},"s":"I",  "c":"STORAGE",  "id":20536,   "ctx":"initandlisten","msg":"Flow Control is enabled on this deployment"},
{"t":{"$date":"2020-10-28T13:16:23.355+00:00"},"s":"I",  "c":"FTDC",     "id":20625,   "ctx":"initandlisten","msg":"Initializing full-time diagnostic data capture","attr":{"dataDirectory":"/data/db/diagnostic.data"}},
{"t":{"$date":"2020-10-28T13:16:23.357+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"/tmp/mongodb-27017.sock"}},
{"t":{"$date":"2020-10-28T13:16:23.357+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"0.0.0.0"}},
{"t":{"$date":"2020-10-28T13:16:23.357+00:00"},"s":"I",  "c":"NETWORK",  "id":23016,   "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}},
{"t":{"$date":"2020-10-28T13:16:24.178+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.0.6.4:37100","connectionId":1,"connectionCount":1}},
{"t":{"$date":"2020-10-28T13:16:24.180+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1","msg":"client metadata","attr":{"remote":"10.0.6.4:37100","client":"conn1","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}},
{"t":{"$date":"2020-10-28T13:16:25.406+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.0.6.4:37102","connectionId":2,"connectionCount":2}},
{"t":{"$date":"2020-10-28T13:16:25.408+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn2","msg":"client metadata","attr":{"remote":"10.0.6.4:37102","client":"conn2","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}},
{"t":{"$date":"2020-10-28T13:16:25.859+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.0.6.4:37104","connectionId":3,"connectionCount":3}},
{"t":{"$date":"2020-10-28T13:16:25.862+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn3","msg":"client metadata","attr":{"remote":"10.0.6.4:37104","client":"conn3","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}},
{"t":{"$date":"2020-10-28T13:16:26.381+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.0.6.4:37106","connectionId":4,"connectionCount":4}},
{"t":{"$date":"2020-10-28T13:16:26.383+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn4","msg":"client metadata","attr":{"remote":"10.0.6.4:37106","client":"conn4","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}},
{"t":{"$date":"2020-10-28T13:16:27.853+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.0.6.4:37108","connectionId":5,"connectionCount":5}},
{"t":{"$date":"2020-10-28T13:16:27.854+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn5","msg":"client metadata","attr":{"remote":"10.0.6.4:37108","client":"conn5","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}},

Log vom NodeRed Container startup:

Code:

28 Oct 12:46:33 - [info] [mqtt-broker:ba471952.81c848] Verbindung zum Broker mqtts://localhost:8883 hergestellt.,

28 Oct 12:46:33 - [info] [aedes broker:Secure MQTT Broker] Start persistence to MongeDB,

28 Oct 12:46:33 - [info] Flows gestartet,

28 Oct 12:46:33 - [info] [aedes broker:Secure MQTT Broker] Binding aedes mqtt server on port: 8883

Ergänzung (28. Oktober 2020)

Der Test Flow mit einem Mongo Out speicher fröhlich Testdaten (Timestamp) in der Mongo ab.

snaxilian · 31. Oktober 2020

Dann hör halt auf irgendwelche Klickibunti-Tools wie Portainer zu nutzen und starte deine Umgebung abseits davon mal direkt auf der Shell dann kommst auch an die Logs ran oder finde heraus wie du mit Portainer die Logs bekommst oder willst du uns erzählen, dass Portainer dies nicht kann?^^

jokakilla · 31. Oktober 2020

Warum gleich so unfreundlich?
Portainer liefert halt nur die Logs pro Service aber nicht aggregiert.

Hier noch die docker-compose logs Ausgabe:

Code:

Attaching to root_mongo_aedes_1, root_nodered_1
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.542+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.545+00:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.546+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.546+00:00"},"s":"I",  "c":"STORAGE",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"8ab6df2ac3f3"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.546+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.1","gitVersion":"ad91a93a5a31e175f5cbf8c69561e788bbc55ce1","openSSLVersion":"OpenSSL 1.1.1  11 Sep 2018","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu1804","distarch":"x86_64","target_arch":"x86_64"}}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.546+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"18.04"}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.546+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"}}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.547+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/data/db","storageEngine":"wiredTiger"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.547+00:00"},"s":"I",  "c":"STORAGE",  "id":22297,   "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:54.547+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=3435M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.175+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1604150515:175113][1:0x7f65e912da80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 2 through 3"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.229+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1604150515:229546][1:0x7f65e912da80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 3 through 3"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.314+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1604150515:314435][1:0x7f65e912da80], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Main recovery loop: starting at 2/4096 to 3/256"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.432+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1604150515:432648][1:0x7f65e912da80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 2 through 3"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.502+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1604150515:502693][1:0x7f65e912da80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 3 through 3"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.562+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1604150515:562007][1:0x7f65e912da80], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global recovery timestamp: (0, 0)"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.562+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1604150515:562076][1:0x7f65e912da80], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global oldest timestamp: (0, 0)"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.605+00:00"},"s":"I",  "c":"STORAGE",  "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":1058}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.605+00:00"},"s":"I",  "c":"RECOVERY", "id":23987,   "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.608+00:00"},"s":"I",  "c":"STORAGE",  "id":22262,   "ctx":"initandlisten","msg":"Timestamp monitor starting"}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.610+00:00"},"s":"W",  "c":"CONTROL",  "id":22120,   "ctx":"initandlisten","msg":"Access control is not enabled for the database. Read and write access to data and configuration is unrestricted","tags":["startupWarnings"]}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.610+00:00"},"s":"W",  "c":"CONTROL",  "id":22178,   "ctx":"initandlisten","msg":"/sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'","tags":["startupWarnings"]}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.613+00:00"},"s":"I",  "c":"STORAGE",  "id":20536,   "ctx":"initandlisten","msg":"Flow Control is enabled on this deployment"}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.615+00:00"},"s":"I",  "c":"FTDC",     "id":20625,   "ctx":"initandlisten","msg":"Initializing full-time diagnostic data capture","attr":{"dataDirectory":"/data/db/diagnostic.data"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.617+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"/tmp/mongodb-27017.sock"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.617+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"0.0.0.0"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:55.617+00:00"},"s":"I",  "c":"NETWORK",  "id":23016,   "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:56.958+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.21.0.2:39814","connectionId":1,"connectionCount":1}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:56.967+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1","msg":"client metadata","attr":{"remote":"172.21.0.2:39814","client":"conn1","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.028+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.21.0.2:39816","connectionId":2,"connectionCount":2}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.029+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn2","msg":"client metadata","attr":{"remote":"172.21.0.2:39816","client":"conn2","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.040+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.21.0.2:39818","connectionId":3,"connectionCount":3}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.056+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn3","msg":"client metadata","attr":{"remote":"172.21.0.2:39818","client":"conn3","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.076+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.21.0.2:39820","connectionId":4,"connectionCount":4}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.076+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.21.0.2:39822","connectionId":5,"connectionCount":5}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.076+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.21.0.2:39824","connectionId":6,"connectionCount":6}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.078+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn4","msg":"client metadata","attr":{"remote":"172.21.0.2:39820","client":"conn4","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.079+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn5","msg":"client metadata","attr":{"remote":"172.21.0.2:39822","client":"conn5","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}}
mongo_aedes_1  | {"t":{"$date":"2020-10-31T13:21:57.079+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn6","msg":"client metadata","attr":{"remote":"172.21.0.2:39824","client":"conn6","doc":{"driver":{"name":"nodejs","version":"3.5.7"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.0-12-amd64"},"platform":"'Node.js v10.22.1, LE (unified)"}}}
nodered_1      |
nodered_1      | > node-red-docker@1.2.2 start /usr/src/node-red
nodered_1      | > node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"
nodered_1      |
nodered_1      | 31 Oct 13:21:55 - [info]
nodered_1      |
nodered_1      | Welcome to Node-RED
nodered_1      | ===================
nodered_1      |
nodered_1      | 31 Oct 13:21:55 - [info] Node-RED version: v1.2.2
nodered_1      | 31 Oct 13:21:55 - [info] Node.js  version: v10.22.1
nodered_1      | 31 Oct 13:21:55 - [info] Linux 4.19.0-12-amd64 x64 LE
nodered_1      | 31 Oct 13:21:55 - [info] Loading palette nodes
nodered_1      | 31 Oct 13:21:56 - [info] Dashboard up and running
nodered_1      | 31 Oct 13:21:56 - [info] Dashboard version 2.23.4 started at /ui
nodered_1      | 31 Oct 13:21:56 - [info] Settings file  : /data/settings.js
nodered_1      | 31 Oct 13:21:56 - [info] Context store  : 'default' [module=memory]
nodered_1      | 31 Oct 13:21:56 - [info] User directory : /data
nodered_1      | 31 Oct 13:21:56 - [warn] Projects disabled : editorTheme.projects.enabled=false
nodered_1      | 31 Oct 13:21:56 - [info] Flows file     : /data/flows.json
nodered_1      | 31 Oct 13:21:56 - [warn]
nodered_1      |
nodered_1      | ---------------------------------------------------------------------
nodered_1      | Your flow credentials file is encrypted using a system-generated key.
nodered_1      |
nodered_1      | If the system-generated key is lost for any reason, your credentials
nodered_1      | file will not be recoverable, you will have to delete it and re-enter
nodered_1      | your credentials.
nodered_1      |
nodered_1      | You should set your own key using the 'credentialSecret' option in
nodered_1      | your settings file. Node-RED will then re-encrypt your credentials
nodered_1      | file using your chosen key the next time you deploy a change.
nodered_1      | ---------------------------------------------------------------------
nodered_1      |
nodered_1      | 31 Oct 13:21:56 - [info] Starting flows
nodered_1      | 31 Oct 13:21:56 - [info] [aedes broker:Secure MQTT Broker] Start persistence to MongeDB
nodered_1      | 31 Oct 13:21:56 - [info] Started flows
nodered_1      | 31 Oct 13:21:56 - [info] [aedes broker:Secure MQTT Broker] Binding aedes mqtt server on port: 8883
nodered_1      | 31 Oct 13:21:56 - [info] Server now running at http://127.0.0.1:1880/
nodered_1      | 31 Oct 13:21:57 - [info] [mqtt-broker:ba471952.81c848] Connected to broker: mqtts://localhost:8883

snaxilian · 31. Oktober 2020

jokakilla schrieb:
Warum gleich so unfreundlich?

Du überinterpretierst und verwechselst sachliche Knappheit mit Unfreundlichkeit

Aedes benötigt ein extra Plugin für persistente Daten: https://github.com/moscajs/aedes#persistence und ich habe zumindest beim groben überfliegen des Dockerfiles von NodeRed nirgends einen Hinweis darauf gefunden, dass das entsprechende npm package mit installiert wird. Wobei es ggf. als Abhängigkeit mitgekommen sein könnte oder so. Zumindest wäre das ein Ansatz dem ich nachgehen würde.

jokakilla · 1. November 2020

Danke...guter Hinweis. Ich habe versucht das Persistenz Plugin noch über npm nachzuinstallieren. Das hat leider nix geändert.
https://github.com/martin-doyle/node-red-contrib-aedes/blob/master/package.json
Scheint als würde er das als Abhängigkeit angeben.

Ich glaub ich schau mal in den Source

https://github.com/martin-doyle/node-red-contrib-aedes/blob/master/aedes.js

jokakilla · 1. November 2020

🤦‍♂️
Der MQTT Client (https://github.com/knolleary/pubsubclient) setzt das retain flag per default auf false wenn man nichts anderes angibt.
Die Persistenz initialisiert die Datenbank offenbar lazy und speichert Daten auch nur wenn das retain flag auf true ist.

So hab ich jetzt nach dem Neustart zwar keine Timeline mehr aber wenigstens noch den letzten Wert der vor dem Neustart gesendet wurde.

snaxilian · 1. November 2020

Oder du nimmst das bestehende Dockerfile und erweiterst es um das fehlende npm package und baust dir daraus eben dein Image inkl. mongodb Anbindung. Aber dann hast bei jeder neuen Version ggf. mehr Aufwand falls sich etwas größeres ändert und ggf. mit deinen Settings kollidiert.

Musst halt abwägen ob es dir das wert ist^^

Suche

Node Red Aedes MQTT Broker Persistenz in MongoDB

jokakilla

Lt. Junior Grade

snaxilian

Admiral

jokakilla

Lt. Junior Grade

snaxilian

Admiral

jokakilla

Lt. Junior Grade

snaxilian

Admiral

jokakilla

Lt. Junior Grade

jokakilla

Lt. Junior Grade

snaxilian

Admiral