Elektroda.com
Elektroda.com
X

[Tutorial] Provisioning using HomeAssistant

nielspiersma 2877 25
  • OpenBeken Provisioning.
    As an enthusiastic user of HomeAssistant and Smart Home solutions in general, I am always into something new. Recently I’ve become acquainted with OpenBeken. Not wholly voluntary, but is a response to the fact that more Smart Home Devices started using BK7231 vs. good-old ESP82xx.

    Undoubtedly I like the fact of being able to log in to a device and configure it to my needs. However, from a manageability perspective, you need the ability to mass-configure devices. When looking at phone provisioning and large-scale OS deployments, provisioning is standard. There is no way you can log in to 200 web interfaces and set individual settings.

    Luckily for a few weeks, OpenBeken has introduced the “MQTT Group Topic,” which has eased our life tremendously.
    Furthermore, the OpenBeken developers are actively adding configuration parameters with the ability to set them (up) using the MQTT “cmnd”. Combined with the “MQTT Group Topic,” I can now quickly configure multiple devices.

    So why is this important to us? Well, let me start by using our own office as an example. Everyone knows about the high costs of electricity and the new rules coming from the EU regarding Co2 emissions and taxes. Saving on electricity consumption is directly impacting these costs. In our office, we have ceiling lights. About 25 of them. Each is equipped with 8 led lamps but still consumes around 100 Watts when lit. Multiply that by 25, and you have a whopping 2500 watts. We found that, on average, we only need ten ceiling lights powered on at any given moment is fine. Thus reducing our average usage by more than 50%. All our ceiling lights were connected to standard earthed sockets. Reason enough to buy 30 or so Smart Sockets. Initially, I tested the Woox R6080, and after sacrificing one unit, I got it running with OpenBeken. At about the same time, I successfully managed to get cloud-cutter to work, which made my work even more straightforward. It took away the need to desolder 25 Tuya boards.

    I flashed all 34 Smart Plugs in two evenings. So now my real problems have begun. How to configure all these babies in bulk. I wasn’t planning on logging on to each device setting the required settings. So I started playing around with the same commands I used for bulk configuring my Tasmota devices. Unfortunately, not all the commands I needed were implemented at that time. Thanks to the developers, quite some features were added. But things really got up to speed once the MQTT Group Topic was introduced. This made it possible for me to update all 34 plugs in one go. As well as fine-tuning. This is a breeze as I have the same plugs and the same underlying hardware.

    Let me show how I’ve done this;
    Flash your device and add it to your WiFi network. No pin settings are needed; we will configure all settings from HomeAssistant.
    First, I recommend choosing an MQTT Group Topic based on your hardware chip. This will make updating relatively easy and at least somehow prevent you from pushing N setting to T chips and vice versa. Setting up the MQTT Group Topic is done under “Configure MQTT.” Please make sure you set up your username and password correctly so HomeAssistant can “take control.”

    [Tutorial] Provisioning using HomeAssistant

    To follow up on this, your Beken Device must report its status (regularly) to HomeAssistant. I recommend setting it up with a special Startup command. This is done in the Change Startup command text.

    You should add “scheduleHADiscovery 10”

    [Tutorial] Provisioning using HomeAssistant

    Now reboot your OpenBeken device and check if it shows up in HomeAssistant

    [Tutorial] Provisioning using HomeAssistant

    We are now ready to configure the device.
    Firmware update example.
    Now that your device is integrated in HomeAssistent, you might want to update it remotely. Download the latest firmware and store it in a repository accessible via http. This can be a any type of http server. Let us assume your file resides on a local server with IP address 192.168.1.66 and is called fw.rbl. The URL for the update would http://192.168.1.6/fw.rbl. In HomeAssistant create a new script with the following settings;

    [Tutorial] Provisioning using HomeAssistant

    For completeness the yaml code;

    alias: OpenBeken Firmware Update
    sequence:
    - service: mqtt.publish
    data:
    topic: cmnd/BK7321N/ota_http
    payload: http://192.168.1.66/fw.rbl
    mode: single
    icon: mdi:cellphone-arrow-down



    When you run the script, it will request all OpenBeken devices subscribed to “BK732N” to download new firmware from the web server.
    I want to focus on the “topic” and “payload,” as this is where the actual magic happens.
    cmnd/BK7321N/ota_http
    • cmnd instructs sending a command.
    • BK7321N is the MQTT (group) Topic. If you want an individual update, use the MQTT Client/device name
    • ota_http is the command your want to send. You can use the commands from https://github.com/openshwprojects/OpenBK7231T_App/blob/main/docs/commands.md and from https://github.com/openshwprojects/OpenBK7231T_App/blob/main/docs/commands-extended.md
    I can tell, it is fun to watch 35 devices updating and rebooting. The aforementioned “scheduleHADiscovery” warrants that your HomeAssistant information is updated. When a new IP address is given or the firmware version changes, this is correctly displayed in HomeAssistant. So you can keep track of the firmware versions and see which devices are not up to date.
    Another example is setting the Flags. Sometimes is handy to have flags set. For instance, I like flag 6 being set. This makes sure all relays are triggered instantly. I also have the desire setting flag 10. Setting both flags will give us a flag value of 1088.
    Again, we will create a script that updates the flag settings.
    [Tutorial] Provisioning using HomeAssistant

    Again small breakdown:
    cmnd/bekens/ota_http
    • cmnd instructs sending a command.
    • bekens is the MQTT (group) Topic.
    • flags the command we want to send
    • payload 1088 in the case representing Flag 6 and Flag 10
    Run the script and all your devices in the MQTT Group “bekens” will get these injected.
    We can also setup the device using the backlog command. This allows us to send multiple commands to a device. Quite handy for setting up the configuration.

    For example, setting up the Action LSC Smart Plugs
    Pins are empty:
    [Tutorial] Provisioning using HomeAssistant

    The device has Wifi set up, has MQTT Group setup, has scheduleHADiscovery set as startup command, and the device has MQTT host, user, and password setup.

    Now configure all pins.

    [Tutorial] Provisioning using HomeAssistant

    Yaml code:
    o service: mqtt.publish
    o data:
    o topic: cmnd/obkBF518BE1/backlog
    o payload: SetPinRole 6 LED_n;SetPinChannel 6,1;SetPinRole 7 Btn;SetPinChannel 7,1;SetPinRole 8 rel; SetPinChannel 8,1;SetPinRole 26 WifiLED; SetPinChannel 26,0
    Run the script and:

    [Tutorial] Provisioning using HomeAssistant

    Pins setup.
    This shows the power of the MQTT Group Topic and the way the OpenBeken developers have come making provisioning a breeze. I am quite sure any future command or setting will be addressable via a MQTT cmnd command.
    You can also play around with HomeAssistant templates, giving you even more power on the commands you want to execute.
    I hope you enjoyed this write up on provisioning on OpenBeken.

    Cool? Ranking DIY
    About Author
    nielspiersma
    Level 6  
    Offline 
    nielspiersma wrote 39 posts with rating 6. Been with us since 2022 year.
  • #2
    p.kaczmarek2
    Moderator Smart Home
    Very good writeup. It will be certainly useful for people, even I can learn new interesting things from that.

    I will just add here two more notes.
    1. If you need anything else (a command, a feature, or a bufix) - message me, I will try to help.
    2. OpenBeken, as you may probably know, supports BK7231T, BK7231N, BL602, W800/W801, W600/W601, XR809, T34 and... and Windows
    Yes, you read right - you can run OpenBeken on Windows, along with IoT device simulator/schematic draw tool and full MQTT support:
    [Tutorial] Provisioning using HomeAssistant
    [Tutorial] Provisioning using HomeAssistant
    [Tutorial] Provisioning using HomeAssistant
    What this mean for end user, you may ask? Well, it means that soon you will be able to test all MQTT stuff, all scripting, automation, etc, without even having an IoT device! Just Home Assistant Virtual Machine + OBK simulator will suffice.
    OBK simulator will be released soon, no binary is yet available.

    Being able to run OBK on Windows also speeds up development process a lot.

    PS: In a meantime, I have also added a LFS_* commands for writing files to our LittleFS file system.
  • #3
    nielspiersma
    Level 6  
    That is so good to hear. Let me know once the emulator is available. Would love to test that. Especially testing my mqtt provisioning .


    Cheers
    Niels
  • #4
    ferbulous
    Level 15  
    Hi, thanks for sharing this method
    Followed the instructions but it's not triggering the ota firmware download on my devices
    Using script and just testing with developer tools

    [Tutorial] Provisioning using HomeAssistant

    [Tutorial] Provisioning using HomeAssistant

    Maybe I'm missing something?
  • #5
    p.kaczmarek2
    Moderator Smart Home
    You must restart your device in order for GroupTopic to work.
  • #6
    ferbulous
    Level 15  
    @p.kaczmarek2 hi, i also restarted the device but it's not working yet
  • #7
    nielspiersma
    Level 6  
    Correct,

    After setting up mqtt group topic you need to restart the device.

    Niels

    Added after 4 [minutes]:

    I am not sure, but I only tested with port 80 not with port 7000.

    I'll setup a server on port 7000 and will test it

    In them mean time, van you provide a tcpdump or Wireshark capture from your web server? Confirming the http request is actually hitting the web service?

    Niels
  • #8
    ferbulous
    Level 15  
    @nielspiersma i'm using http-here to share the files

    root@rpi4:/home/pi/firmware# http-here --host 0.0.0.0 7200


    when i ran the mqtt publish, no changes in the openbk console/tcpdump
    but if I just copy and paste the link in browser, it downloads the firmware and this is the log (assuming i'm using tcpdump right)

    root@rpi4:/home/pi# tcpdump -i any -c5 -nn port 7200
    tcpdump: data link type LINUX_SLL2
    tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
    listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144                                                                              bytes
    01:21:28.030050 wlan0 In  IP 192.168.1.32.57501 > 192.168.1.40.7200: Flags [S],                                                                              seq 1383112925, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], lengt                                                                             h 0
    01:21:28.030178 wlan0 Out IP 192.168.1.40.7200 > 192.168.1.32.57501: Flags [S.],                                                                              seq 217126541, ack 1383112926, win 64240, options [mss 1460,nop,nop,sackOK,nop,                                                                             wscale 7], length 0
    01:21:28.030327 wlan0 In  IP 192.168.1.32.57502 > 192.168.1.40.7200: Flags [S],                                                                              seq 3285208169, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], lengt                                                                             h 0
    01:21:28.030366 wlan0 Out IP 192.168.1.40.7200 > 192.168.1.32.57502: Flags [S.],                                                                              seq 1245807257, ack 3285208170, win 64240, options [mss 1460,nop,nop,sackOK,nop                                                                             ,wscale 7], length 0
    01:21:28.039814 wlan0 In  IP 192.168.1.32.57501 > 192.168.1.40.7200: Flags [.],                                                                              ack 1, win 513, length 0
    5 packets captured
    7 packets received by filter
    0 packets dropped by kernel
    root@rpi4:/home/pi#
  • #9
    nielspiersma
    Level 6  
    Your mqtt url is connecting to port 7000.
    Your tcpdump shows port 7200.

    Can you confirm the ports you are using?

    Niels
  • #10
    ferbulous
    Level 15  
    @nielspiersma it's 7200 now
    and i've the changed the mqtt publish to use 7200

    http://192.168.1.40:7200/OpenBK7231N_1.15.318.rbl
    http://192.168.1.40:7200/fw.rbl

    do i need to leave the original firmware as or just rename it?
    not working just yet with both filenames tested (if it's related)

    
    alias: Openbeken firmware update N
    sequence:
      - service: mqtt.publish
        data:
          topic: cmnd/BK7321N/ota_http
          payload: http://192.168.1.40:7200/fw.rbl
    mode: single
    icon: mdi:briefcase-download
    
  • #11
    nielspiersma
    Level 6  
    Both in the home assistant script and on your http server?

    Added after 52 [minutes]:

    The file name just has to match.

    I named it fw.rbl so it was easy.

    Just rename it to fw.rbl so it matches the url.

    I'll update my post later so it is more clear


    Niels
  • #12
    p.kaczmarek2
    Moderator Smart Home
    Please try debugging the issue to determine where exactly is your point of failure.
    1. Please try to broadcast a "POWER toggle" command using the same method and topic from HA and check if it works
    2. Please try using the OTA command with your URL directly into OBK Web app console and see what happens
    When diagnosing a problem, it's always good to check your process step by step and determine where exactly things starts to go wrong.

    Btw: windows simulator almost ready, with full MQTT support.
  • Helpful post
    #13
    ferbulous
    Level 15  
    @p.kaczmarek2

    Noted, will try a few other mqtt commands once i home to test it out

    Added after 5 [hours] 28 [minutes]:

    Looks like it's working now, not sure why it wasn't working earlier

    
    
    root@rpi4:/home/pi/firmware# http-here --host 0.0.0.0 7200
    Serving HTTP on 0.0.0.0:7200...
    192.168.1.179 - - [14/Jan/2023 14:59:00] "GET /fw.rbl HTTP/1.1" 200 -
    192.168.1.166 - - [14/Jan/2023 14:59:35] "GET /fw.rbl HTTP/1.1" 200 -
    192.168.1.43 - - [14/Jan/2023 15:00:08] "GET /fw.rbl HTTP/1.1" 200 -
    192.168.1.247 - - [14/Jan/2023 15:00:35] "GET /fw.rbl HTTP/1.1" 200 -
    192.168.1.162 - - [14/Jan/2023 15:01:03] "GET /fw.rbl HTTP/1.1" 200 -
    192.168.1.105 - - [14/Jan/2023 15:01:37] "GET /fw.rbl HTTP/1.1" 200 -
    192.168.1.226 - - [14/Jan/2023 15:02:05] "GET /fw.rbl HTTP/1.1" 200 -
    192.168.1.171 - - [14/Jan/2023 15:02:33] "GET /fw.rbl HTTP/1.1" 200 -
    
  • Helpful post
    #14
    p.kaczmarek2
    Moderator Smart Home
    @ferbulous you need a restart after MQTT group name change and you had a typo in port or something, if I've seen correctly.

    I think I will add some kind of notification about MQTT params restart requirement.
  • #15
    chri5k
    Level 5  
    I am also having difficulty getting the OTA update to trigger via MQTT. I have tried several variations of the strings and rebooted the module between each change to the MQTT configuration of the module. I was able to get the update to work using the module web app so it seems the URL / web server are working to correctly.

    Here is my YAML in HA
    alias: OpenBeken Firmware Update
    sequence:
    - service: mqtt.publish
    data:
    topic: cmnd/BK7321T/ota_http
    payload: http://192.168.4.200/OpenBK7231T_1.15.573.rbl
    mode: single
    icon: mdi:cellphone-arrow-down

    Here is my current MQTT configuration
    [Tutorial] Provisioning using HomeAssistant

    Here is the output from MQTT Explorer. This MQTT browser shows the MQTT messages in realtime.
    [Tutorial] Provisioning using HomeAssistant

    [Tutorial] Provisioning using HomeAssistant
  • #16
    p.kaczmarek2
    Moderator Smart Home
    Hello, there are two things you can do.

    1. replace command in your script to see if it's received correctly:
    
    sequence:
      - service: mqtt.publish
        data:
          topic: cmnd/BK7321T/POWER
          payload: Toggle
    

    Does it toggle power? If so, then communication of command to BK is ok.

    2. Execute the command you gave in "Execute console command" field to see if the HTTP address is reachable by HTTP OTA.

    Doing those two things will tell you where it exactly breaks and what is the problem.

    Added after 36 [seconds]:

    Btw, I will try on my devices in a moment to double check your syntax.

    Added after 24 [minutes]:

    OK I am testing...
    Device:
    [Tutorial] Provisioning using HomeAssistant
    [Tutorial] Provisioning using HomeAssistant
    HTTP server:
    [Tutorial] Provisioning using HomeAssistant
    File on htdocs:
    [Tutorial] Provisioning using HomeAssistant
    Used command:
    
    ota_http http://192.168.0.118/OpenBK7231T_1.15.576.rbl
    

    Before sending from HA:
    [Tutorial] Provisioning using HomeAssistant
    In Web App Log, I can see device began OTA:
    [Tutorial] Provisioning using HomeAssistant [Tutorial] Provisioning using HomeAssistant
    Device is back with latest version!
    [Tutorial] Provisioning using HomeAssistant
  • #18
    p.kaczmarek2
    Moderator Smart Home
    Power toggle does not work for you? Wait, it works for me, look:
    [Tutorial] Provisioning using HomeAssistant
    And look, Web App log says it was received:
    [Tutorial] Provisioning using HomeAssistant

    let me check with HA sequence code now...
  • #19
    chri5k
    Level 5  
    Neither POWER Toggle, On or Off commands work. I can see the commands published in MQTT. The module log does not show it read the command from MQTT. I can go into Homeassistant and and control the lights on or off via MQTT. For some reason the modules are not picking up the commands from the cmnd topic.

    I get no logs when executing the toggle script in HA.

    I do get logs when I control the lights via MQTT in HA.
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/1/set
    Info:MQTT:MQTT in topic kennelext/1/set
    Info:MQTT:MQTT topic not handled: kennelext/1/set
    Info:MQTT:MQTT client in mqtt_incoming_data_cb data is 1 for ch 1
    Info:MQTT:Channel has changed! Publishing 1 to channel 1
    Info:MQTT:Publishing val 1 to kennelext/1/get retain=0
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/1/get
    Info:MQTT:MQTT in topic kennelext/1/get
    Info:MQTT:MQTT topic not handled: kennelext/1/get
    Info:MQTT:channelSet NOT 'set'
    Info:MQTT:Channel has changed! Publishing 1 to channel 4
    Info:MQTT:Publishing val 1 to kennelext/4/get retain=0
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/4/get
    Info:MQTT:MQTT in topic kennelext/4/get
    Info:MQTT:MQTT topic not handled: kennelext/4/get
    Info:MQTT:channelSet NOT 'set'
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/1/set
    Info:MQTT:MQTT in topic kennelext/1/set
    Info:MQTT:MQTT topic not handled: kennelext/1/set
    Info:MQTT:MQTT client in mqtt_incoming_data_cb data is 0 for ch 1
    Info:MQTT:Channel has changed! Publishing 0 to channel 1
    Info:MQTT:Publishing val 0 to kennelext/1/get retain=0
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/1/get
    Info:MQTT:MQTT in topic kennelext/1/get
    Info:MQTT:MQTT topic not handled: kennelext/1/get
    Info:MQTT:channelSet NOT 'set'
    Info:MQTT:Channel has changed! Publishing 0 to channel 4
    Info:MQTT:Publishing val 0 to kennelext/4/get retain=0
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/4/get
    Info:MQTT:MQTT in topic kennelext/4/get
    Info:MQTT:MQTT topic not handled: kennelext/4/get
    Info:MQTT:channelSet NOT 'set'
  • #20
    p.kaczmarek2
    Moderator Smart Home
    Do you have the same version as me? Maybe there was a bug... but I never experienced anything like that.

    Have you tried rebooting your device?

    Btw, this also works for me:
    Code: yaml
    Log in, to see the code

    [Tutorial] Provisioning using HomeAssistant
    https://www.home-assistant.io/integrations/button.mqtt/


    Btw...


    chri5k wrote:
    I can go into Homeassistant and and control the lights on or off via MQTT. For some reason the modules are not picking up the commands from the cmnd topic.

    Are you aware that the cmnd topic is used to turn lights on or off via MQTT in Home Assistant Discovery?
    Yes, it's done by cmnd/OBKsomething/led_enableAll 1 or 0....
    So the cmnd topic is somehow working at least in that case.
    It has to. If you are able to turn lights ON or OFF...

    Added after 7 [minutes]:

    EDIT:
    I see you posted this log:
    
    I do get logs when I control the lights via MQTT in HA.
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/1/set
    Info:MQTT:MQTT in topic kennelext/1/set
    Info:MQTT:MQTT topic not handled: kennelext/1/set
    Info:MQTT:MQTT client in mqtt_incoming_data_cb data is 1 for ch 1
    Info:MQTT:Channel has changed! Publishing 1 to channel 1
    Info:MQTT:Publishing val 1 to kennelext/1/get retain=0
    Info:MQTT:MQTT client in mqtt_incoming_publish_cb topic kennelext/1/get
    Info:MQTT:MQTT in topic kennelext/1/get
    Info:MQTT:MQTT topic not handled: kennelext/1/get
    Info:MQTT:channelSet NOT 'set'
    

    This log looks like a SWITCH control, not LED light.

    Can you tell us whether your device is a LED light (and uses cmnd/obkLED123/led_basecolor_rgb) or is it a SWITCH?

    Added after 1 [minutes]:

    @chri5k can you update your device via Web App OTA (drag and drop RBL file) and recheck if they receive CMND topic now?

    I think you have a bit old version that was missing something...
    use this OTA:
    [Tutorial] Provisioning using HomeAssistant
  • #21
    chri5k
    Level 5  
    Interesting...

    When I try the update from an HA script it does not work. When I try it from an MQTT button as you suggest it works. Same thing with the POWER TOGGLE command. What is really odd is according to the MQTT browser it is two different topics. On screen the two topics look identical. Seems like something in the way HA scripting publishes MQTT topics creates a topic OBK can not read. Also it causes the MQTT browser to see it as a different topic. The top BK7231T topic in the picture is from the script and the one below it is from the button.

    Oh well, I can do OBK updates from a button just as easily as a script.

    [Tutorial] Provisioning using HomeAssistant
  • #22
    p.kaczmarek2
    Moderator Smart Home
    Maybe text encoding issue? Can you check your strings with some kind of UTF inspector?

    Can you try to retype all topics by hand (with your keyboard)?
  • #23
    chri5k
    Level 5  
    I am an idiot. It was a typo in the command topic. I must have looked at that line 100 times. I typed 7321 in the HA script instead of 7231! Sorry for confusion.
  • #24
    p.kaczmarek2
    Moderator Smart Home
    Oh no, now I see it as well. Well, at least we've tested the OTA functionality well.
  • #25
    chri5k
    Level 5  
    And I learned about creating MQTT buttons and had fun doing Tcpdumps of the MQTT traffic. I was looking at the hex and realized the typo when the bytes didn't agree. Apparently, I can read Hex better than plaintext. LOL.

    I guess we can close this one out.

    PS. I sent a contribution to the project via Paypal.
  • #26
    p.kaczmarek2
    Moderator Smart Home
    Thanks, it will be useful for getting next devices or dev boards for testing new platform. Don't worry about typo, I didn't see it as well, and I even started suspecting the encoding issue because we already had a user with some kind of browser bug that caused incorrect characters for whitespace while copying scripts from forum and OBK parser failing to parse his commands...

    If you have any other suggestions or features related to OBK, let me know.

    And now, to get back on topic from first post, I have recently posted another topic about mass configuration/control on Windows:
    Automatic configuration of Tasmota and OpenBeken devices on Windows with cURL HTTP API
    That approach could also be used for provisioning obk devices.