Elektroda.com
Elektroda.com
X

Search our partners

Find the latest content on electronic components. Datasheets.com
Elektroda.com

WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT

p.kaczmarek2 21318 74
This content has been translated flag-pl » flag-en View the original version here.
  • #61
    btsimonh
    Level 10  
    [quote="Btw: @btsimonh if you want to do OTA upgrade, you should first download bytecode to specific OTA area and then allow old update mechanism to write it to user area, I think"[/quote]
    yes, that's what the http upgrade code claimed to do... but instead it erased sector 0!.. will try to look tonight. What I don't know yet is if/how to trigger the bootloader to decompress & flash from the OTA area - possible just by reboot, but will look in much more detail before trying this time!
  • #62
    jagheterfredrik
    Level 3  
    But that TLV belongs to the radio calibration section (BK_PARTITION_RF_FIRMWARE) though, right?
  • #63
    p.kaczmarek2
    Level 26  
    It seems that config save to flash is working for me now. Next major step towards full BK7231T has been just done!

    I have added my structure to aforementioned code, just like that:
    Code: c
    Log in, to see the code

    it also required some tweaks in read/write functions, because people writing original firmware were very lazy and typed some constant offsets by hand in some places instead of doing things more automatically (they have info_item_st::len field available but still use number value typed by hand).

    EDIT: Right now I am only saving pins config but I will also try to access WiFi one and soon people won't have to recompile firmware in order to pair their WB2S devices with MQTT and Home Assistant
  • #64
    jagheterfredrik
    Level 3  
    I quickly tried compiling the bdk_freertos repo from bekencorp's github. Using the latest GNU Arm Embedded Toolchain (10.3-2021.10), the only change was to add "-Wno-address-of-packed-member" to the CFLAGS of application.mk line 247, chmodding a few build tools to be executable and then build using `make bk7231u`.

    This repo is a lot cleaner than the tuya one and comes with a softap demo but I assume will require SPI flashing as the UART flashing code is part of the tuya stack?
  • #65
    p.kaczmarek2
    Level 26  
    if I remember correctly, I managed to fix all compilation errors with bdk_freertos but I couldn't get it to run by UART bootloader. I am not sure now what was the issue, I think there was even no package.exe and no _UA_ file generator there.
    Wait, check post #24:
    https://www.elektroda.com/rtvforum/topic3806769.html

    I don't think that Tuya code is a problem, most likely I will be removing it part by part soon
  • #66
    btsimonh
    Level 10  
    jagheterfredrik wrote:
    UART flashing code is part of the tuya stack?

    bootloader... not tuya :)

    how do you tag someone here?

    @p.kaczmarek2 - tried building you repo, and apart from fixing some strcpy_safe, it built. However, it still has Tuya, and also does no connect to my wifi (tries, but openWRT complains... and kills it).

    rebuilt mine (without tuya) and connects fine.
    On google told me it could be country codes ref wifi, but although I recall something about setting something like that, I can't find a mod in my version which could account for it. It's different to my initial issue where I needed to use 'advanced' for WPA2....

    ideas?

    s
  • #67
    p.kaczmarek2
    Level 26  
    btsimonh wrote:

    On google told me it could be country codes ref wifi, but although I recall something about setting something like that, I can't find a mod in my version which could account for it. It's different to my initial issue where I needed to use 'advanced' for WPA2....

    I remember that.
    Code: c
    Log in, to see the code

    I am still using my old version, because your mod was seemingly supporting only WiFi on specific channel (here is specified channel 5, in your code it was 11).
  • #68
    btsimonh
    Level 10  
    my mods of your app to remove Tuya & make it connect to my wifi:

    https://github.com/btsimonh/tuya-iotos-embeded-sdk-wifi-ble-bk7231t/tree/openbk7231T_mod

    Note changes to build.sh to make it faster (don't delete .o)
    Note removal of tuya lib.

    Still builds & runs :). But also connects to my wifi, and has no Tuya code we can't see...

    Comments: Webserver does not 'finish' connections (chrome spins, refresh icon remains as a cross).
    Flash read is working :).
    We should drop new_config.h from repo, and replace with 'example_new_config.h' - maybe copy in build script if not present?
    Will now play :).

    p.s. my 2.4 wifi is on channel 4? :(

    p.p.s. above branch updated for channel zero and WPA2 :)

    info: http://developer.mxchip.com/api/structapinfo__adv__t.html#details - maybe.

    p.p.p.s - there is something very odd going on - not got to the bottom of it, but suffice to say wifi connection and debug output is not reliable for me.
  • #69
    p.kaczmarek2
    Level 26  
    btsimonh wrote:

    Comments: Webserver does not 'finish' connections (chrome spins, refresh icon remains as a cross).

    I can confirm it happens, I will look into it.
    Luckily I can also run my HTTP server on Windows (you can also do this - there is msvc project for that) so it will be easy to investigate.
    EDIT: it does not happen on Windows, because the issue is in the tcp_client_thread, it seems that I just have to close TCP client socket after sending

    btsimonh wrote:

    Flash read is working :).

    Great news!

    btsimonh wrote:

    We should drop new_config.h from repo, and replace with 'example_new_config.h' - maybe copy in build script if not present?

    I'd rather say that new_config.h will not be needed soon, everything will be configurable by WWW panel

    btsimonh wrote:

    p.p.s. above branch updated for channel zero and WPA2 :)

    Thanks, I will check this out!

    btsimonh wrote:

    p.p.p.s - there is something very odd going on - not got to the bottom of it, but suffice to say wifi connection and debug output is not reliable for me.

    I don't know anything about that. I have no experienced such issue so far.
    I will test more on more boards and also I will check out your changes, maybe your "reconnect and disconnect" in timer loop is wrong or something...


    BTW: I am reverse-engineering another WB3S device, that one with MCU:
    WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT
    See the schematic I have drawn so far:
    WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT
    One can already infer some things from the connection. For example, flashing WB3S will require disconnecting the MCU UART connection or locating MCU reset pin to disable it temporary...
    I will also record the data exchange on this UART soon.
    Teardown results will be posted as separate topic in our Teardown sections of Elektroda
    WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT
    edm-01aa-eu, my dimmer is a version of https://templates.blakadder.com/unsupported/eardatek_EDM-01AA-EU.html
    but it has different pinout, compare where VDD and GND are on blakadder photos, for me it's using WB3S pin configuration annd TXD1 and RXD1 for communication with MCU
  • #70
    btsimonh
    Level 10  
    So.. the first problem I had was serial logging stalled at 'ent' - incomplete 'entering low level'.
    this is likely to be some conflict in logging mechanisms (or non-threadsafe vsprintf use).
    So I changed all logging in the main file to bk_printf.
    But after that, it logs nicely, but did not connect to wifi.
    I think it's the setup_deviceNameUnique();
    With that commented, it now does connect.
    Mods here:
    https://github.com/btsimonh/tuya-iotos-embeded-sdk-wifi-ble-bk7231t/tree/openbk7231T_mod

    It ran overnight....

    I'll try to focus on
    1/ logging with mutex to ram, a thread to send to serial safely, and to send to TCP (i.e. you connect to a port, and it sends the logs at you...).
    2/ I'll have another go at OTA.
    (with these two, we can become mostly free of wires :) )
    3/ I'll start a new repo for my 'app' as a test of separating apps from the SDK.

    On parameter storage, is this independent of the Tuya SDK Lib? i.e. do we have the source code?
  • #71
    p.kaczmarek2
    Level 26  
    btsimonh wrote:

    I think it's the setup_deviceNameUnique();

    Why? Is it working with sprintf replaced with strcpy in that function?

    Why did you do such a change again?
    WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT

    btsimonh wrote:

    On parameter storage, is this independent of the Tuya SDK Lib? i.e. do we have the source code?

    I think we have source code, you can track functions that I am using:
    save_info_item (net_param.c) calls bk_flash_write from
    https://github.com/tuya/tuya-iotos-embeded-sd..._os/beken378/func/user_driver/BkDriverFlash.c
    then it calls ddev_write
    https://github.com/tuya/tuya-iotos-embeded-sd...bk7231t_os/beken378/driver/common/drv_model.c
    then it uses function pointer to write, I didn't track it futher, I will look into it soon

    btsimonh wrote:

    I'll try to focus on
    1/ logging with mutex to ram, a thread to send to serial safely, and to send to TCP (i.e. you connect to a port, and it sends the logs at you...).
    2/ I'll have another go at OTA.
    (with these two, we can become mostly free of wires :) )
    3/ I'll start a new repo for my 'app' as a test of separating apps from the SDK.

    I will be happy to integrate your changes into the main master repo when they are stable, altough I'd suggest doing OTA first and not threading-breaking mods which are not crucial at the moment, but, as I understand it, you've already fixed threading problems?


    As per EDM-01AA-EU - it seems version KER_V1.6 is using WB3S and version 1.5 (from blakadder site) is using Realtek chip (RTL8710BN). I still haven't managed to find a RESET pin of MCU, so flashing would require cutting temporary a trace between WB3S and MCU, unless we have a way to do OTA on original WB3S software (without flashing it with our own bootloader). Something like tuya-convert for WB3S
  • #72
    btsimonh
    Level 10  
    p.kaczmarek2 wrote:
    btsimonh wrote:

    I think it's the setup_deviceNameUnique();

    Why? Is it working with sprintf replaced with strcpy in that function?

    Not sure. I thought maybe it was using wifi too early? Commenting this seems to make wifi connection work, but there are quite a few things going on here. e.g. I am concerned that a short power down does not necessarily lose everything in RAM/registers, and so when developing the firmware, a 'change' may work just because a register is no longer written, and happens to have a good value after a power cycle :(

    p.kaczmarek2 wrote:

    Why did you do such a change again?
    WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT

    I am not convinced that the 11 is 'wrong' - I think it just speeds up connection if it IS the right channel.
    e.g. with 11, it does connect to my channel 4 - 2-5 seconds after boot.
    Anyway, I don't use that function anymore, so put it back to 'stock'. Instead adjusted connecttowifi in our code..
    Does yours work with the 'advanced' call? (but it may work for me without advanced now, if there were other issues as above... I will test tonight.)

    p.kaczmarek2 wrote:

    I will be happy to integrate your changes into the main master repo when they are stable, altough I'd suggest doing OTA first and not threading-breaking mods which are not crucial at the moment, but, as I understand it, you've already fixed threading problems?

    I wrote the logger this morning, but won't test until tonight. It's one pair of files to add and then addLog(fmt, ...) to write (which we can #define to whatever we want). OTA after logger test, then I will submit for review :)

    p.kaczmarek2 wrote:
    a way to do OTA on original WB3S software (without flashing it with our own bootloader). Something like tuya-convert for WB3S

    I have posted on the tuyaconvert github ref the possibilities. But I think Tuya have locked to their cert now, so it's probably NOT going to work any more....
    br,
    Simon
  • #73
    p.kaczmarek2
    Level 26  
    btsimonh wrote:
    OTA after logger test, then I will submit for review :)

    I will happily check your pull request when it's ready. In a meantime, I prepared third device template, as seen here:
    Code: c
    Log in, to see the code

    Of course, in the long term it makes no sense to store templates in the code, and I will go for something like Tasmota-template system (where you copy a string into WWW panel field and voila) with a template list on separate Elektroda topic so other people can create and share their own templates, but for now I think templates in C code are acceptable.

    I have also found another smart device in my drawer:
    WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT WB2S/BK7231 Tutorial - writing custom firmware - UDP/TCP/HTTP/MQTT
    Is there a WB3S inside? Let's hope!


    btsimonh wrote:

    I have posted on the tuyaconvert github ref the possibilities. But I think Tuya have locked to their cert now, so it's probably NOT going to work any more....

    I really hoped that it's possible to get OTA working with original firmware. We have most parts of the source code, maybe there is some kind of vulnerability that we could exploit? Otherwise I will have to do the PCB track cutting trick for the WB3S + MCU 8051 device combo in order to flash it.
  • #74
    btsimonh
    Level 10  
    logging pushed on https://github.com/btsimonh/tuya-iotos-embeded-sdk-wifi-ble-bk7231t/tree/openbk7231T_mod
    logs to serial.
    If you connect to port 9000, (e.g kittyportable, raw), you will get the same logs.

    It logs to a ram circular buffer, and then there is a thread which writes to serial, and a thread which listens for TCP, and if connected, spawns another thread which sends to the connected port... If you connect twice to TCP, you will get half logs on each :(.
    Step 1 of my task complete :).

    Update - OTA data handler pushed on the same branch. Not tested with ACTUAL OTA yet, just flashing of a few bytes. Need to connect it to some form of data input, and test OTA - job for tomorrow.
  • #75
    p.kaczmarek2
    Level 26  
    What about doing UDP logging? In most cases it's enough to debug quickly on LAN, despite it is connectionless and not reliable.

    btsimonh wrote:

    If you connect to port 9000, (e.g kittyportable, raw), you will get the same logs.

    Or simply Putty.

    I have released a progress update and summary in separate topic.
    Please continue discussion there:
    [BK7231T] My HTTP server, configurator, MQTT support from Home Assistant


    Soon I will release first easy to use version which will not be requiring users to recompile the firmware. It will setup an open WiFi network and allow users to configure everything by web panel.