logo elektroda
logo elektroda
X
logo elektroda
Advanced Search
logo elektroda

Beken Flash Dump Partition Info Extraction with RT-Threads Partition Tool - Development Thoughts?

divadiow 855 49
ADVERTISEMENT
Report a violation of the law
Reply | New topic
  • Helpful post
    #1 21734412
    divadiow
    Level 37  
    It's oftentimes useful to know what the partition offsets and lengths are on any given device that may be up for conversion to OpenBK*. This can be useful when determining if a device should have its bootloader overwritten by using the QIO release in order, for example, to allow OTA updates to work. OpenBK will write the ota file to a fixed offset and if the bootloader isn't looking for updates to initiate from the same location, then OTA will fail.

    From the backup of the factory firmware it can usually be worked out from the boot log or by analysing the partition table starting "01PE" what the offsets are, but this requires a bit of sleuthing and getting your head around reverse-byte order little-endian in order to read what's been configured.
    eg https://www.elektroda.com/rtvforum/topic4074263.html#21217939

    I've been fiddling with the RT-Threads Partition Tool, which lets you edit/add partitions to a virgin bootloader.
    eg, open bootloader_bk7231n_uart2_v1.0.13.bin that can be found in the Beken SDK and the tool will show you what's set

    Partition table from bootloader file in RT-Thread Partition Tool window

    That's great, but it doesn't work on a standard 2mb dump.

    BK7231N_QIO_ATLO-WB3K-TUYA_2025-16-1-14-11-09.bin

    RT-Thread Partition Tool window with bootloader and firmware files loaded

    With this in mind:
    insmod wrote:
    Yes, and it's that way on every beken, whether encrypted or not.
    Every 32 bytes of data is followed by 2 bytes of crc

    You can create a script that would remove those bytes to get readable strings, if required.


    and using @Paver3109's uncrc script from https://github.com/fusetim/insecurity-camera-tools/tree/main/firmware the crc can be removed

    Code: Text
    Log in, to see the code


    but the resultant file is still not readable in the GUI tool nor CLI

    RT-Thread Partition Tool window showing no partition table read from BIN file

    Code: Text
    Log in, to see the code


    This script will scan the first 2mb and extract the section of 2 more where "01PE" is seen 64 bytes apart
    Code: Python
    Log in, to see the code


    It launches a Windows picker for the input file and will give a result summary and output a file suffixed with _partitions to filename

    Success dialog showing 01PE group found and partition file saved

    While the result still does not reveal anything if opened with the GUI tool, it will work with the CLI
    Screenshot of rt_partition_tool_cli.exe showing flash partition data in console

    Let's test with some Bekens of various flavours, 2mb and 4mb.

    4MB BK72326 - Tuya_T3-3S_readResult_BK7231N_QIO_7236_fixed-insmod.bin
    Code: Text
    Log in, to see the code


    needs de-deduping

    non-Tuya 4MB Matter BK7231M - SparkleIoT_XH-CB2S_Mini_Switch.bin
    Code: Text
    Log in, to see the code


    Non-Tuya BK7231U - Arnoo_Winees_WS010078261_AiDot_RGBW_A60_(2ndGen)_(light.A001139)_Leedarson_CC8000_2.0.4.51.bin
    Code: Text
    Log in, to see the code


    This one was interesting, and why the search for 01PE was set to cover 2mb, There's the 3 01PEs at the beginning of the flash and then another set further in
    Hex view displaying occurrences of the 01PE string in a flash dump file

    Interesting, but probably not a reason to scan beyond initial 01PE area.

    BK7231T_Denver-SHP-200MK2.bin
    Code: Text
    Log in, to see the code


    non-Tuya BK7238 - Zengge_AK001-ZJ21413_77_24_20240423_ZG-BK_Symphony_USB_SPI_LED.bin
    Code: Text
    Log in, to see the code


    Tuya BK7238 1 - Tuya_Switch_(schemaID-e1mzkfec)_keyfxuj3d5wg53wx_jsejjp8bin9lllh3_T1-2S-NL_1.1.4.bin
    Fail - no 01PE in dump

    Tuya BK7238 2 - Tuya_IHSENO_HAZ-IR-V3_(schemaID-e1mybg3w)_keycmhjv873dhvsn_kv4htgxocgwjfabc_T1-3S_0.0.5.bin
    Fail - no 01PE in dump

    non-Tuya BK7252N - Pinmei_A9_BK7252N_BKFIL.bin
    Code: Text
    Log in, to see the code


    4MB Tuya BK7252U - Tuya_Doorbell_IP06-V6.3-BekenSPI_2025-08-10-21-29-16.bin
    Fail - no 01PE in dump

    4MB Tuya BK7252U - Tuya_A9_Cam_SC-10024_V1.0.0_(VGA卡片机-schemaID-000003uanz)_keyph8vwhadwsx3a_u0kmecmr7j8roazz_NEO_SPI_1.0.4.bin
    Code: Text
    Log in, to see the code


    and so on. If the dump has 01PE partition style then it seems we can easily read it.

    So, what next?

    -extract Python script from RT tool GUI and CLI EXEs?
    -consolidate different steps into one script
    -make a C application and integrate with Easy Flasher- new tab?
    -add additional checks - eg TLV start location?

    thoughts?

    I attach the scripts used above, and some extras, including extraction of all groups of 01PE if found. RT Partition Tool too.

    Flashdumps tested with can be found at https://github.com/openshwprojects/FlashDumps/tree/main/IoT
    Attachments:
  • ADVERTISEMENT
  • #2 21734694
    p.kaczmarek2
    Moderator Smart Home  
    Interesting, is it the same as the RBL extraction that I started some time ago in FlashDumps?
    Visual Studio in debug mode with C# code and console window displaying RBL data
    https://github.com/openshwprojects/FlashDumps
    https://openshwprojects.github.io/FlashDumps/
    Or wait, it seems it's not the same... RBL just finds RBL identifiers in the binary, not 01PE
    Helpful post? Buy me a coffee.
  • ADVERTISEMENT
  • #3 21734706
    divadiow
    Level 37  
    still, interesting. what were you hoping to achieve or was it just to see what was readable from rbls?
  • #4 21734718
    p.kaczmarek2
    Moderator Smart Home  
    I was trying to find how many bootloader versions we know, but it didn't work too well for me. Maybe it needs more work. I think your partition table is also very good and useful to implement
    Helpful post? Buy me a coffee.
  • ADVERTISEMENT
  • #6 21735619
    p.kaczmarek2
    Moderator Smart Home  
    Why it needs to call rt_partition_tool_cli.exe? I think I am confused.

    divadiow wrote:
    OpenBK will write the ota file to a fixed offset and if the bootloader isn't looking for updates to initiate from the same location, then OTA will fail.

    Maybe we could do that in OBK itself?
    Helpful post? Buy me a coffee.
  • #7 21735633
    divadiow
    Level 37  
    p.kaczmarek2 wrote:
    Why it needs to call rt_partition_tool_cli.exe? I think I am confused.


    that's the tool that does the final read of partitions. I haven't opened up exe to see how it's scripted. or even tried to script without looking. it'd be with GPT help either way.

    p.kaczmarek2 wrote:
    Maybe we could do that in OBK itself?


    by having OBK read download partition offset from table?
  • #8 21735643
    p.kaczmarek2
    Moderator Smart Home  
    I am not yet sure how it works internally. I've been thinking about doing flash scan for 01PE entries in OBK itself, but I would need to skip CRCs, right?
    Code: C / C++
    Log in, to see the code
    Helpful post? Buy me a coffee.
  • #9 21735648
    divadiow
    Level 37  
    p.kaczmarek2 wrote:
    but I would need to skip CRCs, right?

    yes. the py in that zip is the combination of the two parts in post 1. uncrc then 2. partition table extraction

    Added after 1 [minutes]:

    I'll open up the cli.exe now to look at the insides
  • Helpful post
    #10 21735668
    p.kaczmarek2
    Moderator Smart Home  
    I am testing in the simulator with beken binary dump
    Visual Studio environment with C code and OpenBeken simulation logs shown
    Helpful post? Buy me a coffee.
  • #11 21735699
    divadiow
    Level 37  
    interesting kinda

    py on uncrcd full dumps. tables plus some junk

    Console screenshot showing test.py results for various .bin files and flash errors

    Code: Python
    Log in, to see the code
  • #12 21735710
    p.kaczmarek2
    Moderator Smart Home  
    To be sure - we are searching for 01PE because it's not at the constant offset?
    Helpful post? Buy me a coffee.
  • #13 21735714
    divadiow
    Level 37  
    my experience has been that it differs

    eg

    Screenshot of hex editor with four BIN files opened showing binary and decoded text
  • #14 21735727
    p.kaczmarek2
    Moderator Smart Home  
    I see.
    You can try running this now:
    https://github.com/openshwprojects/OpenBK7231T_App/pull/1847
    Code: text Expand Select all Copy to clipboard
    
startDriver BKPartitions

    should print offsets...

    Added after 17 [seconds]:

    wait a sec, i didn't enable it on beken i think
    Helpful post? Buy me a coffee.
  • Helpful post
    #15 21735784
    divadiow
    Level 37  
    N
    System logs of an IoT device booting, showing Wi-Fi network and MQTT status

    T gave no result
  • #16 21735804
    p.kaczmarek2
    Moderator Smart Home  
    But T also has it, right? Maybe I can't read 0x0-0x11000 section on T and I need to apply 0x20000 offset...
    Helpful post? Buy me a coffee.
  • #17 21735807
    divadiow
    Level 37  
    p.kaczmarek2 wrote:
    But T also has it, right?


    all I have tried with script have. Dump of same T tested with run through script =

    Code: Text
    Log in, to see the code
  • #18 21735956
    p.kaczmarek2
    Moderator Smart Home  
    I've pushed a little change. I added 0x200000 offset. I'm curious whether it will work - it's the same trick which is used by flasher.
    Helpful post? Buy me a coffee.
  • #19 21736095
    divadiow
    Level 37  
    negative. 1847_merge_c03a1c868c29
    Screenshot of a system log with network and MQTT connectivity status

    Added after 11 [hours] 12 [minutes]:

    divadiow wrote:
    Non-Tuya BK7231U - Arnoo_Winees_WS010078261_AiDot_RGBW_A60_(2ndGen)_(light.A001139)_Leedarson_CC8000_2.0.4.51.bin


    interesting.
    Memory map of 2 Mb flash divided into boot, app, download, RF, NET, and USER sections
    Attachments:
  • #20 21737153
    p.kaczmarek2
    Moderator Smart Home  
    Does it look valid?
    Code: text Expand Select all Copy to clipboard
    
Info:CMD:AS: found magic at 0xF7A2
Info:CMD:AS: nearest at 0xF790
Info:CMD:Partition found at 0xF7EC: bootloader flash=beken_onchip_crc offset=0x0 size=65536 extra=0 layout=fal64
Info:CMD:Partition found at 0xF82C: app flash=beken_onchip_crc offset=0x10000 size=1083136 extra=0 layout=fal64
Info:CMD:Partition found at 0xF86C: download flash=beken_onchip offset=0x12A000 size=679936 extra=0 layout=fal64

    I pushed a fix, still probably N only.

    Added after 1 [hours]:

    added sanity check (partition length) useful for some binaries:
    Code: text Expand Select all Copy to clipboard
    
Info:MAIN:Started BKPartitions.
Info:CMD:AS: found magic at 0x88F0
Info:CMD:AS: nearest at 0x88EE
Info:CMD:Abnormal partition length, probably invalid, skipping
Info:CMD:Abnormal partition length, probably invalid, skipping
Info:CMD:AS: found magic at 0xE0CC
Info:CMD:AS: nearest at 0xE0B8
Info:CMD:Partition found at 0xE138: bootloader flash=beken_onchip_crc offset=0x0 size=65536 extra=1253905839 layout=fal64
Info:CMD:Partition found at 0xE178: app flash=beken_onchip_crc offset=0x10000 size=1063936 extra=3505373052 layout=fal64
Info:CMD:Partition found at 0xE1B8: download flash=beken_onchip offset=0x125000 size=737280 extra=241988858 layout=fal64
    Helpful post? Buy me a coffee.
  • #21 21737515
    divadiow
    Level 37  
    1847_merge_c8e0bc728e21

    BK-T
    Code: Text
    Log in, to see the code


    BK-N
    Code: Text
    Log in, to see the code
  • #22 21737516
    p.kaczmarek2
    Moderator Smart Home  
    Wait, what? It suddenly started working for T? Well, there indeed was a bug...

    Why is the partition twice in T?
    Helpful post? Buy me a coffee.
  • #23 21737523
    divadiow
    Level 37  
    p.kaczmarek2 wrote:
    Wait, what? It suddenly started working for T?

    seems so. I wonder which build fixed it, I could check :/

    p.kaczmarek2 wrote:
    Why is the partition twice in T?

    hmm. yes.

    only 3 01PE
    Hex memory dump showing 01PEbootloader, 01PEapp, and 01PEdownload labels
  • ADVERTISEMENT
  • #24 21737538
    p.kaczmarek2
    Moderator Smart Home  
    So it works, more or less. Does it mean that we can use this approach to locate OTA target offset? Would that help on some non-standard devices? @insmod
    Helpful post? Buy me a coffee.
  • #25 21737549
    insmod
    Level 29  
    >>21737538
    Doubtful.
    If i remember it correctly, uascent bootloader OTA address is 0x147000.
    It will overwrite LFS, and perhaps RF partition and config?

    Added after 3 [minutes]:

    Let me try to make 1.0.1 uascent bootloader...
  • #26 21737552
    p.kaczmarek2
    Moderator Smart Home  
    What if we scan partitions on the first OBK boot and dynamically find good offset for LFS? But probably it's not worth it, idk...
    Helpful post? Buy me a coffee.
  • #27 21737555
    divadiow
    Level 37  
    insmod wrote:
    Let me try to make 1.0.1 uascent bootloader...


    well, yes. Ultimately a replacement Uascent BL would be best. But has proven problematic so far (for me)

    combined_uncrc_extract_run.py Uascent_uHome_Matter_4CH_White_Switch_CSA23257MAT40770-24_UAM027-A0-V1.0_CBU_3.6.bin
    Code: Text
    Log in, to see the code


    Added after 1 [minutes]:

    replacement BL for any non-OBK standard layout*
  • #29 21737573
    divadiow
    Level 37  
    no boot. no log. I've never seen the standard encrypt v0.3.1 make a working BL. have had to use cmake_encrypt_crc.exe
Reply | New topic
Report a violation of the law

Topic summary

The discussion focuses on extracting Beken flash dump partition information using the RT-Threads Partition Tool to facilitate OpenBK* firmware conversions and enable OTA updates. The main challenge is identifying partition offsets and lengths, particularly the "01PE" partition table signature, which varies in location and requires handling little-endian reverse byte order and CRC16 checksums embedded every 34 bytes. A combined Python script (combined_uncrc_extract_run.py) was developed to remove CRC16 checksums and extract partition tables from firmware dumps, followed by using the RT-Thread partition tool CLI to parse and display partition layouts. The tool supports different partition table formats (fal64 and fal48) and outputs partition names, flash types, addresses, and sizes. Attempts to integrate partition detection directly into OpenBK firmware were discussed, including scanning flash memory for "01PE" signatures and skipping CRCs. The conversation also covers challenges with bootloader compatibility, encryption, and CRC validation, especially for Uascent and Tuya bootloaders on BK7231N and BK7231T chips. Testing showed that OTA functionality depends on correct partition offsets and bootloader keys, with some success in creating replacement bootloaders that support OTA on non-standard layouts. The importance of TLV detection for MAC and RF data restoration was noted. Suggestions include adding partition detection to flashing tools like Easy Flasher and improving CRC16 verification in scripts. Overall, the approach aids in troubleshooting OTA failures by accurately identifying partition layouts and offsets in Beken-based devices.
Summary generated by the language model.
Today's popular
ADVERTISEMENT