Beken BK7231/BK7252 SPI flashing and recovery - new flasher tool and protocol specs

@divadiow

Can you see what you can find from this dump

DeDaMrAz wrote:

Can you see what you can find from this dump

what beyond the usual boot log and kv extraction are you after?

@divadiow nice, but is it really more stable? Futhermore, I remember i tried to add retry on beken, and bk7231 chips didn't cooperate. It's different on RTL?

>>21855435

it's always completes for me and file comparisons are identical. It's had lots of testing. Previous flasher would not begin read for me usually. This behaves more like PGTool for timings and connection.

Added after 1 [minutes]:

it allowed this user to backup/flash and it has improved since then https://www.elektroda.com/rtvforum/topic4166319.html#21853518

divadiow wrote:

[01-01 00:00:00 ty N][43ed][device_config_load.c:1100] product have measure , chip is 3 vol is 2200  res is 0

What is this related to?

On this device ADC pin is connected, but will have to investigate more, looks like an NTC/PTC is attached there.

divadiow wrote:

more like PGTool for timings and connection

but it goes way beyond what LT/PGTool does for retries, checks, resiliency

Added after 9 [hours] 20 [minutes]:

to illustrate with some extremes:

20cm cables, 1.5m baud
animated. some retries, recoveries, completion


30cm cables with join at 20cm, 1.5m baud.
took 19 mins so no gif, but resultant file was identical to one above. speed because baud was auto dropped to 115200 after failures

insmod wrote:

Z2 flasher, use dump words instead of dump bytes for faster read times?

switched

DumpBytes - r24 - Read
~2min 26s - 921600
~1min 36s - 3000000

DumpWords - r25 - Read
~2min 6s - 921600
~1min 19s - 3000000

14%-18% speed improvement

Added after 8 [minutes]:

r26 - fixed a little bug

2min 2s - 921600 - no retries
1min 18s- 3000000 - 7 retries

Added after 15 [minutes]:

got a working TR6260 waiting in the wings too

I believe Easyflash for TR6260 requires GRAN=32 so have built WinEF_GRAN32_x64.dll and WinEF_GRAN32_x86.dll - that correct @insmod ?

Ok so how is the state of "make resilient Realtek Z2 read/write, add erase, update erase all text content" PR? Is it ready to merge?

sure. I would say yes. I've done a ton of testing.

Added after 2 [hours] 57 [minutes]:

thanks for merging! hope it's all good. felt good.

Thank you for contribution. If you are interested, maybe you can also help a bit with https://github.com/openshwprojects/EasyGUIFlashTool ? At least make it usable for basic, most common cases

could have a go. I think TR6260 for classic tool is almost good too, so will finish that today I think

Added after 6 [hours] 24 [minutes]:

TR6260 is done I believe.
-read/erase assumes flash is 1mb
-erase whole flash support. erases before write operations
-read/write backups and OpenTR6260 release (including bootloader and partition file)
-supports OBK config read/write - GRAN32 implemented without EF dlls

animated



bauds not seen in utpmain are blocked


https://github.com/openshwprojects/BK7231GUIFlashTool/pull/112

XR806 erase/read/write




https://github.com/openshwprojects/BK7231GUIFlashTool/pull/113

Nice, you're going fast. What are the speeds? Still, need to resolve a conflict with current main first.

p.kaczmarek2 wrote:

What are the speeds?

the supported XR806 speeds? I think in total it's 110, 300, 600, 1200, 2400, 4800, 9600, 19200, 38400, 57600, 115200, 230400, 460800, 921600

fixed merged conflict and a couple of other bits. I think it's good to go. Revisions, fixes, PRs to anything 'I' create always welcome.

Added after 1 [minutes]:

divadiow wrote:

the supported XR806 speeds? 115200, 230400, 460800, 921600

this is despite the bauds available in PhoenixMC. BROM <= 1 (XR809) has a stub

Added after 12 [minutes]:

divadiow wrote:

this is despite the bauds available in PhoenixMC.

that's the analysis from the Linux PhoenixMC anyway. Maybe it can do higher. Windows GUI shows higher



Added after 1 [hours] 38 [minutes]:

XR806 can be faster/better and I am looking at what next, but current PR works.

Added after 10 [hours] 1 [minutes]:

divadiow wrote:

the supported XR806 speeds?

PhoenixMC.exe response.



but timings, sector size, chunks etc don't change so not much to be gained.

got latest EF XR806 2mb whole-flash read speed to ~47s @921600

XR806 Pt2 is better.

- Support 'upgrade' command for firmware with Console code enabled (seen in OpenXR872)
- Reads BROM version (no longer complains about non-BROM 3 detection - WXU is BROM 4 and devboard is BROM 3)
- Reads flash ID
- Logs flash size

- Full flash read (2mb 41s @921600, 37s @3000000)
- Custom-offset read
- Chunked read transfer with retry / recovery handling

- Full-chip erase
- Explicitly full-chip-only behaviour for XR806

- Main firmware write from .img file
- Main full-flash restore from raw .bin file
- Custom-offset raw write from .bin file
- Backup-then-flash workflow
- Chunked write transfer with retry / recovery handling

.img on main write path:
- Parsed as XR image layout
- Validated before programming
- Written from base flash offset 0x000000
.bin on main write path:
- Treated as raw flash data
- Written from base flash offset 0x000000
- Not truncated to image partition/layout size
.bin on custom write path:
- Treated as raw bytes only
- Written to the user-selected offset
- No image-layout parsing

XR image validation:
- AWIH header validation
- Header checksum validation
- Section data checksum validation
- Section-chain walking
- Effective image size calculation
- Image layout logging

XR image safety checks:
- Rejects invalid AWIH magic
- Rejects bad header checksum
- Rejects bad data checksum
- Rejects next-section pointers that fall inside the current section
- Rejects next-section pointers outside file range
- Rejects non-header targets in the section chain
- Rejects section data that overruns the file
- Rejects overlapping, looping, or non-advancing section chains

Safety and resilience:
- Initial sync retries before failure
- Session recovery and reconnect logic on read/write failures
- Flash-range bounds checking before write
- BROM response payload sanity checking
- Progress and policy logging for key operations

operational behaviour:
- Main write path performs full-chip erase before programming
- Custom raw write path does not erase first
- Backup-and-flash in one session
- Firmware is not run/device not rebooted after flash write



https://github.com/openshwprojects/BK7231GUIFlashTool/pull/114

Added after 5 [minutes]:

appears to work fine for XR872 too, because it's BROM2+, so XR872 should be easy adaptation next

Added after 8 [hours] 52 [minutes]:

a couple of tweaks. flush after erase, added chip type response.
example with erase operation:


Added after 8 [hours] 46 [minutes]:

-add XR872, same abilities as XR806
-sort chip list alphabetically
-make log box auto-scroll instead of only catching-up on newline
-update readme
-set BK7231N as default chip (was BK7231T, alphabet sort made index 0 default to BK7231M)
-set 115200 as default baud (was 230400)

erase

flash OpenXR872 (to 1mb XF16)

read at 1.5m baud (resync/retries are expected - PhoenixMC drops to 115200 for this XF16)

read at 115200

EF/PhoenixMC dumps are identical

auto-scroll behaviour (animated):


https://github.com/openshwprojects/BK7231GUIFlashTool/pull/115

Added after 7 [minutes]:

sorted chip list

Almost perfect, however, I would think twice about changing enumeration order. Are we really sure it's not saved anywhere as index? But maybe not...

ah, I did check. it's purely display order. nothing seems to index them numerically. References are to BKType rather than a positional index.

Added after 1 [hours] 38 [minutes]:

@p.kaczmarek2 also make scan status box resize proportional to main window (@dedamraz request) + add subnet picker from local machine network interfaces
https://github.com/openshwprojects/BK7231GUIFlashTool/pull/116

divadiow wrote:

also make scan status box resize proportional to main window

❤️

Hmm okay, I guess it's acceptable, then. Thanks.

Are XR flashing protocols much different?

yay, thanks for merging!

p.kaczmarek2 wrote:

Are XR flashing protocols much different?

mostly the same. but with BROM < 2 (eg XR809) although it supports the same basic BROM-direct commands like

GetFlashID 0x18

, it requires stub loader for erase/write/read operations.

BROM >= 3 (BROM 4 seen on real WXU and BROM 3 seen on AllWinner XR806 dev board) support an alternate set of opcodes, I have not used these. Legacy family used for all our XRs. Not sure of the purpose/advantage of the newer family.

legacy:
GetFlashID 0x18
ReadSector 0x1A
EraseFlash 0x19
WriteSector 0x1B

alternate:
GetFlashID 0x1C
ReadSector 0x1E
EraseFlash 0x1D
WriteSector 0x1F

Not entirely sure when and why alternate family gets used.

add same for XR809 - stub path
https://github.com/openshwprojects/BK7231GUIFlashTool/pull/117

Nice, thank you. What's now missing from main (old) Easy Flasher?

chip-wise? the rest of the ESPs I guess. Also BK7252 wrap-around still needs fixing.

potentially TXW81X, but I hadn't really considered it. There's also the other RTLs, not sure if @insmod was going to do them one day or.. dunno.

XRs really need to be either merged together into one, or majority of the code to be moved to some base class, like XRBaseFlasher.

Is there really a need to check for allowed baud?
Since it's passed as a value and not as an index like on RTLB/D, perhaps it can be removed?

Does XR809 support 230400-460800 baud?
XR809 can erase by blocks. Do others support it too?

Perhaps create alternate stub for all of them, using LN ramcode as a base?

insmod wrote:

XRs really need to be either merged together into one, or majority of the code to be moved to some base class, like XRBaseFlasher

I guess that would make sense

insmod wrote:

XR809 can erase by blocks. Do others support it too?

Yes

I had a feeling XR wouldn't sync at bauds it didn't report as supported. Can check in morning

Hadn't even considered custom stub for them all.

Feel free to sort it all out!

Added after 6 [hours] 50 [minutes]:

insmod wrote:

XR809 can erase by blocks. Do others support it too?

there isn't custom erase function in EF so didn't look to implement. earlier XR806 builds erased whole chip in 64k sectors

Added after 11 [hours] 34 [minutes]:

I recall now. the restrictions in XR806 won't get hit anyway because the supported list includes all the EF baud options.

XR809 fails the baud change command for 230400, 460800 and 2m

XR872 fails the baud change for 230400, 460800 but does have a go at 2m


re XR809 the command sent for those unsupported bauds is sound.


PhoenixMC response for each:


restrictions removed test build: https://github.com/divadiow/BK7231GUIFlashTool/actions/runs/23495167292

https://github.com/openshwprojects/BK7231GUIFlashTool/pull/119

to fix https://www.elektroda.com/rtvforum/topic4162022.html#21875955

Added after 2 [minutes]:

should classic 'json' presentation be removed? it would mean needing to be sure json->obk template parsers (eg in web app) are able to handle pasted enhanced json

Are there any regressions? Dumps that used to be decoded correctly but now fail?

hmm. some of the 'classic' jsons are different. not sure you'd call them regressions. tbh it'd be easier/cleaner if the classic was done away with completely





Added after 3 [hours] 21 [minutes]:

OK. PR now =

313 files checked
classic JSON: identical on every tested dump that extracted before
enhanced JSON: identical on every tested dump that extracted before
LN882H problem dump: no longer crashes, now cleanly lands in enhanced fallback
the 2 known fromFile_failed files: unchanged (I know what one of these is (new KV type) and I'll look into the other one)
6 ESP8266 dumps: improved from no_keys to enhanced_fallback

Will it work with our tools after update, or does webapp needs to be separately updated? TemplateImporter?