logo elektroda
logo elektroda
X
logo elektroda
Advanced Search
logo elektroda

Hosting OpenBekenIOT Web App's 'Logs' function in the LittleFS of the Device (BK7231T)

jkwim 3174 36
ADVERTISEMENT
Reply | New topic
  • #1 20928161
    jkwim
    Level 12  
    Opening a new thread to continue discussion from Docker Image for OpenBeken IoT Web App Hosting - Post#13

    It was concluded that the default LittleFS size is 32kB.

    I am planning to host the following files in the device itself so that the 'Logs' option can be accessed in a closed network.

    Some points to consider before doing this:

    https://www.elektroda.com/rtvforum/topic4022819.html#20925110

    https://www.elektroda.com/rtvforum/topic4022819.html#20925206

    Plan:


    Host following files in LittleFS:
    Code: text Expand Select all Copy to clipboard
    /startup.js
/httpVueLoader.js
/vue.min.js
/logs.vue

    Note that the paths within startup.js needs to be modified to access them within a single directory.

    Code: text Expand Select all Copy to clipboard
    var s1 = document.createElement('script');
s1.setAttribute('src', window.root+"vue.min.js"); <================

document.head.appendChild(s1);

s1 = document.createElement('script');
s1.setAttribute('src', window.root+"httpVueLoader.js"); <=============
document.head.appendChild(s1);

    Also modify the path for the logs.vue and delete all other components:

    Code: text Expand Select all Copy to clipboard
        window.components = [
        { name:'logs', path: window.root+'logs.vue'+unique },

    ];


    Additionally will have to host one of the following files:

    indexdevice.html
    indexlocal.html

    from https://github.com/OpenBekenIOT/webapp/blob/gh-pages/indexlocal.html
    I haven't figured out which one yet.

    To proceed I have used online JavaScript Minifier to reduce the size of the .js files.

    Now with default LittleFS size I have consumed about 16kB as follows:

    File list with sizes: /httpVueLoader.js - 5867, /logs.vue - 8758, /startup.js - 1087.

    The file vue.min.js is about 92kB.

    So my estimation shows that if LittleFS size can be increased to 128kB I should be able to get all the files needed in there (based on my assumptions of selected files).

    How would 128kB LittleFS size impact OTA operations?

    I really do not mind if it gets wiped out during OTA as I can always get them back in manually. I do not expect to do frequent OTA updates on this device anyway.

    @taktlos
    @Angel0fDeath
    @p.kaczmarek2
  • ADVERTISEMENT
  • #2 20928195
    p.kaczmarek2
    Moderator Smart Home  
    You may get LittleFS being erased by OTA but it's not a problem, just keep a backup, you can easily download LFS backup here:
    User interface for managing LittleFS filesystem with backup download option highlighted.
    Use lfs_format to create LittleFS of required size, see our command docs:
    https://github.com/openshwprojects/OpenBK7231T_App/blob/main/docs/commands.md
    Helpful post? Buy me a coffee.
  • #3 20928296
    jkwim
    Level 12  
    For testing I tried this on another device which is a BK7231N. The previous device is being used currently so I did not want to mess it up.

    This is my observation:

    1. Increased the LittleFS size to 128kB (4 x times the previous size)
    Code: text Expand Select all Copy to clipboard
    lfs_format 0x20000

    Code: text Expand Select all Copy to clipboard
    Debug:LFS:lfs_format(0x40ab78, 0x400694 {.context=0x0, .read=0x70ac9, .prog=0x70a5d, .erase=0x709f5, .sync=0x709f1, .read_size=1, .prog_size=1, .block_size=4096, .block_count=32, .block_cycles=500, .cache_size=16, .lookahead_size=16, .read_buffer=0x0, .prog_buffer=0x0, .lookahead_buffer=0x0, .name_max=0, .file_max=0, .attr_max=0})
Debug:LFS:lfs_format -> 0
Debug:LFS:lfs_mount(0x40ab78, 0x400694 {.context=0x0, .read=0x70ac9, .prog=0x70a5d, .erase=0x709f5, .sync=0x709f1, .read_size=1, .prog_size=1, .block_size=4096, .block_count=32, .block_cycles=500, .cache_size=16, .lookahead_size=16, .read_buffer=0x0, .prog_buffer=0x0, .lookahead_buffer=0x0, .name_max=0, .file_max=0, .attr_max=0})
Debug:LFS:lfs_mount -> 0
Info:LFS:Mounted existing LFS


    Code: text Expand Select all Copy to clipboard
    lfs_size


    Code: text Expand Select all Copy to clipboard
    Debug:CMD:cmd [lfs_size]
Info:CMD:unchanged LFS size 0x20000 configured 0x20000
Info:CMD:[WebApp Cmd 'lfs_size' Result] OK


    2. Tried to copy vue.min.js.txt (92kB). Ignore the .txt extension for the moment. Device reboots and upload not successful. Cannot capture the last error in the logs.

    Code: text Expand Select all Copy to clipboard
    Debug:LFS:lfs_file_open(0x40ab78, 0x41e420, "vue.min.js.txt", 103)
Debug:LFS:lfs_file_open -> 0
Debug:LFS:lfs_file_write(0x40ab78, 0x41e420, 0x421223, 635)
Debug:LFS:lfs_file_write -> 1022
Debug:LFS:lfs_file_write(0x40ab78, 0x41e420, 0x4210a0, 1022)
Debug:LFS:lfs_file_write -> 1022
Debug:LFS:lfs_file_write(0x40ab78, 0x41e420, 0x4210a0, 1022)
Info:MAIN:Main_Init_Before_Delay
Info:CFG:####### Boot Count 38 #######
  • #4 20930090
    Angel0fDeath
    Level 13  
    @jkwim I think you need to reboot the device after lfs_format and before trying to copy files. Did you try this?
    Also, try to increase lfs size to let's say 150 or 160k. Take into account lfs block size. If a block is occupied even with 1 byte of data by one of the files, you cannot write data from another file in this block. So when doing your calculations, take this into account. I think the min (and best) block size for LittleFS is 104 bytes, but if your calculations are too precise, this could be an issue. I'm not sure about the implementation of LittleFS in openBeken, but you can check the sources. You also have a system area in LittleFS (can't remember the size right now). So according to me, just too precise calculations. Just increase the size. When everything is okay, you can try to make more precise calculations to reduce lfs size...

    EDIT: From your output above (format command) - you have 32 blocks 4k each (4096 bytes per block)..... Definitely too small fs for your files - for each file you store you either lose 1 sector - up to 4K (most common situation) or not (very rare case - file to have exact filesize to fit in blocksize). Again - too precise calc without taking into account some important things.
    I hope this will help you to determine the correct fs size for your needs.

    EDIT2: In this case you can calculate your space needed for each file - (((filesize in bytes) div 4096) + 1) * 4096). Then sum the results for each file.
  • ADVERTISEMENT
  • #5 20931820
    jkwim
    Level 12  
    Tried even increasing to 256kB. Rebooted after format.

    When I tried to upload a 92kB file, device reboots.

    Something else must be going wrong during uploading.

    Also when I increased it to 256kB I get a Heap Low warning.

    Code: text Expand Select all Copy to clipboard
    Error:MAIN:Low heap warning!


    So I do not think that LittleFS is expanding on to OTA area but perhaps beyond.
  • #6 20931925
    Angel0fDeath
    Level 13  
    I don't know if I'm correct or not (probably @p.kaczmarek2 can help), but low heap is like low memory. Try to make tar archive of all your files you need on LittleFS (keeping it large enough - for instance 160 - 200KB) and drop the archive in the box of web app. Or try to copy first the largest file....

    EDIT:
    I formatted lfs with lfs_format 0x32000
    Then I was able to copy file more than 100KB

    Filesystem browsing interface in a web browser.

    Then I am able to copy more files:

    Screenshot of the OpenBeken interface with HTML file editing in the device's filesystem.

    As you can see files copied to LFS are 156k (exact as per occupation by block size of 4K) and about 147k real size, but still works. I copied the largest one first. It takes about 30-40 sec. to copy the largest file (very slow, but it works... at least by me).


    EDIT2: Can you finally post some more details about your device. It is possible your device to have less flash or some other peculiarities...

    Added after 2 [hours] 44 [minutes]:

    Quote:
    Tried even increasing to 256kB. Rebooted after format.


    Hmm... Just tried with 256K fs and it seems you are correct. Also tried with 220K fs - the same result. However with 200K it was ok by me.
  • #7 20932850
    jkwim
    Level 12  
    I will try the suggestions and post the outcome.

    In the meantime, found these Link

    BK7231 Memory Layout/Partitions
    Here's the memory layout used by Tuya.

    BK7231T:
    Name Code Start Length
    Bootloader BK_PARTITION_BOOTLOADER 0x00000000 0x11000 //68KB
    Application BK_PARTITION_APPLICATION 0x11000 0x121000//1156KB
    ota BK_PARTITION_OTA 0x132000 0x96000, //600KB
    RF Firmware BK_PARTITION_RF_FIRMWARE 0x1e0000 0x1000
    NET info BK_PARTITION_NET_PARAM 0x1e1000 0x1000


    BK7231N:
    Name Code Start Length
    Bootloader BK_PARTITION_BOOTLOADER 0x00000000 0x11000 //68KB
    Application BK_PARTITION_APPLICATION 0x11000 0x121000//1156KB
    ota BK_PARTITION_OTA 0x12A000 0xA6000 //664KB
    RF Firmware BK_PARTITION_RF_FIRMWARE 0x1D0000 0x1000
    NET info BK_PARTITION_NET_PARAM 0x1D1000 0x1000



    Bootloader and Application partitions should be self-explanatory; the UART programming will not work if bootloader partition is overwritten. The OTA partition is used to store pending update (RBL data) that will be unpacked and executed with next reboot (it will be unpacked into Application partition).
    NET Info partition is a generic data partition with a key-value system used to store SSID, etc, it is overwritten in OpenBeken with our own structures. RF Firmware/calibration partition stores RF calibration data and MAC address.
    NOTE: BK_PARTITION_NET_PARAM partition can be decoded and read on any BK device with a simple UART programmer, so your SSID and password may be leaked if you throw out used IoT device.
    There are also some undocumented partitions in the code, for example a quick reconnect block data for WiFi were stored in N SDK at the hardcoded address.

    Added after 29 [minutes]:

    However when you look closer I think the starting address of OTA partition for BK7231N above cannot be 0x12A000. Probably there is a mistake there if the Application partition size is 1156kB.

    I could not yet figure out how the LitteFS allocation is done within the Application Partition.
  • #8 20933352
    Angel0fDeath
    Level 13  

    @jkwim I don't think lfs is in the application partition. Just the application partition is smaller than 1156K and lfs is immediately after the application partition.
    Nevertheless, I don't see how this info will help you to resolve your issue. Just try to fit in 200K lfs and probably it will work...
    Or maybe an alternative solution will be better. You can host the web app on any device in your network running 24/7 like a home automation server, NAS, even in your router if the router is running OpenWrt. Well, not much space in the router, but in general, it's possible. I'm hosting the web app on my NAS with Docker.
  • #9 20933398
    jkwim
    Level 12  
    Angel0fDeath wrote:
    I don't know if I'm correct or not (probably @p.kaczmarek2 can help), but low heap is like low memory. Try to make tar archive of all your files you need on LittleFS (keeping it large enough - for instance 160 - 200KB) and drop the archive in the box of web app. Or try to copy first the largest file....

    EDIT:
    I formatted lfs with lfs_format 0x32000
    Then I was able to copy file more than 100KB

    Filesystem browsing interface in a web browser.

    Then I am able to copy more files:

    Screenshot of the OpenBeken interface with HTML file editing in the device's filesystem.

    As you can see files copied to LFS are 156k (exact as per occupation by block size of 4K) and about 147k real size, but still works. I copied the largest one first. It takes about 30-40 sec. to copy the largest file (very slow, but it works... at least by me).


    EDIT2: Can you finally post some more details about your device. It is possible your device to have less flash or some other peculiarities...

    Added after 2 [hours] 44 [minutes]:

    Quote:
    Tried even increasing to 256kB. Rebooted after format.


    Hmm... Just tried with 256K fs and it seems you are correct. Also tried with 220K fs - the same result. However with 200K it was ok by me.


    Could you try uploading the same file as me?
    vue.min.js

    No matter what I do the end result is the same.

    I tried two devices BK7231T(WB3S 1.17.113) and BK7231N(CB3S 1.17.255), both are 4-Gang Smart Wall Switches.
  • #10 20933489
    Angel0fDeath
    Level 13  

    @jkwim Tested and it looks like everything is okay:

    Software interface for managing LFS files via browser on BK7231T device.

    LFS formatted with lfs_format 0x32000
    The device is BK7231T, WBS3
    However, the firmware is 1.17.425. It's probably a good idea to upgrade the firmware.

    Added after 10 [minutes]:

    Quote:
    Hmm... Just tried with 256K fs


    I also noticed the following - after the tests yesterday, I performed an OTA update to clear everything. Of course, all files from LFS were wiped. I thought LFS was back to 32K, but now when I was doing the last format command, I noticed LFS was remembered as 0x40000, i.e. 256K. I didn't check LFS size before formatting though...

    Added after 11 [minutes]:

    @jkwim

    As you can see from the next screenshot, I am able to fill the whole 200K fs. File Breaker_rev4.html was the last one copied, but it is already beyond LFS size....

    Screenshot of a web application for managing LittleFS files.

    So, upgrade the firmware and format LFS with no more than 200K (0x32000) and try again - these are my recommendations
  • #11 20934224
    jkwim
    Level 12  
    Hmm...

    I wonder whether I am doing anything wrong not to get the same result as yours. I will give a thought again.

    I did create some test files of different sizes using:

    Code: text Expand Select all Copy to clipboard
    perl -e "print '1' x 32739" > 32739.txt


    The max size that worked was 48kB. I could add another few files to bring the total to 100kB or so. However when I backed them up in to a tar and tried to upload again (after deleting the original files) only one or two files get restored.

    I had the LittleFS set for 0x32000 (same as yours). I upgraded the BK7231N device to 1.17.404 and tried.

    (There would have been a performance issue when I ran the original tests because I has opened two windows of WebApp, one for Logs and one for FS. I saw some error talking about failure to start HTTP TCP thread. So I suspected that I may be stressing the device too much. So I repeated the test using only one instance of WebApp).

    Perhaps I should try on a different device not a smart wall switch when config may be slimmer.
  • #12 20934454
    Angel0fDeath
    Level 13  

    For vue.min, you can try not to copy the file but to create the file on the device, then open it and copy and paste the code. Then save the file.
    I don't know what is wrong with you, but definitely with me, there is no problem.
  • #13 20937919
    jkwim
    Level 12  
    I was busy for the last few days and tried to revisit the issue using a different device.
    Upgraded the device to 1.17.404

    NO PROGRESS!

    Wonder what I may be doing wrong.

    1. Tried two different browsers, Firefox & Chrome
    2. Captured Wireshark trace and confirmed that HTTP request contains the entire file content. So I can eliminate any PC side issue.

    16kB size appears to be working consistently. 32kB size appears to be hit and miss.

    Symptom is same. Device reboots after writing few blocks.

    Screenshot of an application's interface with file management functions, such as file upload and file list.

    The only other option that I have is to change the wireless router (which I doubt will make any difference).

    Added after 5 [hours] 9 [minutes]:

    Change of router did not help.

    So it should be something happening at the device.

    Some sort of buffering issue or corrupt data?
  • ADVERTISEMENT
  • #14 20940515
    Angel0fDeath
    Level 13  

    I also don't know what is wrong.
    Did you try to create an empty file on the device and copy/paste the code into the editor?
    About browsers - I'm using Microsoft Edge.
  • #15 20940595
    p.kaczmarek2
    Moderator Smart Home  
    I didn't have time to investigate it yet, but do you have PowerSave or any driver running in a background?
    Helpful post? Buy me a coffee.
  • #16 20940756
    jkwim
    Level 12  
    I am also confused about my observations.

    The last device that I used (3rd device) is a "16A Mini Smart Switch CB2S BK7231N" 1.17.437

    Screenshot of a smart switch interface displaying status information.

    Since you mentioned I checked and Powersave was ON

    Code: text Expand Select all Copy to clipboard
    Info:CMD:CMD_PowerSave: will set to 1
Info:CMD:[WebApp Cmd 'powersave' Result] OK


    1. I have tried opening blank file and pasting text
    2. I even tried pasting text incrementally to see where it fails. Around 17kB it fails
    3. Uploaded number of smaller files (max 16kB per file) and I can continue till about 100kB. I changed the partition size to 128k for this test.
  • #17 20941264
    Angel0fDeath
    Level 13  

    I performed the tests on the smart breaker BK7231T with the WB3S module. After the LFS format and reboot, no drivers are running, since I enable the drivers via autoexec.bat. Also, all flags are off except flag 46, which is enabled for normal operation of the device.
    For the test, try to disable all drivers and flags. Even the NTP driver.
  • Helpful post
    #18 20941452
    p.kaczmarek2
    Moderator Smart Home  
    I've did some testing and managed to get the problem that @jkwim mentioned. After few more tests, I've concluded that adding a RTOS delay in the file upload function can help.

    I've drag and dropped all WebApp files on LittleFS:
    File editor interface with a list of files in the LittleFS system.

    Here's a test build with stability fix included:
    https://github.com/openshwprojects/OpenBK7231T_App/pull/1056
    Can you both , @jkwim and @Angel0fDeath test this build and let me know if it works better for you?
    Helpful post? Buy me a coffee.
  • #19 20942411
    Angel0fDeath
    Level 13  
    @p.kaczmarek2 @jkwim Ok. This time I made a test with 3 different devices.

    Test1 - device BK7231T with WB3S (Breaker) - firmware 1.17.432 - works perfectly.
    Test2 - device BK7231T with WB3S (Breaker) - firmware OpenBK7231T_delay_for_littlefs_ca445c8961ba - works perfectly.
    Test3 - device BK7231T with WB3S (IR Remote with temp/hum) - firmware 1.17.432 - I was not able to copy the file vue.min.js. However, I'm able to copy a file with a size of 44259 bytes.
    Test4 - device BK7231T with WB3S (IR Remote with temp/hum) - firmware OpenBK7231T_delay_for_littlefs_ca445c8961ba - works perfectly.
    Test5 - device BK7231N with CB2S - firmware 1.17.432 - I was not able to copy the file vue.min.js (about 92K). I was not able to copy a file with a size of 44259 bytes.
    Test6 - device BK7231N with CB2S - firmware OpenBK7231N_delay_for_littlefs_ca445c8961ba - I was not able to copy the file vue.min.js. However, I'm able to copy a file with a size of 44259 bytes.
    lfs size in all tests above is 200K (0x32000).

    It looks like there is some problem with N devices. Whatever I do, I'm not able to copy vue.min.js on N device.


    Added after 3 [hours] 18 [minutes]:

    I was finally able to copy vue.min.js on N device, but with some little work around.
    I copied the file to T device and downloaded .tar file, then I dropped the tar file on N device and it works.

    Firmaware on N device - OpenBK7231N_delay_for_littlefs_ca445c8961ba

    Attached is corresponding tar file from T device
    @jkwim Please try with the attached tar file
    Attachments:
  • #20 20942900
    Angel0fDeath
    Level 13  

    And one last test: I was not able to upload tar file with firmware 1.17.443 on N device. Didn't test it with T device
    @p.kaczmarek2 I think your fix is working, so you can merge it into the official release.
  • ADVERTISEMENT
  • #21 20942990
    p.kaczmarek2
    Moderator Smart Home  
    Maybe this fix can be tuned futher to fix your N issues?
    Helpful post? Buy me a coffee.
  • #22 20943072
    Angel0fDeath
    Level 13  

    @p.kaczmarek2 In general, I don't have an issue with this. I'm hosting a web app on my NAS with Docker. If @jkwim can confirm my tests and if it is working for him, it will be good. Just trying to help.
    But I don't know what is going on. The reason we are not able to upload/copy the file is device reboots - this can be seen in the logs page (log level debug), and I think this is probably due to low memory, but not sure. I don't know why this does not happen with T devices. Moreover, I cannot test this on other types of devices - just don't have such...
    Such spontaneous reboots usually occur (by me with my esp8266 and esp32 devices) when the internal watchdog is not fed (for instance within a very long cycle), so from time to time in such cases, I'm issuing the yield() command. I don't know if this is applicable for BK7231 devices. The other thing - if it is low memory - probably the upload/copy buffer should be checked... Really don't know, just ideas.
  • #23 20943160
    jkwim
    Level 12  
    p.kaczmarek2 wrote:
    I've did some testing and managed to get the problem that @jkwim mentioned. After few more tests, I've concluded that adding a RTOS delay in the file upload function can help.

    I've drag and dropped all WebApp files on LittleFS:
    File editor interface with a list of files in the LittleFS system.

    Here's a test build with stability fix included:
    https://github.com/openshwprojects/OpenBK7231T_App/pull/1056
    Can you both , @jkwim and @Angel0fDeath test this build and let me know if it works better for you?


    is there a direct link to a binary that I can quickly download?
  • #24 20943214
    Angel0fDeath
    Level 13  
    @jkwim Here;
    Code: text Expand Select all Copy to clipboard
    https://pipelinesghubeus25.actions.githubusercontent.com/4Xw7PsXpqsl3BbjpdO8r7bfQ8DnPQoHACkdPhNFilyJSUGfp4T/_apis/pipelines/1/runs/2884/signedartifactscontent?artifactName=OpenBK7231T_App_1056_merge_ad333e21dec2&urlExpires=2024-02-03T18%3A26%3A42.6955336Z&urlSigningMethod=HMACV2&urlSignature=dA4JxryeFqPh9BABlZeomKGjl%2Bg4FjcF5la02yuaw00%3D


    In general - Follow the link p.kazcmarek2 sent. Then in the window "All checks are passed" click on any "Details"
    Then you should have this window:

    Screenshot from GitHub showing the activity page of the OpenBK7231T_App project.

    Click Summary, as indicated in the screenshot above then scroll to the end where you will find zip file with all builds. Download, extract and flash your device with correct rbl file
  • Helpful post
    #25 20943770
    jkwim
    Level 12  
    Angel0fDeath wrote:
    @jkwim Here;
    Code: text Expand Select all Copy to clipboard
    https://pipelinesghubeus25.actions.githubusercontent.com/4Xw7PsXpqsl3BbjpdO8r7bfQ8DnPQoHACkdPhNFilyJSUGfp4T/_apis/pipelines/1/runs/2884/signedartifactscontent?artifactName=OpenBK7231T_App_1056_merge_ad333e21dec2&urlExpires=2024-02-03T18%3A26%3A42.6955336Z&urlSigningMethod=HMACV2&urlSignature=dA4JxryeFqPh9BABlZeomKGjl%2Bg4FjcF5la02yuaw00%3D


    In general - Follow the link p.kazcmarek2 sent. Then in the window "All checks are passed" click on any "Details"
    Then you should have this window:

    Screenshot from GitHub showing the activity page of the OpenBK7231T_App project.

    Click Summary, as indicated in the screenshot above then scroll to the end where you will find zip file with all builds. Download, extract and flash your device with correct rbl file



    Ok, I had to click on "Artifacts" to see this :

    Screenshot showing warning messages from OpenXR and OpenW about deprecated Node.js actions.

    Added after 1 [hours] 48 [minutes]:

    SUCCESS!

    I was able to upload the following in to a 128kB FS in OpenBK7231N_1056_merge_ad333e21dec2 on a N (CB2S) device.

    However, I also encountered the issue with vue.min.js as with @Angel0fDeath. I used his tar file instead.

    Will test on all modules that I have WB3S, CB3S as well.

    User interface of a file upload tool with debug console

    Added after 1 [hours] 35 [minutes]:

    I uploaded following files with modifications made to the file paths as indicated in the first post in this topic.
    List of JavaScript and HTML files with their sizes in bytes.

    The indexlocal.html file contains following:

    Code: text Expand Select all Copy to clipboard
    <!DOCTYPE html>
<html>
    <head>
        <script>
            window.root = '/';
            window.device = 'http://127.0.0.1';
        </script>
        <script src="/startup.js"></script>
starting ...
    </head>
<body>
</body>
</html>


    I accessed the page using:
    Code: text Expand Select all Copy to clipboard
    http://192.168.1.219/api/lfs/indexlocal.html


    I can see the test string that I have added in to indexlocal.html in browser:
    Code: text Expand Select all Copy to clipboard
    starting ...


    And nothing more.

    I used the original indexlocal.html file from WebApp.

    Wonder there is more changes required in the paths.

    Added after 2 [hours] 56 [minutes]:

    Further, confirmed that startup.js is getting invoked by following modifications:

    Following files are in
    Screenshot showing the structure of JavaScript files in a browser along with the size of each file.

    Following components have been allowed in startup.js

    Code: text Expand Select all Copy to clipboard
        window.components = [
        { name:'device', path: window.root+'device.vue'+unique },
 //       { name:'ota', path: window.root+'ota.vue'+unique },
//        { name:'filesystem', path: window.root+'filesystem.vue'+unique },
        { name:'info', path: window.root+'info.vue'+unique },
 //       { name:'flash', path: window.root+'flash.vue'+unique },
        { name:'control', path: window.root+'control.vue'+unique },
//        { name:'tools', path: window.root+'tools.vue'+unique },
 //       { name:'import', path: window.root+'import.vue'+unique },
        { name:'logs', path: window.root+'logs.vue'+unique },
 //       { name:'gpioDoctor', path: window.root+'gpioDoctor.vue'+unique },
        { name:'myComponent', path: window.root+'myComponent.vue'+unique },
    ];



    Screenshot of HTML file editing with code highlight and annotation.

    Screenshot showing the editing of the startup.js file with a highlighted code line

    Screenshot of a browser displaying a simple HTML page.

    Added after 26 [minutes]:

    Some further progress:

    By removing the '/' window.root I was able to get to the screen shown with no data nor links working:

    Device configuration with unknown settings and command input field.
    Screenshot of HTML file editing with source code.

    Here is the debug window from browser

    Screenshot of a browser debugging tool showing loaded HTML, JavaScript, and CSS code.

    So looks like the javascript files are getting read.

    Now it is the matter of how to incorporate with the internal API
  • #26 20944254
    Angel0fDeath
    Level 13  
    @jkwim According to me, when you open this HTML file (indexlocal) from your PC, 127.0.0.1 will be your PC, so again according to me window.device should point to the IP address of your device and window.root should point to /api/lfs/
    In this case, you probably will need also to add the path (/api/lfs) in front of startup.js

    Then you need to configure the device to use its own served web app - go to device's web page then Config -> Configure Webapp and enter the address of your web app i.e. http;//ip/api/lfs/indexlocal.html.
    Now if everything is ok you after pressing Launch Web Application in device's main web page it should work.

    For the other devices you just need to configure Webapp address.

    Then access will be through device web page and pressing Launch Web Application, or directly ip/app?
  • #27 20944917
    jkwim
    Level 12  
    SUCCESS STORY!

    Thanks to @Angel0fDeath and @p.kaczmarek2 for helping along the way!

    Managed to get it working with following setup:

    1. FS size 0x34000 (212kB)

    2. Minified httpVueLoader.min.js using https://www.toptal.com/developers/javascript-minifier. Made the file size 10,890 bytes -> 5,867 bytes

    3. I missed out the most important filesystem.vue in my previous posts. So final content is:

    List of files with sizes next to each file, detailing the configuration of a web application.

    4. Modified startup.js as follows (note the modified paths also):

    Screenshot of a code editor with an edited startup.js file, featuring a script to load vue.min.js and httpVueLoader.min.js files.

    Screenshot of a text editor showing JavaScript code for editing the /startup.js file.

    5. Configured indexdevice.html with IP address (note that indexdevice.html is not really required unless you want access the WebApp as a direct URL)

    Code: text Expand Select all Copy to clipboard
    <!DOCTYPE html>
<html>
    <head>
        <script>
            window.root = '/api/lfs/';
            window.device = 'http://192.168.1.219';
        </script>
        <script src="/api/lfs/startup.js"></script>
    </head>
<body>
</body>
</html>


    6. Configured WebApp

    OpenBK7231N-NW9-219 configuration interface with entered URL.

    7. Viola!

    Screenshot displaying information about the current device, including IP address, MAC address, and a TypeError.

    Additional Comments:


    1. File upload still have issues for larger files.

    filesystem.vue & vue.min.js had to be uploaded as tar files. Probably it is better to follow the same for info.vue.

    2. Restricting the FS size is an issue. 212kB still gives a Low Heap warning.

    3. If there is a way to MINIFY the .vue files we could shrink the file sizes significantly as there is lot of whitespace in these files. Perhaps then we can limit everything to 200kB. This option should be considered.

    4. There will be an error in Debug log about devices.json. This can be ignored.

    Code: text Expand Select all Copy to clipboard
    Debug:API:GET of api/lfs/devices.json
Debug:API:LFS read of devices.json
Debug:API:LFS open [devices.json] gives -2
Debug:API:failed to open devices.json lfs result -2


    5. Having to hard code IP address in WebApp might be an issue for non static DHCP environments. If a capability is developed to read the device IP address and adapt it wouild be nice.

    6. Restoring the full backup tar file may not possible. Workaround would be to manually upload the rest of the files. What I found to be working is to remove the 2 or 3 largest files from the tar file offline. Then upload the largest files as individual tar files first. Then upload the tar file with the remaining files.
  • Helpful post
    #28 20944940
    Angel0fDeath
    Level 13  
    @jkwim I'm glad finally everything is ok by you. Enjoy :)

    EDIT: It looks like there are less problems with upload/copy files to T devices, so you can consider hosting on T device.
    As always @p.kaczmarek2 mentions, you can consider writing a tutorial or posting final solution in new topic in order not to be buried on the last page of this topic (although this is the first page... :) )
    Regards

    Added after 23 [minutes]:

    @jkwim Concerning vue files you can probably check this:
    Code: text Expand Select all Copy to clipboard
    https://marketplace.visualstudio.com/items?itemName=MartinCincurak.vue-component-minify


    Never tried it though...

    Added after 57 [minutes]:

    1 more idea - if your router (although your network is closed something should issue/register IP addresses) has accessible via LAN USB port you can host web app on a small usb flash drive or SD card with reader.
  • Helpful post
    #29 20946139
    jkwim
    Level 12  
    Minification - The Way Forward


    I managed to further improve the storage space requirements by manually Minifying parts of the .vue contents

    1. Managed to shrink the partition size to 196kB 0x30000. Still I get the "Error:MAIN:Low heap warning!"

    2. The .vue files have the following structure:

    Code: text Expand Select all Copy to clipboard
    <template>
  .
  .
  .
</template>

<script>
  .
  .
  .
</script>

<style scoped>
  .
  .
  .
</style>


    3. The <script> section can be minified using https://www.toptal.com/developers/javascript-minifier

    Just cut and paste the contents between the tags (do not include these two tags but only the contents in between)

    Code: text Expand Select all Copy to clipboard
    <script>
  .
  .
  .
</script>


    Once it is minified copy back the minified content replacing the original content.

    4. The <style> section can be minified using https://www.toptal.com/developers/cssminifier

    Just cut and paste the contents between the tags (do not include these two tags but only the contents in between)

    Code: text Expand Select all Copy to clipboard
    <style scoped>
  .
  .
  .
</style>


    Once it is minified copy back the minified content replacing the original content.

    5. For larger .vue files you would find the the file sizes can be reduced significantly

    6. In my case I managed to shrink the file sizes as follows:

    List of files with names and sizes after minification, including several .vue files.

    7. I named the minified files as .min.vue and modified the startup.js to reflect the same:

    Code: text Expand Select all Copy to clipboard
        window.components = [
        { name:'device', path: window.root+'device.vue'+unique },
//        { name:'ota', path: window.root+'ota.vue'+unique },
        { name:'filesystem', path: window.root+'filesystem.min.vue'+unique },
        { name:'info', path: window.root+'info.min.vue'+unique },
//        { name:'flash', path: window.root+'flash.vue'+unique },
//        { name:'control', path: window.root+'control.vue'+unique },
//        { name:'tools', path: window.root+'tools.vue'+unique },
//        { name:'import', path: window.root+'import.vue'+unique },
        { name:'logs', path: window.root+'logs.min.vue'+unique },
//        { name:'gpioDoctor', path: window.root+'gpioDoctor.vue'+unique },
        { name:'myComponent', path: window.root+'myComponent.min.vue'+unique },
    ];


    If anybody wants to test this here is a tar file with modified files. It is a zipped tar file.


    Ofcourse when the GitHub pages get updated you need to manually make the updates again.
    Attachments:
  • #30 20946904
    Angel0fDeath
    Level 13  

    @jkwim I'm really not interested in this approach, just wanted to help, but will find some time during the weekend to test it for better confirmation and will report the result.
    Btw vue.min.js contains probably a lot of functions which you don't use...So you can probably elaborate vue.min.min.js by removing what is not necessary. But be careful :)
    You also don't need indexlocal.html and indexdevice.html - although these files are about 200 bytes, both files are occupying 8K - 2 blocks 4K each. So probably you can reconsider your idea why we need both files.
    Don't forget each file occupies 4K blocks, regardless of the size. So compare filesize of your min.vue files, calculate how much blocks file occupies and if there is no difference with the original file - don't minify this file. If there is a difference of at least 1 block (4K) - then minify it.
    For instance - info.min.vue is 12344 bytes (3.01 blocks which means 4 blocks), but it occupies 16K. If the original file occupies also 4 blocks - no minification is necessary.
    According to me, you can remove a lot of unused functions and significantly lower the size of vue.min. This is the largest file and therefore you can win more space from this file.
    I hope this will help for further improvement.
    @p.kaczmarek2 I'm pretty sure the min block size for lfs is 128 bytes. Is it possible LFS to be with less block size? Or configurable? With 1K blocks less space will be wasted. This is the reason min block size is 128 bytes, not to lose so much space. Then everything @jkwim is doing makes sense.
    I mean something like lfs_format
Reply | New topic
Report a violation of the law

Topic summary

The discussion revolves around hosting the OpenBeken IoT Web App's 'Logs' function on the BK7231T device using LittleFS. Users explore the limitations of LittleFS, initially set at 32kB, and the challenges faced when attempting to upload files such as vue.min.js and httpVueLoader.js. Various strategies are proposed, including increasing the LittleFS size to 128kB, 200kB, and even 256kB, while addressing issues like device reboots and low heap warnings during file uploads. Suggestions include rebooting after formatting, using minified files to save space, and implementing a delay in the upload function to enhance stability. Successful configurations involve modifying file paths in startup.js and using a specific firmware version that accommodates larger file sizes. The conversation concludes with users successfully uploading files and discussing the potential for a tutorial on the process.
Summary generated by the language model.
Popular Topics
ADVERTISEMENT