logo elektroda
logo elektroda
X
logo elektroda
Advanced Search
logo elektroda

ESP8266 Wemos D1 - Handling String Overflow and Reboots with Large Files

jaskol 1683 16
Best answers

How can I avoid heap overflow and reboots when reading and editing a large 20 kB file into a String on an ESP8266 Wemos D1?

You should not read the whole 20 kB file into a `String` with `readString()`, because long/repeated `String` operations can fragment the heap and trigger crashes; use `readBytes()` into a fixed `char[]` buffer instead, or process the file in chunks/stream it directly without keeping the whole file in RAM [#19849244][#19849584][#19849190] `String buffer[500]` is especially bad here because it creates many separately allocated `String` objects on the heap, which quickly runs out of space [#19849584] The free heap reported by the thread was only about 20992 bytes, so a 20 kB file is already right at the limit on the ESP8266 [#19848620] Another practical fix is to keep the HTML template static and move the changing values elsewhere, for example into a separate JS file or generate them client-side, instead of manipulating one large server-side file [#19849593][#19850386] `reserve()` can only help if you already know the size, but it does not solve the basic ESP8266 memory limit [#19848067]
Generated by the language model.
ADVERTISEMENT
Treść została przetłumaczona polish » english Zobacz oryginalną wersję tematu
Report a violation of the law
Reply | New topic
  • #1 19847872
    jaskol
    Level 12  
    Posts: 112
    Rate: 3
    Hello everyone.
    I'm struggling myself, but I'm sure someone has had this problem before and will give some suggestions. I'm loading a file, 20kB, into a string (SPIFFS or LittleFS - makes no difference), substituting some strings and saving in a second file. As long as the file was small (I can't remember, maybe 10kB, but I'm guessing), it all worked fine. Now I see that once every few writes the processor reboots. I've turned on debug and I get the message every so often:
    [String] Reallocating large String(191 -> 192 bytes) [.......] Then I guess the stack overflows, in any case a reboot occurs.
    I can split this file into several smaller ones, but it messes up my concepts. I would prefer to know how to handle this. I'm not a programmer, and I'm sure there's someone here who knows right away whether to put this in some kind of array or solve it in yet another way.

    I'm not a programmer, please don't shout ;) .
    It simply loads the contents of the file into a string variable:

    String Config_Webpage = "";

    File file_init = LittleFS.open("/config_init.html", "r");
    file_init.setTimeout(5000); // could be 1000, could be 5000, makes no difference - anticipate the question why so much

    if (file_init) {
    Config_Webpage = file_init.readString();
    }

    I thought the program was crashing when writing to the file, but I did the traps and debug after RS showed that it was already crashing at the file read stage. Does anyone know how much you can "fit" in a String type variable ?

    I'd just like to add that if I declare an array with a fixed size, this won't quite solve the problem either, as this file can have a different size (+/- 1kB). I wouldn't want to permanently allocate an area of memory, because it's known that there's not enough of that all the time. Unless I can somehow neatly allocate and release.
    Please help, because I don't want to compromise myself any further, and something tells me that such problems were solved ages ago....

    Regards,
    Mariusz
  • ADVERTISEMENT
  • #2 19848021
    mpier
    Level 29  
    Posts: 817
    Help: 153
    Rate: 141
    Hello,
    Now you are just trying to "gracefully allocate and release". Read all the data into a buffer previously reserved, but don't release it after use. Do you even need a whole file in RAM to swap a few bytes?

    Greetings.
  • ADVERTISEMENT
  • #3 19848058
    jaskol
    Level 12  
    Posts: 112
    Rate: 3
    mpier wrote:
    Hello,
    Now you are just trying to "gracefully allocate and release". Read all the data into a buffer previously reserved, but don't release it after use. Do you even need a whole file in RAM to swap a few bytes?

    Greetings.
    .
    Hi.
    I felt something like that, but was afraid to speak out loud ;) .
    I've already started doing this buffer because no one was responding, we'll see.
    If I don't slow it down, at least nothing will occupy that area of memory and maybe there'll be less chance of a "dump" in the future.
    I don't know how else to bite it.
    What I've done is that I have an html master file (config_init.html) where I have a web page with the device configuration.
    I have keywords in it, in which I change the values, and I've already imported the new ones into the second file, the one displayed to the user.
    You make changes on the website, a form is sent, ESP ingests it into a String, the "replace" function changes what it needs to in this String (depending on what you changed on the website) and saves it to the config.html file.
    I've been combining to load a few lines at a time, but it's pointless, as it takes a long time and the String can overflow anyway.
    On the web you enter your email for example, so the size of this file could change.
    I declared a String and thought I'd have peace of mind, because the processor would already take care of it, to "stuff" it properly in RAM.
    Well it does stuff it, but up to a certain point ;) .
    The question is whether it's known how many bytes will fit in there, because maybe I could control that and then step in ?
    I am making this buffer but would love to know if there is not a better way to solve my problem, sometimes a person's eyes flap....

    M.
  • #4 19848067
    mpier
    Level 29  
    Posts: 817
    Help: 153
    Rate: 141
    If it is known in advance that there is not enough memory, it is safe to assume that the programme will not work properly anyway, and if it does work, there is not too little memory.

    You should be able to fit as much as you reserve. See reserve(). Judging by the name it makes sense, you need to check in the sources if it will be ok.

    Can't you save the data separately? You could then populate the data on the fly and leave the HTML template read-only. There must be some ready-made library for this.
  • #5 19848620
    jaskol
    Level 12  
    Posts: 112
    Rate: 3
    mpier wrote:
    If it is known in advance that there is not enough memory, it is safe to assume that the programme will not work properly anyway, and if it does, the memory is not too little.

    You should be able to fit as much as you reserve. See reserve(). Judging by the name it makes sense, you need to check in the sources if it will be ok.

    Can't you save the data separately? You could then fill in the data on the fly and leave the HTML template read-only. There must be some ready-made library for this.
    .

    I do tests and fire up the program with:
    Free heap size: 20992 (bytes)
    From what I've read, that's just my amount of free RAM.
    20kB, that's a bit low. I read that the ESP8266 has 80kB at the start.
    So indeed a miracle is not going to happen. I'll either split the file into smaller parts, or switch to ESP32.
    I don't know how I would populate the data on the fly, I have it stored in an array and this is how I extract it from it,
    I look for a string in the file loaded into String, replace the string with that data and so on until the search is complete.
    Then I save the whole thing to a file.
    I have to load the whole file to search it and replace the data in one go, replace() function.
    I could line by line, but I still have to put everything together later to save to the file - saving after each change rather
    would be time-consuming.
    I could still rigidly make the file and I'd know which line I have which variable to replace, but that would limit me because any expansion of that file would involve re-analysing what's in which line, what's moved, etc.
    The only thing I've come up with now is that I'll make a start and end tag for the "proper" data and instead of loading the file from the beginning, I'll load it, but discard everything that was before the start tag (html headers and such).
    Just how much will I save ? Some small :( .
    I keep testing, but I can see that this is a major limitation.
    You write yourself a program, the memory is running out, although you think the data takes up 66% according to the compiler.
    Later you count and it turns out that there are a few K left and death.
    I have versions with ESP32 and PSRAM, probably time to get interested....

    Mariusz
  • ADVERTISEMENT
  • #6 19849190
    mpier
    Level 29  
    Posts: 817
    Help: 153
    Rate: 141
    After all, you don't have to load the whole file at once. You don't have space for 20kB, but you can easily fit 2kB. You can send the text from the file with the substituted variables directly to the client or write it to a file.
    You have nothing to analyse after changing the contents of the file, if you write, for example, like this:
    Code: HTML, XML
    Log in, to see the code
    .
  • #7 19849244
    Anonymous
    Level 1  
  • #8 19849315
    jaskol
    Level 12  
    Posts: 112
    Rate: 3
    mpier wrote:
    After all, you don't need to load the whole file at once. You don't have space for 20kB, but you can easily fit 2kB already. You can send the text from the file with the substituted variables directly to the client, or save it to a file.
    You don't have anything to analyse after changing the contents of the file if you write, for example, like this:
    Code: HTML, XML
    Log in, to see the code
    .
    The problem is that this file has quite a lot of text in HTML+javascript to make it look like something.
    That's where the 20kB comes from.
    It doesn't want to upload directly, because it's such a MENU and tabbed file - different options divided into sections, tabs switch, but everything happens in one file. There are, for example, 5 or 6 independent forms. Generally, it's rare to change anything in the configuration of the device and you can afford to then wait, say, 3 seconds for a full zap and information on the OLED that "saved".
    If I were to split this into chunks of 2kB each, not quite enough time, but still another code to handle.
    What if in that 2kB I hit half of the name of some variable of mine and read %NAZ, and in the other 2kB is the rest, i.e. WISKO% ?
    It's possible to read in a few lines at a time, waiting for "\n", but I can already see that in a while I'll run out of memory for other things.
    I've started a program that shows me the occupancy with each line loaded from the file, and I can see how it gets off - e.g. when the declared array String buffer[500] overflows. It will load >500 and it falls down. If I give 600, it's still not enough and it crashes, but later.
    It's even the case that if I have the entire target batch and check it anyway, it already does a reboot when RAM is at 400B, and if I throw everything away and leave just the routine to test, I can get down to 4B of memory and it still works.
    I'm already switching a bit to the ESP32, but it's killing me again, because although the RAM occupancy is 14%, but the program memory is 72% ! And it's the same program. And as far as I remember, in ESP8266 I had a Flash of 4MB, while in ESP32 I have LOLIN D32 PRO and 16MB Flash ! Partition scheme I change, but somehow I still feel that some misery remains. 16MB and so busy ?

    Chip is ESP32-D0WD-V3 (revision 3)
    Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
    Crystal is 40MHz
    Auto-detected Flash size: 16MB

    Someone comfort me that I'm doing something wrong or reading something wrong, or will it turn out that I gain RAM and run out of FLASH in a while ?

    Thanks,
    Mariusz

    Added after 1 [minute]: .

    khoam wrote:
    Why are you using file_init.readString()? Doing so can actually overflow the heap when operating on long Strings, and there is also fragmentation of that heap when loading strings repeatedly. Better to use file_init.readBytes() and load the data into a static array (stack allocation).
    Link [/quote

    Thanks, I'll check how it behaves, but anyway, that space for that 20kB has to be found, sooner or later.
    Is it possible to somehow release the occupied space in the array when I no longer need it ?
    I've done this quickly, but I have the impression that if I fill up such an array String buffer[600] with the contents of the file,
    then the subsequent String buffer[600]="" didn't do anything.
    Is there an instruction to "clear" such an array entirely and free up memory?

    Mariusz
    .
  • #9 19849584
    Anonymous
    Level 1  
  • Helpful post
    #10 19849593
    janek_wro
    Level 29  
    Posts: 1281
    Help: 123
    Rate: 290
    jaskol wrote:
    The problem is that this file has quite a lot of text in HTML+javascript to make it look like something.
    That's where the 20kB comes from.
    It doesn't want to upload directly, because it's such a MENU and tabbed file - different options broken down into sections, tabs switch, but it all happens in one file.
    .
    Maybe you are approaching this from the wrong side. Why manipulate a sizable file on this ESP?
    Make one file (hmtl+js), fully static content. Put the variables in a separate js file. Smaller.
  • #11 19850386
    mpier
    Level 29  
    Posts: 817
    Help: 153
    Rate: 141
    jaskol wrote:
    And how in these 2kB will I hit half of the name of some variable of mine and read %NAZ, and in the other 2kB will be the rest, that is WISKO% ?
    .
    You won't hit the half because the %NAZ will always be at the beginning of the buffer, and the %CHANGENAME% will be much shorter than the length of that buffer.

    If you are writing in JS, you can substitute variables in the browser too.

    You must have some other big bug in your program that such problems are coming out.
  • #12 19850480
    janek_wro
    Level 29  
    Posts: 1281
    Help: 123
    Rate: 290
    mpier wrote:
    You won't hit half because the %NAZ will always be at the beginning of the buffer
    Further substantiate this claim. Because somehow I don't feel what you mean. After all, such a placeholder can occur anywhere in the template file, including where the later "break" will fall after reading a fixed number of bytes into the buffer.
    I guess that the reading into the buffer should only occur up to the occurrence of % (or whatever character(s) the template uses). And the rest in the next round.
    Or read 1 line at a time, i.e. up to CRLF. As long as the template has a CRLF, because it does not have to.

    mpier wrote:
    If you write in JS, you can substitute variables in the browser too.
    If the intended use assumes that the client will be online, in the sense of accessing the world, then you can even take the jquery/ui link from the CDN, and use its benefits. Because it's probably not enough space to build in a device?
  • ADVERTISEMENT
  • #13 19851191
    jaskol
    Level 12  
    Posts: 112
    Rate: 3
    janek_wro wrote:
    jaskol wrote:
    The problem is that this file has quite a lot of text in HTML+javascript to make it look like something.
    That's where the 20kB comes from.
    It doesn't want to upload directly, because it's such a MENU and tabbed file - different options broken down into sections, tabs switch, but it all happens in one file.

    Maybe you're approaching this from the wrong side. Why manipulate a sizable file on this ESP?
    Make one file (hmtl+js), fully static content. Put the variables in a separate js file. Smaller.

    YES, this could be the solution to my problem !
    Well, and it will be fast.
    Thanks, I'll check it out, although I've already switched to ESP32 and as soon as I get rid of some urge in that webserver to serve everything in "gz" and in filesystem to add some prefixes (I've set what I need to "1" and it still adds "/littlefs", then maybe I'll stay with ESP32.
    However, if it turns out that with the ESP8266 and 4MB Flash I was at 60% occupancy, and with the ESP32 and 16MB I'm at 70%, I'll get really mad. I'm about to start implementing your idea, on this ESP32, here too it's better to have less to write and faster running code.

    Thanks,
    Mariusz

    Added after 4 [minutes]: .

    khoam wrote:
    jaskol wrote:
    khoam wrote:
    Is it possible to somehow free up the occupied space in the array when I no longer need it ?
    .
    How often will you use this memory area in the program? Just once? You can allocate and release the desired memory area dynamically (using new/delete), but in the case of 20kB I "thinly" see this on the ESP8266 (the program itself also uses the heap). It is certainly better to use a char[] array instead of String class objects.

    jaskol wrote:
    String buffer[500].
    .
    This way it's bound to be problems. Each String can be of different length , there will be allocated 500 areas of different size on the heap and it will run out of space.

    I understand, it's just that char was "not working out" for me something in practice.
    Even once it's ok, I'm worried that in a while I'll have 10kB of free memory left and it'll be over.
    If I free up memory and then use it up with something else and there's not enough to load the file, it'll all blow up too.
    As if they couldn't give a bit more of that RAM, after all, I would pay extra ;) .

    Mariusz

    Added after 8 [minutes]:

    janek_wro wrote:
    mpier wrote:
    You won't hit half because %NAZ will always be at the beginning of the buffer
    Further substantiate this claim. Because somehow I don't feel what you mean. After all, such a placeholder can occur anywhere in the template file, including where the later "break" will fall after reading a fixed number of bytes into the buffer.
    I guess that the reading into the buffer should only occur up to the occurrence of % (or whatever character(s) the template uses). And the rest in the next round.
    Or read 1 line at a time, i.e. up to CRLF. As long as the template has a CRLF, because it does not have to.

    I use string.replace() and it seems to work by looking for occurrences of the given string in the string.
    This is how I replace variables with values. That is, it has to find the whole string %NAME% to replace that with KOWALSKI.
    Yes, I've written about being able to load line by line, but if I go through each line each time and load the next line, the whole thing gets rather nightmarishly long.

    mpier wrote:
    If you're writing in JS, then you can substitute variables in the browser as well.
    If the intended use is that the client will be online, in the sense of accessing the world, then you can even take the jquery/ui link from the CDN, and use its benefits. Because in a device is probably not enough space to build in?
    .
    Yes, I've got jquery in the header because I've found that it works faster when taken from the net than when read from the filesystem.
    I wanted to be independent, I had everything locally and it was definitely running slower than when it was "from the net".
    It's also about page caching, browser caching and such, so it turns out that it's more efficient to load from the web.
    If the network is down, then I'll still be testing, because everyone assumes that the network always works....

    Mariusz
  • #14 19868860
    kaczakat
    Level 34  
    Posts: 1748
    Help: 317
    Rate: 229
    I have the web page saved in the INO file as follows:
    Code: text Expand Select all Copy to clipboard
    WebSocketsServer webSocket = WebSocketsServer(81);

static const char PROGMEM INDEX_HTML[] = R"rawliteral(
<!DOCTYPE html>
<html>
<head>
......
</html>
)rawliteral";
    .
    My first working example is THERE .

    It passes it to the client in its entirety, and then it is the client who substitutes the content for itself via Websockets. You can also move it to a separate project file and conveniently swap it with another one. In ESP, you can also simply use an html file uploaded to SPIFFS, in which case the page can be split into separate files. Attached is a game made for the World Wide Web according to the Youtube channel's HTML course "The Passion of Computer Science", html, css, js, graphics, core ESP 2.5.0. with SPIFFS you upload the ESP code separately and the contents of the flash drive separately.
    Attachments:
    • FSBrowser_gra_w_wisielca.zip (364.96 KB) You must be logged in to download this attachment.
    Helpful post? Buy me a coffee.
  • #15 19868908
    jaskol
    Level 12  
    Posts: 112
    Rate: 3
    kaczakat wrote:
    I have the web page saved in an INO file like this:
    Code: text Expand Select all Copy to clipboard
    WebSocketsServer webSocket = WebSocketsServer(81);

static const char PROGMEM INDEX_HTML[] = R"rawliteral(
<!DOCTYPE html>
<html>
<head>
......
</html>
)rawliteral";
    .
    My first working example is THERE

    It passes it to the client in its entirety, and then it is the client who substitutes the content for itself via Websockets. You can also move it to a separate project file and conveniently swap it with another one. In ESP, you can also simply use an html file uploaded to SPIFFS, in which case the page can be split into separate files. Attached is a game made for the WWW according to the Youtube channel's HTML course "Passion for Computer Science", html, css, js, graphics, core ESP 2.5.0. with SPIFFS you upload the ESP code separately and the contents of the flash drive separately.
    .


    Thanks, great, I'll have a look, but from the method using PROGMEM, or keeping it in the MCU itself at all (to put it in a nutshell, as the file system is also in the same flash, after all), it doesn't quite work for me, as I modify the web page instantly, check how it looks and only when everything is ok do I upload it to SPIFFS. On top of that, in javascript it generates, for example, tables with 20 rows in a loop, while in fact there are only a few lines and a "for" loop in the file. Sure, you could also do it in MCU, but testing would be tedious, making changes too.
    Through PROGMEM I do, for example, replies to the client that something was successful or not. Simple and repeatable things.
    I implemented the idea with variables in javascript and it works great. The whole page can be huge and look nice (using Bootstrap, for example) and when the HTML is loaded, the javascript kicks in and substitutes variables, on the fly, for values. Clear, fast, simple. I'll stick with that, but thanks for the feedback, I'll of course look at your files out of curiosity and try to fish something out for myself ;) .
    Now I'm struggling with writing structures to EEPROM, because while put and get worked for me so far on single variables, when I write whole structures, after reading I don't have what I've written...but I'm still "researching" it, because there might be a problem again with overlapping addresses and such.
    And anyway, once something works, I start to find pleasure in optimising the code, and suddenly it turns out that with all these changes, memory is not so scarce after all ;) .

    Regards,
    Mariusz
  • #16 19868988
    kaczakat
    Level 34  
    Posts: 1748
    Help: 317
    Rate: 229
    For testing, the web page can also be on a PC in an HTML file, just change the line in it to "websock = new WebSocket('ws://192.168.1.30:81/');" and it displays the data talking to this ESP with ip 30.
    Helpful post? Buy me a coffee.
  • #17 19869001
    jaskol
    Level 12  
    Posts: 112
    Rate: 3
    kaczakat wrote:
    So for testing this web page can be and on a PC in an HTML file, just change the line in it to "websock = new WebSocket('ws://192.168.1.30:81/');" and it displays the data talking to this ESP with an ip of 30.
    .

    I'll be testing, as it's certainly ideal for sending real-time sensor data.
    As for the rest, I'll test that too, because so far, in this version of mine, everything works, but I make no secret of the fact that it's slow.
    Maybe with websockets it would be faster, and that could already be tempting....
    I'm reading, watching, checking. I will definitely use something from this ;) .

    Thanks to,
    Mariusz
    • Reply | New topic
    Report a violation of the law

    Topic summary

    ✨ The discussion revolves around handling string overflow and reboots on the ESP8266 Wemos D1 when processing large files (20kB) using SPIFFS or LittleFS. The user experiences reboots due to memory issues when manipulating large strings. Suggestions include using a buffer for data storage instead of loading the entire file into memory, utilizing static arrays, and dynamically allocating memory. Alternatives such as splitting the file into smaller parts or using JavaScript for variable substitution on the client side are also proposed. The user considers switching to the ESP32 for better memory management and performance. The conversation highlights the importance of efficient memory usage and the potential of WebSockets for real-time data handling.
    Generated by the language model.
    Today's popular

    FAQ

    TL;DR: Use ~2 kB chunks; "You don't have to load the whole file at once." Stream/template HTML to avoid ESP8266 String overflows and reboots. [Elektroda, mpier, post #19849190]

    Why it matters: This FAQ helps ESP8266/ESP32 makers fix crashes and slowdowns when handling large HTML/JS files.

    Quick Facts

    How do I stop ESP8266 reboots when reading a big HTML file?

    Avoid readString() with large pages. Read with readBytes() into a fixed-size static buffer and stream to client or file. Substitute tokens as you go instead of building one huge String. “Better to use readBytes() into a static array.” This reduces heap churn, prevents fragmentation, and stops random resets. Keep buffers global or static to avoid frequent allocations. [Elektroda, khoam, post #19849244]

    How much can a String hold on ESP8266?

    There is no fixed limit. It depends on free heap and fragmentation. If you must use String, call reserve(size) up front to grab contiguous space and avoid repeated growth. If reserve() fails, your heap cannot satisfy the request safely. Consider chunked processing or a static char[] buffer to reduce risk. [Elektroda, mpier, post #19848067]

    Why do I see “Reallocating large String(191 -> 192 bytes)” and then a reboot?

    That log means the String requested more heap. Many small reallocations fragment memory and can fail under pressure, causing resets. Large files amplify this, because replacements trigger repeated growth. Replace dynamic Strings with a pre-sized buffer and stream. Reduce intermediate copies and avoid building whole pages in RAM. [Elektroda, jaskol, post #19847872]

    Can I process the HTML without loading the whole file?

    Yes. Stream the file in ~2 kB chunks and replace template tokens like %NAME% on the fly. Write each processed chunk directly to the client or output file. This approach bounds memory use and avoids heap fragmentation. It also scales to bigger pages and multiple forms. [Elektroda, mpier, post #19849190]

    What if a placeholder spans a chunk boundary?

    Carry over any partial token between reads. Stop reading when you see a token start marker (e.g., ‘%’), process or buffer it, then resume. Alternatively, read up to CRLF if your template is line-oriented. This avoids splitting tokens and simplifies replacement logic. [Elektroda, janek_wro, post #19850480]

    Should I pre-allocate a buffer and keep it for reuse?

    Yes. Allocate once and reuse to avoid churn. “Read all the data into a buffer previously reserved, but don’t release it after use.” This stabilizes heap usage during repeated page generation and reduces fragmentation over time. Use a static/global char[] for predictable behavior. [Elektroda, mpier, post #19848021]

    Is loading a 20 kB file into RAM realistic on ESP8266?

    It can fail. One report showed only ~20,992 bytes free heap at start, leaving little headroom for Strings and stack. Load in chunks or switch to ESP32 if you truly need large in-memory buffers. Chunking remains the safest pattern on both platforms. [Elektroda, jaskol, post #19848620]

    How do I free memory after using a String array?

    Setting elements to "" does not return heap to a large contiguous state. Many String objects fragment the heap. Replace String arrays with a single static char[] buffer, or a small ring buffer of fixed-size blocks. Avoid hundreds of dynamic String allocations. [Elektroda, khoam, post #19849584]

    Is there a better architecture for the device UI?

    Yes. Keep the HTML+JS page static and store device values in a separate JS file. Load it and populate fields in the browser. This avoids heavy server-side concatenation and keeps updates small and fast. [Elektroda, janek_wro, post #19849593]

    How can I test the UI fast without reflashing files every change?

    Develop the page on your PC and point a WebSocket to the ESP (e.g., ws://device-ip:81). The browser loads local HTML/JS while live data comes from the microcontroller. This speeds iteration and reduces flash wear. [Elektroda, kaczakat, post #19868988]

    Will switching to ESP32 help, and what trade-offs might I see?

    ESP32 adds RAM, easing buffer pressure, but your firmware size may grow after porting. One case observed 14% RAM used but 72% program memory on ESP32 for the same app. Chunking and templating still matter on ESP32. [Elektroda, jaskol, post #19849315]

    How do I implement streamed substitution? (3-step)

    1. readBytes() into a static char buffer, leaving room for a null terminator.
    2. Scan for tokens in the buffer, replace, and write the output immediately.
    3. Preserve any partial token at the end, then read the next chunk and repeat. [Elektroda, khoam, post #19849244]

    Is line-by-line processing a good idea here?

    Not if it becomes slow or still overflows Strings. One report found multi-line reading “pointless” and slow, with String overflow risks remaining. Prefer chunked binary reads with minimal copies, then write out immediately. [Elektroda, jaskol, post #19848058]

    What pitfalls commonly cause random resets in this scenario?

    Large Arrays of String objects fragment the heap. For example, 500 String entries allocate many different heap regions, raising allocation failure risk. Repeated growth and temporary copies worsen it. Use fixed-size char buffers and stream data. [Elektroda, khoam, post #19849584]

    Can I use PROGMEM for pages or server messages?

    Yes. Store static HTML in PROGMEM and send it as-is. Then push dynamic values via WebSockets or a small JSON endpoint. This reduces filesystem reads and RAM use during page assembly. [Elektroda, kaczakat, post #19868860]

    Should I load libraries like jQuery from a CDN?

    It can be faster than serving from SPIFFS/LittleFS due to browser caching and network speed. One user saw better performance from CDN than local filesystem. Provide a fallback for offline scenarios if reliability matters. [Elektroda, jaskol, post #19851191]
    Generated by the language model.
    ADVERTISEMENT