Why do pages take 6-10 seconds to load and is SPIFSS to blame?

Hi.
I have been using ESP8266 (4MB) with webserver and FTP server. File system on SPIFFS.
Everything worked even ok, but I connected myself ESP32 (4MB) and I was so impressed by the ease of bluetooth operation,
that I modified the code and ported it to ESP32.
And I was very disappointed, because the pages load after 6-10 seconds, while before it took max. 3 seconds.
I don't switch to LittleFS because ftpserver doesn't work then. It connects and ..freezes. I found a thread about this, made the changes, but it didn't do anything.
Also I stayed with SPIFSS and seemingly everything works, but nightmarishly slow.

I'm now trying to figure out how to declare a web page in PROGMEM, but then I lose the ability to edit "on the fly" files "on disk" and view the changes in the browser.
I'm assuming that either the file system is so slow (maybe I have a module with such a poor memory bone - although I've tried on 2 different ones and it's the same everywhere), or the wifi itself is working so poorly.
The latter I can't think of, but that the libraries for SPIFFS have some bugs, I'll believe it.
Perhaps someone has had a similar problem ?

Thanks in advance for your help,
Mariusz

jaskol wrote:

I also stayed with SPIFSS and seemingly everything works, but nightmarishly slow.

.
Yes, SPIFFS is more than 5 times slower than FFAT or LittleFS on ESP32. Espressif announced some time ago that SPIFFS will be phased out in future versions of ESP-IDF.

jaskol wrote:

but the fact that the libraries for SPIFFS have some bugs is something I will already believe

.
The arduino library for SPIFFS is one big act of desperation It hasn't really been updated for a few months now.

jaskol wrote:

I don't switch to LittleFS because ftpserver doesn't work then

.
Are you familiar with this project? Link .

khoam wrote:

You know this project? Link

.
I think I've tried this "ftp" as well, but since you're hinting, this one probably works.
I'll check, it would probably solve a lot of problems .

As for the speed, I think I have the cause !
On the ESP8266 I was copying a file, making changes on the fly and saving.
Sometimes it saved me an empty file.
What I do now is that I read the source file into memory, make modifications to the content there and save under the target name.
It works ok, but it also happened to me that sometimes the file was 0kb after saving.
I put in a timeout, for rich, 3000ms, because it worked fine and probably after 500ms it was done. I put it in and forgot about it.
After switching to the ESP32, it was this timeout that got me bogged down !
When I reduced it to 100ms, it works a lot faster, but I'm already scared of what it might do. And it's still not crazy, the page takes a while to load....
If LittleFS is 5 x faster, then it is worth the money. Just hope the FTP works with it and I have no more wishes.
I'm getting to work, thanks for the suggestions .

Mariusz

P.S..
I've now read that this FTP server just works with SPIFFS, so that was already there.
Well it does work, I was immediately exposed to the limit when I started to upload a file of more than 65k.
I swapped to LittleFS and I have, what I wrote about - it is impossible to upload anything.

Filezilla reports:
Response: 150 Accepted data connection to port 50009
Response: 552 Probably insufficient storage space
Error: File transfer failed
Status: Connection closed by server

Also still miserable.
And what has pretty much killed me is that I added BLE support and I have a program memory overrun of 106%.
I'm looking for a flash version with not 4MB, but 16MB.
Also I'm treading water.

FTP only works with SPIFFS and has a file size limitation.
On top of that it works slowly.
I have not found an FTP server that works with LittleFS.
The 4MB version of the ESP32 has a memory occupancy of 69% when BLE is enabled, which means there is no point in buying it without 16MB memory.
This looks weak .

M.

jaskol wrote:

I swapped to LittleFS and got what I wrote about - nothing can be transferred.

.
And how did you create the partition for LittleFS and initialise the file system?

jaskol wrote:

And what has pretty much killed me is that I added BLE support and I'm running over program memory - 106%

.
The Arduino BLE library is quite extensive and virtually everyone encounters this problem when connecting WiFi to BLE.

jaskol wrote:

I'm looking for a version with flash not 4MB, but 16MB.

Look on soyter.co.uk. But you may just need to change the partition scheme and enlarge the space for the code. I explained it in this thread: Link .
Alternatively, you can create your own partition scheme: Link .

khoam wrote:

And how did you create a partition for LittleFS and initialise the file system?

.
I used the description on the WWW, here for example:
https://www.mischianti.org/2020/02/08/ftp-server-on-esp8266-and-esp32/
I copied mklittfefs,exe to the directory with esptool.exe and the other *.py.

It looks strange. I have a menu to do a file upload, I have already accepted that FTP will not work.
When I fire up the test program, it writes and reads the partitions, but when I fire up my target program, do the upload, it is silent.
"There is no such file" and that's it. I just don't see any partition definition file, I select different boards in Arduino, most recently Wemos, because that's what it says on it. But when I select a different one I have different definitions, and in the /...and here long path/partitions directory I have files and none of them have anything starting with littlefs in the name.
But when I upload through the menu in Arduino, it shows the files in the data directory beautifully, it says it's uploading, no errors.
I've been struggling with this all day, it's tragically done. Sure, you never look at a gift horse, but it will never cease to amaze me - that people do something so confusing, so seemingly described in one place, and yet not working. The examples can be multiplied.

jaskol wrote:

I'm looking for a version with flash not 4MB, but 16MB.

.

khoam wrote:

Search at soyter.co.uk. But you may just need to change the partition scheme and enlarge the space for the code.

khoam wrote:

.
I've already bought one, but without soldering there is no way around it. I couldn't find any version with a power supply chip or RS232 <> USB converter already on the PCB.
Someone was replacing the flash itself, I was thinking about that too, but it turns out that it involves digging around in the bootloader or some other files and all in all, it either works or not. I'll see if this version of mine identifies itself well, but I see something, that for blinking LED after Wifi, it's all suitable, but if you want to do something more, then problems begin .

khoam wrote:

I explained it in this thread:
Link .
Alternatively you can create your own partition scheme: Link

.

.
I was just starting to wonder how this works .
Thanks, I'll definitely give it a try, but I think I'll go back to the ESP8266 because at least everything worked there, and something tells me that with this ESP32 I'm wasting
time with some bugs, because if I get info that the files have uploaded but they're not there, then something is wrong.
I'll learn on the 8266 and get smarter and come back to the subject on the 32.

M.

.