Efficient hosting of a simple HTML page on a microcontroller - GZip compression in HTTP

.
OpenBeken has long had integration of the LittleFS file system allowing files to be stored and managed directly in the microcontroller's Flash memory. This allows you to conveniently create device-specific configuration panels, such a panel itself is implemented in HTML and its communication with the firmware is done through a REST interface, which I have already presented in the topic OpenBeken as a mini HTTP hosting - writing pages in Javascript, Tasmota's REST API etc . Here, in turn, I would like to show some very simple but efficient optimisation that I have recently implemented for this system.

One of the goals of the exercise is to efficiently host a simple Web application written in VUE - all directly on the device. Let's see what we are dealing with - 11 files, sizes up to 38kB.
.
That's 167kB in total, a lot for a flash memory file system. On the BK7231 we already have trouble with this, as it interferes with the section from the OTA and for larger builds the update removes the LFS. However, there is a solution...

HTTP has a built-in and generally supported mechanism to reduce the size of transmitted data based on Gzip compression. It requires packing the transferred bytes with this compression and adding the appropriate parameter to the packet header:
.
Worse, however, is the fact that you now have to compress the sent data on the server side, doesn't it?
This is where the second optimisation comes in. Our microcontroller is only supposed to host the page, it doesn't need to edit it, so we can compress the files before putting them in LittleFS .
I've created a helpful button in the web app for this, meaning the compression is done in the browser rather than on the MCU:
.
Let's see the implementation of the compression:
.
There really is only enough - Javascript does the rest. The MCU itself doesn't even know that a gz file is being uploaded. Zero changes to the firmware.

However, you still need to change the sending of files from the MCU to the browser. Here you just need to detect the gz extension and add the information to the HTTP header:
.
It's time to see how much memory this will save us:
.
167kB versus 40kB is a very big difference - that's more than four times less! In this way, it is not only possible to save space in the flash memory, but also to speed up the transfer, because less data is sent faster. All this at practically zero cost.

You could end here, but I will take this opportunity to present some other side ideas and improvements I have concerning the LFS.
1. LFS profiler .
This simple program checks the timing of sending and receiving files from the LFS via HTTP GET and POST requests. The times and results of the operations are shown in the table. Below are the results for LFS on an external flash drive connected by software.
.
2. Supporting an external Flash bone .
At the moment I have realised this based on software SPI, as the hardware SPI is not derived on the WB3S/CB3S, only on the CBU...
Additional Flash memory for measurements for free? Communication protocol, write, read, erase .
3. LFS optimisation .
One of the simpler LFS optimisations is to increase the size of the working buffer. This allows faster read/write operations (more bytes are processed at once) at the cost of slightly higher RAM usage:
.
4. Execution of Berry scripts on the "server side" .
Another interesting possibility implemented in the firmware is to use Berry as a PHP equivalent. My system allows its scripts to be inserted into HTML code. The scripts are then executed at the GET/POST request handling stage, all on the source MCU. Below is an excerpt from the associated test:
.
Berry in the PHP role will be discussed in a separate topic.

In summary , that famous one simple trick for hosting larger pages and scripts on the MCU is to store an already compressed version of the file and serve it with the appropriate header. The MCU essentially doesn't have to do anything here, well, except for that one HTTP header field what it says about compression.
In further topics I will show what I have used such a system for and what I have managed to host.