[Solved] ESP8266 (Arduino): Optimisation of server polling frequency in lighting control

Let me start by describing the problem. I have a lot of ESP at home. The whole control model is based on connectivity to a server, specifically PHP scripts. The WEB application connects to the server and sends or receives information (e.g. lights a lamp, reads the temperature, etc.). On the other hand, I have ESPs that respond accordingly. If they just send data (e.g. temperature) then there is no problem – they do it at a certain interval. Worse when they are actors – then they ask the server if there is something to do e.g. whether to switch on a relay. In the case of lighting control, however, I don't want to wait long for a response, hence the polling needs to be frequent e.g. every 0.5s. This is not a problem for the server, but nevertheless generates unnecessary traffic and even so these 0.5s are noticeable. Of course, the solution would be to set the ESP in the position of the server, but here other problems arise, e.g. time control – for example, if someone turns on an outdoor light and forgets to turn it off, then with regular polling, there is no problem for the PHP server to conclude that the sun has risen and there is no need to shine.
So I came up with a solution that combines the advantages of both solutions (client and server) for actors. It involves cyclically querying at a longer interval (e.g. 20s), while at the same time using the web server function in ESP exclusively to signal – „stop waiting and query, because I have something for you”. In theory, this approach doesn't change the philosophy and doesn't require a lot of software redesign, yet it should be fast and effective.
My idea is very simple (below is the skeleton of the idea):
.
And now the questions – is such an immediate server shutdown (without downloading) correct? Will everything allow itself and will there be no memory leak? Can I use the „client” variable immediately for uploading?

I mean:
.
That is, I check if something is there and don't even read it, but immediately client.stop()

As for the location of the WiFiClient (inside the loop) you are absolutely right. I think I used a not very good example once.

Hello,
in the past if I remember correctly, http and www were used to serve pages to the browser. If you chose php because you have a good understanding of the issues, then you could use websocket for two-way communication. If not necessarily, then the easiest thing would be to write a separate server to communicate with clients (I mean service, because whether the "server" is a server or a client is irrelevant), here it's all the same whether in php, c or other. You solve both problems in one go.

I am not explicitly allocating anything. The point is that when server.available() is called, some memory will already be allocated because some data has already been received (unless I'm thinking wrong). I don't read that data, I just call stop(). Second point - from your description I conclude that it's better to use two objects of the WiFiClient class - one for receiving and one for sending then I won't have to worry about the time for closing.

Maybe I have described the idea wrong. In a loop which is called quite often (10ms) at first I check if there was a call to the ESP server (server.available(); ) and then instead of reading from it and being interested I want to immediately ignore everything that came in and close the call. I don't want to read anything from it. It's just supposed to be this kind of 'trigger'. Then there is my actual connection (which I have been using for years) - this time ESP is the client, not the server. I started this thread to consider this problem - if the ESP server "reports" server.available(), and I brute-force it immediately with client.stop(); would this be correct.

OK, but it can't return the wrong client to me because one has any port from Apache and port 80 on ESP server, and the other has any port on ESP client and 80 port on Apache.

khoam wrote:

So only one external client connects to the local server in the ESP? I don't quite understand this statement. Maybe some simple diagram?

.
Exactly as you wrote.
Currently it is as follows:
1. I turn on the light in the browser (app).
2. this information flies to the main server (to PHP).
3. PHP writes the information "you need to turn on the light for 7 minutes" into a file.
4. completely asynchronously ESP connects to the same server and asks "do you need to turn on the light?".
5. the server replies "yes light on" or "no light on" (depending on the time)
6. the ESP performs the specified action.
In order for the whole thing to be responsive, the ESP needs to ask itself frequently, e.g. every 0.5s, and this is what I wanted to change.

After the change it should be like this:
1. in the browser (app) I turn on the light.
2. this information goes to the main server (to PHP).
3. PHP writes the information "you need to turn on the light for 7 minutes" into a file and additionally sends a signal to the actual ESP "stay awake and get on with it"
4. completely asynchronously the ESP connects to the same server and asks "do you need to turn on the light?".
5. the server replies "yes light up" or "no light up" (depending on the time)
6. the ESP performs the specified action.
Now the ESP can connect every 20s (e.g. to find out that it's already daytime and it's pointless to light up), because an additional signal (trigger) has forced the immediate triggering of point 4. As you can see, this additional trigger can, in my conception, only be fired from the main server. It also does not carry any data, it is simply to force ESP to immediately execute 4.

I know the whole concept can be remodelled, but it works for me. I just want to optimise it. I am currently adding an OTA to my solutions. When ESP starts, there is a version check and a possible firmware swap (the version is also checked by PHP). On this occasion, I will be collecting the IP addresses of the modules so that PHP at point 3 knows which ESP to urge.

@khoam, thank you for your guidance. In the end, I decided on 2 WiFiClient objects (declared outside the loop of course) - one for the server and one for the client, and left the client.stop without parameters. The server client will have a lot of time because when there was a request to the ESP server, there is an ESP client action (and that action is running on another object), which is after all ongoing. Then the start of the loop has a 10ms delay. After the initial perplexities of not working (I just forgot about server.begin(); ) the whole thing works wonderfully - I built a test circuit with an LED - virtually instant responsiveness.

It was time to move the test solution to the existing solution. Overall it works well - instant responsiveness as it was during testing. Unfortunately there is a noticeable delay every so often (once every few dozen actions). I estimate it to be about 1s. I have analysed the PHP code and it is not the cause. The ESP remains. It is definitely not related to my "trigger" getting lost, because the delays would be random and much longer (up to 20s). The only place is:
.
There is a 1 second delay here. I changed the program and instead of delay(1000) I gave delay(100). Since then I have not seen a delay once. Of course, the problem itself probably still occurs, but the 100ms is not noticeable, and the next call already happens. I'm writing about this because I was treating the line "if (!client.connect(host, httpPort))" as a really fallback solution. And here it appears that a few percent of calls may be affected. Is this normal?

I think I have found the reason. Well, when the WiFiClient is inside the function, as I read, it is not necessary to call client.stop() (it is called when releasing the object). When the WiFiClient object is brought outside the loop function client.stop() is mandatory, and I didn't have it.
For now, I went back to delay(1000) to see if the delay would show up again. However, it works flawlessly.

I was based on this thread: Link Several people commented that without the "stop" something didn't work for them. I can only confirm that "stop" has also improved my situation.

The communication problem has finally been solved. After moving WiFiClient outside of the function (global object) the setting client.setTimeout(10); applies not only to subsequent calls (client.readString() but also to other operations, in particular client.connect(host, 80); Here these 10ms are often not enough and therefore did not always work. Now before client.connect(host, 80) I added client.setTimeout(300); and it is OK. In general, I have to say that the concept of periodically checking the state every 20-30 seconds with an additional wakeup by calling the server on the ESP works well - instant responsiveness, and if anything there is a status query every 20 seconds anyway.
Thank you @khoam Topic closed