How do I configure the ESP-01 to transfer data via WiFi to a computer?

Question

Hello everyone Please give me some advice on how to solve this: I need to measure only one physical quantity (e.g. temperature) and wirelessly display its time graph on the computer. So far I have implemented it like this: SENDER: sensor connected to Arduino and via nRF2401 data sent out into...

rb401 · Accepted Answer

. Oh, I know this pain. I installed Indy(10.2.0.3) some time ago and remember, there were just a lot of problems. I used some descriptions on the net because without them the chances of success are small. I remember that some files have to be moved and grouped differently in folders. I even have separate folders for this library, already working, but it's hard to tell what the problem was so quickly. I probably used this page (How to install [2]): Indy_with_Lazarus . because I see that I have a saved folder there.

rb401 · Accepted Answer

. I don't think you need to set this, unless the computer is on two networks at the same time (two network cards) then you have to somehow determine which one you are referring to by entering the IP that the computer has on the specifically selected network. . In the application you enter an arbitrary port number of your choice for this communication. And that port 80 in ESP is the web server port, which is of absolutely no interest to you here (you can of course use it in parallel for some purpose but it has nothing to do with UDP transmission). What is important (somewhat) is the IP address of the ESP. To send a UDP packet you only need to know the IP of the recipient and the port number on which it is listening. The UDP port number on the ESP from which the packet will be sent is not important. It will be assigned automatically. In your case, you could use the specific IP of the recipient's computer, but that's out of the question for obvious reasons, so you'll use the broadcast address, which for this network where your ESP is now is 192.168.1.255. You could also try something even more universal and give it a 255.255.255.255 address, but I'm not 100% sure that will work, although there's something on my network that sends UDP packets to that address and Lazarus picks it up. I have no idea what variant you're using ESP in, whether on its own or in AT mode. But as if you don't, sending a UDP packet is a simple matter (one to three instructions) where you have to enter the IP, the remote port and the content of the packet. Another tidbit that might come in handy. If you receive a UDP packet then in the IdUDPServer1UDPRead procedure, in addition to the content of the packet, you can find out specifics about the sender. ABinding.PeerIP as a string gives you the IP address and ABinding.PeerPort as an Integer is the port number of the sender (useless here but I give it here for the sake of order). So, if e.g. ESP is configured in client mode and gets its address from DHCP, and then you don't know what IP it got, receiving such a UDP packet you have it traced and you can e.g. visit its webpage or establish a parallel connection addressing it specifically (via UDP or TCP).

szymon122 · Answer

. One-way communication is sufficient but this has nothing to do with WiFi mode. Whether it should be server or client is up to you. The client will connect to your home WiFi network and send data through it, or it works as a server, creates its own WiFi, the computer connects to it and receives data. Your choice, that and it will work. But I recommend the former, you won't have to switch networks on your computer.

rb401 · Answer

. Then use pure UDP, without going into other protocols. Since you have a walking version on COM, very little changes to the program will suffice, as pretty much everything works the same. On ESP it's sending a UDP packet (all you need to know is the IP (but not necessarily) and the port of the recipient), and on the Lazarus side you just need, for example, an Indy packet (and a strict UDP component from the Indy Servers-Core tab). On Synapse it can also be done, but on Indy it's probably more downhill.

krzbor · Answer

The Indy package also has the ability to create a web server. ESP can be a client that periodically sends data. I don't know what you need such an application for, but it will only work when the program is running. It might be worth considering an Apache+PHP-based solution for receiving and storing data, and only create a display interface in Lazarus that runs on files saved by PHP - although this is form over substance, as it is then better to write everything in HTML and Javascript for the web browser.

rb401 · Answer

Well, exactly. And here it's about the mundane replacement of the serial cable in the simple cyclic transmission of a single number, not very often (temperature) to a specific application. Anything above the simple use of UDP (preferably broadcast) or even bare TCP(though it complicates things a bit more), any use of e.g. HTTP, standing up servers etc. seems to me to be shooting a cannon at a fly.

mgrems · Answer

Thanks for the tips.
I've been trying to install Indy for a few hours now. So far without success. But I'm of good mind...
I've outlined the problem, which may be trivial, but it's not about uploading the temperature once in a while, I gave that as an example. It's about sending data that changes quite dynamically, maximum, it's about 20 times per second. The data is changing, and I only want to record it when something interesting happens. I was also thinking about a transmission triggered from the Lazarus app. This is specifically about a physical experiment, and since cables get in the way a bit I want to eliminate them. This was already working on the aforementioned NRF2401, but the idea of sharing the recorded data with the 'wider public' came to me.
Here's the question again: the fact that the ESP can work in multi-connection support mode (CIPMUX=1) has nothing to do with two devices receiving the same data right? One data if it goes somewhere is gone (?).

I'm back to Indy, but waiting for more suggestions

mgrems · Answer

Success! I have installed Indy 10 I'm getting to work, I'll see what I produce

rb401 · Answer

. Congratulations. Well deserved. . If you are going to send data via ESP in UDP packets to the broadcast address of that network, ESP is completely uninterested in how many recipients there are, whether one or a thousand. ESP will send once, and the packet will reach all the devices on that local network. And it will only have an effect on those that are listening on that particular port (i.e. your application). The rest will ignore them. And there is no contingency handling, because there simply isn't any. And then there's the other advantage of a broadcast address, that you don't have to remember or choose any IP addresses either on one side or the other. The only condition is that the information transmitter and receivers are on the same network. You can also make the communication the other way round, analogously via UDP broadcast, but to a different port or to a specific IP of your microcontroller, as the application is able to determine this from the incoming data packets.

mgrems · Answer

Well I guess that's what I mean, but for now I have a problem with Lazarus:

.

I insert the Client component into the formatter and when trying to compile it throws a fatal error as shown in the image. I have tried inserting paths in the compiler options but to no avail, what else to try?

Added after 14 [minutes]: .

I have fixed it, the problem is now gone :-)

rb401 · Answer

. It works for me . I have the path to the lazarus folder in Indy under Other unit files(-Fu):. I'll look around some more where else there are references to Indy. Interestingly, in Package>Install/Uninstall.... I have indylaz 10.2.0.3 in the left list, and when I click on it, it says Current state: installed, DesignTime at the bottom and not RunAndDesignTime as with other packages. But it works. Although in your case it looks like DesignTime is there and RunTime is not. Maybe compile lazarus/indylaz.lpk again, but before pressing Compile, go to Options>Package Options>IDE Integration and set DesignTime and runtime. And note the hint as you mouse over the Designtime option, as that's sort of your situation. . It's ok.

mgrems · Answer

. I wrote two applications: CLIENT and SERVER (UDP), the client sends a message and the server receives it and it works. Both have BroadcastEnabled on True, ports on 12000, but on the server Binding I set it to 0.0.0.0:12000. what is this parameter? When I removed this it also works . So it doesn't matter what the address is, but it matters what the port is, is that right? And I don't understand now how this relates to my local network? How am I supposed to receive data from the ESP if its address is 192.168.1.109 and port 80? Is it enough if I set the port to 80 instead of 12000 in my application? kind regards - Maciej I have included the ability to change ports in both applications and it works, as the ports are the same, data is received

mgrems · Answer

Ufff.....
It took me a while, but I managed. Below is the result of my struggles so far:

.

In the end, the ESP-01 is a client with the address 192.168.1.107 (assigned) and port 169 (from me). The Lazarus application runs as a server with IP 192.168.1.103 (assigned).

The new idea of my project now is that:
1. the ESP will run itself in sleep mode,
2. the application (server) on the computer will send a "start" command to the client (ESP), wake it up, and it will start measuring and sending data,
3. the application will start ingesting this data and applying it to a time chart, as it collects the necessary data, it will send a 'stop' to the client and the client will go to sleep.
4. at the same time this data can be received on another computer with the same application.

I got a bit stuck on the ports, because when the application was running as a client it would receive some port (from the "cap" ?) it would communicate with the ESP port that corresponded to the server, and the server had to have that port assigned. When the application ran another time it received a different port ... and I don't know anymore...

Now I'm going to struggle with sending random numbers through the ESP and registering them on the computer.

rb401 · Answer

. This is quite normal behaviour. The client gets an unused port number from the system for the time it sends a UDP packet. As soon as it sends it, the port is closed and released. And for some reason, when sending again, the client gets a port that was not used last time (usually with a number one higher). If you are stubborn, you can reserve a port for sending. The UDP server component of Indy, also has a method that sends packets (procedure Send(AHost: string; const APort: Integer; const AData: string); ) and then indeed, all packets sent by this method go out with the port number as set to receive. But I don't think this adds anything practical. In your case, with this new concept you only need to listen on arbitrarily determined ports from both sides, and the port number from which the client is currently connecting becomes quite irrelevant.

TvWidget · Answer

For what you want to implement, I think websocket would be convenient. It allows you to send data online between the web page used for visualisation and the sensor.

rb401 · Answer

. But note that there is no WWW here but a specialised application processing data on the fly and this running in parallel in multiple copies. And secondly, the implementation of simultaneous handling of multiple TCP sessions with overheads on the websocket protocol would, in all probability, simply not work out in time on something like ESP. And the advantages that websocket brings (separate sessions, authentication, etc.) are not really needed here.

TvWidget · Answer

At the beginning, the author of the question mentioned visualisation on the website. Then he changed the concept but I don't know if definitively.
For many applications, of course UDP multicast or broadcast protocol will be good.

mgrems · Answer

TvWidget, the author (i.e. me) just wrote in the first post that I know how to upload it to the web, but I don't want that. But thanks for the discussion, everyone brings something new to solving the problem. Added after 4 [hours] 50 [minutes]: . . This is what I used.

mgrems · Answer

Welcome after a long break.
Thanks to your help I have reached a satisfactory stage. Now the cosmetics remain.
The ESP-01 connects to the wifi, and waits for contact from the application from a computer in the same network. After confirming the communication, the application sends a "start" signal and the ESP starts sending a randomly generated number from the range 0-999 every 1s.

.

The application receives the packets and transfers the values to a time chart.

.

Now I need to hook up a sensor to the ESP and start sending real measurements. Once I get to the end I will upload the end result.

Thanks and best regards - Maciej

rb401 · Answer

. This looks nice. Personally, I would rather test with consecutive numbers to see if any packet didn't arrive. This is, after all, UDP, by definition not guaranteed to arrive or to be error-free in content. Admittedly, some misrepresentation of packet content should be rejected at the wifi layer but a radio, like a radio, can have temporary problems. At least theoretically, as it is difficult for me to estimate what the reliability of the network looks like for example in one room and with ESP having a fairly strong transmitter. Alternatively, I would add some numbering to the packets in the target form for control, to at least have an indicator somewhere on the side that everything is ok, that nothing is lost.

dondu · Answer

A good solution is to ping the router very frequently and check the WiFi signal level.
Not only do we confirm connectivity with the router, but we also have information about the connection itself.
Only such information together with the indicator (mentioned by rb401) is worth sending to the application from time to time.

It is also worth adding confirmation by the application that it has received the sent information, which is of course the most correct solution.

rb401 · Answer

. Receiving acknowledgements and handling this can overload the ESP and complicate the project unnecessarily, and here, as far as I understand, it's essentially just about providing data for visualisation. So that just in case just detecting in the application some loss of individual packets would be an indicator of whether any further action is needed at all, or just not to bother. And sending an additional number, incremented each time, costs virtually nothing. Because the problem of losing packets can be incidental, e.g. some lightning in the vicinity, etc. . I don't want to prejudge anything here, but personally I would be calmer with a counter of lost samples somewhere on the side of the graph.

dondu · Answer

. On what basis do you make this assertion? . Your whole argument is a nightmare approach to device design. That's what we have our head and fingers for, to write a program in such a way to make our own protocol resistant to interference and packet loss using UDP. Proposing a lost sample counter cries to heaven. The program should make sure that the sample has been received by the application and if not then resend it until successful.

rb401 · Answer

. Previously on conjecture and now on practical tests. Assuming that the measurements go in UDP broadcast packets, the maximum practical packet sending rate is somewhere between 6-7 per second (for packets of 30 to 300 bytes, but the rate depends little on the size). Above this value packets get lost and the cause is not just the ESP but the principle of WiFi. The transmission of UDP packets with a specific recipient address, is much faster. For example, 800 packets/s with a data length of 30 bytes versus 430 if the packets had 300 bytes of data. But here in this solution, broadcast is very convenient for some reasons mentioned earlier. So that still adding the handling of acknowledgements and what is worse the handling of contingencies can cause the communication to be overwhelmed, if it is already close to the end of its capacity. If it was to be reliable as you postulate, it would be necessary to send specifically addressed packets and acknowledgements, but at the cost of a large extension, adding some mechanism for reporting the subscription of measurement data by each current application. But the question is whether there is a point and a rational need? It is probably better to know that it is good. I don't know why the idea of this meter upsets you so much. If there is a zero or a few on it during operation it is no tragedy. Especially since if the measurement samples are collected a little more often than the necessary minimum, the occasional lost samples can be interpolated, without harming the essence of the thing.

dondu · Answer

. And this is what I pointed out to you, because you are introducing a case into the discussion that does not occur here due to the measurement intervals. Let me explain again - your way is the worst possible way to design a device. Instead of counting errors and interpolating measurements you should: . It's just a matter of developing your own protocol to ensure that all the data arrives even if the lack of connectivity lasted for, say, an hour. ------------ Wrong conclusion . I merely highlighted that this solution is the wrong way to respond to problems.

rb401 · Answer

. You don't need to develop, because what you are proposing exists e.g. MQTT, where there are ready-made client components on the side of e.g. Lazarus. Except that I haven't come across an implementation of an MQTT broker on the ESP8266, even in the SN version. And I don't think there will ever be one. When sewing something of your own with such functionality, there is also a risk that the reliability of the whole will not be high anyway, due to various bandwidth or memory overheads. But on the basis of my experience with UDP broadcasts, I came up with another easy-to-implement idea, which would make the system completely immune to any occasional loss of UDP packets, although you probably won't like it much either, even though it may cause the counter you don't like to lose its sense of existence. As I noted, with UDP broadcast transmission there is a sharp limit on the number of packets sent over time and the packet length itself is less critical. It might be worth sending in every single UDP packet not only a packet of newly measured data, but also additionally e.g. 9 previous packets. Numbered packets, of course. Then the loss of single UDP packets or even a longer continuous sequence of them (up to 9 packets) will not cause any damage to the data, because each packet of measurements is always transmitted anyway, for example, 10 times. And all this for free, without the need for acknowledgements and emergency handling, etc. . Well, and without a lot of work. On the transmitting side, a circular buffer for measurement packets, which packets the entire UDP packet every time the measurements arrive. And on the receiving side, a counter for the number of the last processed packet and a simple search in the incoming packets for the measurement packet with the next number not yet processed. So that even the deterioration of connectivity to a situation where the majority of packets were lost would not in any way disturb the reliability of data delivery. At most, it would temporarily stop the visualisation.

mgrems · Answer

welcome after the break

As far as losing data is concerned, this is not the case. It is true that data is collected every 1 second, but on the other hand it is not strategic data. It is about measuring a certain physical quantity (with a higher resolution than 1 second). If something is lost along the way, the measurement can always be repeated.
For today, I plugged in a thermometer and while logging the data for about half an hour nothing got lost. Below is the result of recent struggles:

.

Now fine-tuning and operation.
Thanks to everyone for their constructive comments. I consider the topic to be completed. Greetings

How do I configure the ESP-01 to transfer data via WiFi to a computer?

Post #1

Post #2

Post #3

Post #4

Post #5

Post #6

Post #7

Post #8

Post #9

Post #10

Post #11

Post #12

Post #13

Post #14

Post #15

Post #16

Post #17

Post #18

Post #19

Post #20

Post #21

Post #22

Post #23

Post #24

Post #25

Post #26

Post #27

Post #28

Topic summary