ESP32 Serial Communication Issue with Arduino Mega and PC Application

Hello
Trying to make a control for a project:
https://www.elektroda.pl/rtvforum/topic3613898.html
In general, yes:
I have an ESP32 which is the main unit
There is an Arduino mega which sends data every 2ms via Serial2
There is also a PC Application which communicates with the ESP32 via Serial
The problem is that when I press a button on the application, the application sends a command via serial to the ESP32, but sometimes the reading is not correct

When I didn't use Serial2, everything was ok. When I use Serial2, I find that sometimes the information from the application is cut off. I thought, that there is some independent queuing and storing information from each serial, but here it turns out, that when information from Serial and Serial1 comes simultaneously, the one which comes first is read correctly, and the second one is not.
How can this problem be solved?

I am uploading the ESP32 code below, if that would help


Added after 4 [minutes]:

I thought about using a second processor, but it doesn't work

.
In the video above you can see what happens when I send text. In the code below:
.
you can see that what comes from the application is supposed to be sent back to it in the same form, but unfortunately this is not the case. Perhaps what seems odd is the use of '@' as the end of the message. It turns out that the application doesn't seem to send '@' or '@', so when I tried to specify the end of the line with either of those characters, I didn't get any string back.

Added after 3 [minutes]: .

When I disconnect the data cable from the ArduinoMega, there are no problems, every message from the application comes in the same form as it was sent, but then I don't have the current positions from the encoders

Added after 3 [minutes]:

I don't have converters, I'm aware that the ESP32 has different voltages to the Mega, but it's ok with that. The ESP32 downloads data correctly to RX2 with 5V all the time, and does so seamlessly. The app, on the other hand, transmits data over USB.

Unfortunately this didn't do anything, I changed the data transfer from Mega every 10ms. In the video below I wanted to show by moving the robot's axes that the positions are sent correctly, although you may never actually know if something won't come wrong
It's better, but there's no guarantee that a message from the app won't arrive at the same time as a message from Mega. Will definitely order such a converter on Monday
https://www.youtube.com/watch?v=4IcpJ1bDmdM&feature=youtu.be
I don't know what's up with the video transfer, I had to put on Youtube

Added after 1 [hour] 3 [minutes]:

And yet , the transfer from Mega to ESP32 is not ideal. By entering 0.commute.200.150.@ in the serial monitor, I force axis 0 to commute to position 200 at 150. I tested once:
0.commute.200.150.@
and back
0.approach.-200.150.@
i.e. left at 200 and right at -200
I noticed that every now and then it stops before the destination point, which it shouldn't, because stopping is getting to the set position. i set up a trap so that the position value is displayed at the time of arrival and i got something different than i should, so it's wrong!
Could I ask for the name of these converters? I will definitely be ordering something like that on Monday. The reason for adding the Mega as an encoder position reader was that the ESP32 was misreading the pulses from the encoders, but fed either 0V or 5V to the inputs. Maybe if I followed the 3.3V logic the readings would be correct. I will check this tomorrow. Well nothing thanks for diagnosing the problem

Very interesting topic, I tried even before in a similar way to flip the reading from Mega to Core 0, but I didn't succeed. I added something like this according to the recommendations from the link
.
The downside is that in these shuffles you have to add delay(>=1), because if this is to be executed continuously it has to be in the fro() petla, is that right?
In addition, I have introduced a condition that if the read position from megi, deviates by 10 positions from the previous transfer, this read is to be ignored. When it comes to readings from megi there is a big improvement, this priority I set to 2.
The reading from the app is also important, see if you can the video below
https://youtu.be/p85hkcGzs8Q
By pressing a particular button, I simply send the information by serial. If at the same time the Mega is transmitting data, the data from the app is not received in its entirety
At time 1:20, you can see as I press the button the application sends 3 lines:
serialPort1.WriteLine("0.access.200.100.@");
serialPort1.WriteLine("1.commute.200.100.@");
serialPort1.WriteLine("2.commute.-200.90.@"); .
As you can see the first line of code doesn't read correctly because axis[0] didn't move, it was only when I clicked the button again that the 3 axes were in the 200 positions.
How would it be possible to make it so that the data from the application is read by the second processor, i.e. Core0?

Aha, that's how it works! What I have posted is just reading from the Mega via serial2. The app communicates via USB, so it must be Serial. I'll get around to adding the other thread soon and let you know

Super , I let the robot go into a loop and after more than 40 smooth trajectories I stopped. Thanks a lot for your help. Really strong is this ESP32

Do not connect 5v and 3.3V devices - in any case not directly. The uP inputs are protected by diodes. If you connect a 5v signal to the 3.3V circuit, you will of course force a high state, but a considerable current will flow through the protection diode when 4v is exceeded. The fact that something won't burn up is only due to the fact that the current is limited by the transmitting circuit (here the Mega). Therefore, if you don't want converters, you can:
- transmit from a 3.3V to 5V circuit - connect directly (mostly works, as the 3.3V will already be recognised as a high state). I write mostly, because the catalogue notes usually indicate a slightly higher voltage for "1" in 5V circuits
- Transmitting from the 5V circuit to 3.3V - use a divider (this is the best solution) or connect via a resistor of approx. 2.2k. Using a resistor uses a protection diode, but simply limits the current flowing through it.
I think the 'poorest' version (1 resistor) will not drain the budget, and certainly the ESP will not be damaged.

I've recently stopped with the topic, due to damage to the heater from the 3D printer (I'm waiting for a new shipment from China), but I wanted to check the heating of the axis 2 motors with increased current on the axis motors.
https://youtu.be/jlWJpc1fdXc
After the second hour, the third axis started to box, it didn't stop at the target point just drove over it and stopped at its end position (mechanically it couldn't go any further and the stepper motors started to box)
I saw that a piece of code could be the cause of this:
.
I get a reading from the Mega every 5ms, if it reads wrong a couple of times, that is, the reading is out of the +15 or -15 position range from the previous reading then the motor will actually travel those 15 positions and even if it sends the correct position it still won't be overwritten in esp32.

I went back to rigidly assigning what I read
.
I did this for each of the three axes and strange things started to happen, as if the robot didn't know what to do.
The code for sending data from the Mega looks like this:
.

And the current reception on the ESP32 is implemented by Core0 and looks like this:
.
I've set up a trap that if the new axis position is out of the +-15 range from the previous reading, so that it displays to me what the ESP32 has received from the Mega, and this is where the problems come out:
When the stepper motors are not engaged and I manually move the axes there is no problem at all
.
When the trajectory is called by typing "drive.@" the robot starts to arrive at its target positions, when it arrives it sends "no_axis.arrived". In the above picture I arrived at the target points by manually moving the axes and zero problem. I then attached the stepper motors and attached the trajectories:
.

It doesn't matter if I run the TX signal from the Mega through a logic state converter, directly or with a 2k2 resistor. There is always a similar number of errors, sometimes a little more, sometimes a little less. Perhaps surprisingly there are a few more errors when using the converter,
It is already clear that this is caused by interference, if nothing bad happens without steppers.


This is what the data looks like on the oscilloscope this with a 2k2 resistor on the transmission line (which is now)


and yes after adding a 47pF capacitor the TX line of the Megi and GND. After zooming in, which I don't have in the picture, there is a really nice signal, but still wrong readings are coming from Megi.
.
Do you have any ideas how such connections could still be 'sub-filtered' to avoid erroneous data transfer?
As for the encoder position readings themselves via the Mega, it is 100% fine here. As you can see in one of the pictures above.

I think it's definitely the fault of the interference itself, but as much as I'd like to write the code correctly, especially as I'm not a good programmer. Regarding the interference itself, I'll try to get a piece of shielded cable from work tomorrow and see if there is any difference.
Regarding the data exchange code itself in processor 0, would that be a better solution?
.
I uploaded something like this still with a safety feature that if there is a bad reading 3 times in a row then stop the robot and the same thing after about 35 minutes of repeating the trajectory the robot stopped. When I trigger the trajectory execution again the robot drives from where it stopped, it has perfectly reached the target position, hence my assumption that the encoder reading is 100% correct.

Added after 9 [minutes]: .

Also looking at the oscilloscope, I can see a data transfer on it every 5ms or so also seems fine.

Well I will try to test it today after work and will let you know. As for the baud rate it is set to 115200.

Yet it was interference, a twisted pair without a screen did not help. I unbolted the profibus plug from the training PLC. The profibus cable has two wires and a shield, which I connected to ground on one side and the other, and for 30 minutes zero erroneous messages. You can immediately see that acceleration and braking are smoother. In my spare time I will still test how the system behaves at higher transmission speeds. Actually, it's interesting, because I remember that when I reduced the speed to 9600 (checking on the oscilloscope, of course, that there are sufficient gaps between data packets) there was much more interference.
Oh well, many thanks again for your help, I am now waiting for a new heater for the 3D printer I will try to add the other axes