logo elektroda
logo elektroda
X
logo elektroda
Advanced Search
logo elektroda

How to Implement a Clock in OpenBeken Devices Without Using NTP Protocol

max4elektroda 2985 45
ADVERTISEMENT
Reply | New topic
  • #1 21031093
    max4elektroda
    Level 20  
    My first contact to OpenBeken was when I got some unexpected Smart plugs with LN882H.
    To make full use of them, the BL0937 had to be included and this forced to enable other drivers, especially NTP.

    Sure, yo can't keep track on energy stats without a time, but the idea was born to add a clock without NTP.

    There are some cons, especially that NTP will keep the clock accurate, which I learned was not the case with my first approach.
    But, there are some pros:
    No need for a network: You can set and use time even in the AP mode
    No need for NTP protocol
    The first approach was very simple and needed only very limited resources.
    And finally: you usually don't need exact clock: If the sum of energy values for one day is inaccurate for some seconds, who cares?

    O.k., lets finally get to the basic idea:
    If I only add one simple variable containing the "time at startup", I can calculate the actual time by adding the value of g_secondsElapsed.

    This worked as expected - for some short time - but in my case it was drifting ~ 2 minutes per day :-(.
    So my first improvement was: making g_secondsElapsed more accurate (requiring two further variables):
    Every ten seconds I calculate the number of RTOS ticks elapsed and update g_secondsElapsed to this value.

    Now time is o.k. for me - it differs about 1 to 2 seconds a day, I can live white this, and maybe a correction can make it even better.

    First approach was to implement all this in user_main and new_common - files "known" to all platforms, so I could compile it for all platforms in that stage.

    Now to the more complex work: To make full use of the device time, I would like to use the time related functions, which are all "bound to NTP", but not really related to it (e.g. making NTP_GetMonth() a simple GetMonth() or use events without NTP).
    My next goal is to split the NTP driver into the actual NTP ("protocol") part and some generic time functions.

    To make this work, I will had to introduce "new source files", which need altered Makfiles to work.

    And I found some possible "issues" with NTP, which could be addressed at the same time:
    If I stop the NTP driver, this will stop the incrementing the time variable in "NTP_OnEverySecond()" - but it will continue to report "synced", even if the time "stops".
    As I learned that the OnEverySecond-loops are not very reliable, I would propose to change NTP to the mechanism I used:
    When NTP is synced, just save the corresponding startup time - if you add the (optimized) g_secondsElapsed, the time will not drift so much - and even can be used as a base for continuing the clock this way.


    So I would like some input on the general idea, maybe it can be implemented in a better way.
    And I would really like some input, if this doesn't break anything but works as expected on other platforms.

    Actual screenshot of "index" page:
    Screenshot of the OpenLN882H user interface.

    Changes to LN882Hs Makefile are:

    Code: text Expand Select all Copy to clipboard
    
--- sdk/OpenLN882H/project/OpenBeken/CMakeLists.txt  2024-03-26 16:04:01.812615610 +0100
+++ sdk/OpenLN882H/project/OpenBeken/CMakeLists.txt     2024-04-02 15:35:22.872182469 +0200
@@ -56,8 +56,10 @@
 #    app/src/driver/drv_max72xx_internal.c
 #    app/src/driver/drv_max72xx_single.c
 #    app/src/driver/drv_mcp9808.c
-#    app/src/driver/drv_ntp_events.c
-    app/src/driver/drv_ntp.c
+#    app/src/driver/drv_ntp_events.c           # replaced by drv_clock_events.c
+    app/src/driver/drv_clock_events.c
+    app/src/driver/drv_ntp.c                   # general time functions moved to drv_deviceclock.c
+    app/src/driver/drv_deviceclock.c
 #    app/src/driver/drv_pt6523.c
     app/src/driver/drv_pwm_groups.c
     app/src/driver/drv_pwmToggler.c


    The actual progress are here: PR#1152


    For OpenBK7231T/N and OpenBL602 the compilation seems o.k., for the others you will need altered Makefiles, too:

    Code: text Expand Select all Copy to clipboard
    --- sdk/OpenW800/app/Makefile	2024-03-26 16:04:03.812580727 +0100
+++ sdk/OpenW800/app/Makefile	2024-04-02 21:24:35.486435795 +0200
@@ -30,6 +30,7 @@
 CSRCS += $(_SHARED_APP)/hal/w800/hal_pins_w800.c
 CSRCS += $(_SHARED_APP)/hal/w800/hal_wifi_w800.c
 CSRCS += $(_SHARED_APP)/httpserver/hass.c
+CSRCS += $(_SHARED_APP)/driver/drv_deviceclock.c
 CSRCS += $(_SHARED_APP)/httpserver/http_fns.c
 CSRCS += $(_SHARED_APP)/httpserver/http_tcp_server.c
 CSRCS += $(_SHARED_APP)/httpserver/http_basic_auth.c

--- sdk/OpenW600/app/Makefile	2024-03-26 16:04:02.376605773 +0100
+++ sdk/OpenW600/app/Makefile	2024-04-02 21:01:54.011157424 +0200
@@ -47,6 +47,7 @@
 CSRCS += $(_SHARED_APP)/driver/drv_httpButtons.c
 CSRCS += $(_SHARED_APP)/driver/drv_main.c
 CSRCS += $(_SHARED_APP)/driver/drv_ntp.c
+CSRCS += $(_SHARED_APP)/driver/drv_deviceclock.c
 CSRCS += $(_SHARED_APP)/driver/drv_tasmotaDeviceGroups.c
 CSRCS += $(_SHARED_APP)/driver/drv_test_drivers.c
 CSRCS += $(_SHARED_APP)/driver/drv_bridge_driver.c

--- sdk/OpenXR809/project/oxr_sharedApp/gcc/Makefile	2024-03-26 16:04:04.420570122 +0100
+++ sdk/OpenXR809/project/oxr_sharedApp/gcc/Makefile	2024-04-02 20:52:18.203372715 +0200
@@ -67,6 +67,7 @@
 SRCS += ../shared/src/driver/drv_ntp
 SRCS += ../shared/src/driver/drv_tuyaMCU
 SRCS += ../shared/src/driver/drv_uart
+SRCS += ../shared/src/driver/drv_deviceclock

 SRCS += ../shared/src/i2c/drv_i2c_main
 SRCS += ../shared/src/i2c/drv_i2c_mcp23017
  • ADVERTISEMENT
  • #2 21031682
    p.kaczmarek2
    Moderator Smart Home  
    Nice work, but that label is funny. Please move it to Config and rename to, idk, something that will be easily to comprehend for average users, maybe "Offline time system".

    Regarding SDKs, I can accept your PRs in each repo, but the problem is that later I may need to pull them to the main tree.... so it would be good to know whether current version is ready to be merged?

    Or maybe even better... please add the #define to obk_config.h I requested, the one that enables or disables your feature, and submit it to your PR in disabled state. Then I will be able to merge your current PR, fix SDKs, but it won't affect devices. Later you will be able to continue development in another PR.

    Just my few requirements:
    - rename the functions from GetHour to Clock_GetHour, etc
    - make sure that it's possible to enable/disable system in obk_config.h and if disabled it does not goes into build
    - make sure to avoid breaking existing systems/compatilibity
    Helpful post? Buy me a coffee.
  • ADVERTISEMENT
  • #3 21032076
    max4elektroda
    Level 20  
    Thanks for your feedback, I will try to implement the suggested changes.

    I was not sure about the "#define" statements, but I think I just took the wrong example to look at ;-)
    If I take the obvious "NTP" as an example, I see a reference to this #define only once in src/driver/drv_main.c (plus one #ifndef in src/httpserver/json_interface.c for some "fake calls").

    So, if I'm not mistaken, the unset #define ENABLE_NTP will only prevent starting the driver, it will not prevent it being compiled into the image and so on. Especially it won't have any impact e.g. on the image size.

    After your repeated advice, I took a look at another example ( #define ENABLE_LITTLE_FS ) and now I know, what you mean: This driver's code is inside an #if statement!



    While writing all this stuff down, I realized I should probably better split this into several parts:

    1. Improving "g_secondsElapsed" as a basic for the clock. This would be beneficial for the whole system and I would propose it as a general improvement (so no "#defines" here)
    2. Some changes to the GUI especially to Javascript code. This should reduce the image size (so no "#defines" here, too)
    3. Implementing my basic idea of a clock - this could be inside some "#define" blocks, but it's only some bytes
    4. Implementing the "complete clock", including timezone offset, daylight saving time ...
    5. Splitting actual NTP code to an "NTP driver" part and a "Clock driver" for all functions to get access to the clock; and renaming "drv_ntp_events" to a general "drv_clock_events".
    maybe it's an idea to put the stuff from #4 to the "Clock driver" so also an NTP driven clock can use this for timezones, DST ..

    This part clock driver will use some space, so it should be inside some #ifdef section.
    I must take care, that implementing some #ifdef here will change the behavior to the actual NTP code, which is present in the image (if chosen in Makefile), regardless of the state of #define ENABLE_NTP.

    Any suggestions or objections to this plan?
  • #4 21032115
    p.kaczmarek2
    Moderator Smart Home  
    Actually in many cases you don't need to put whole code into the #if statement. It's recommended, but not required. The linker and compiler works in such a way that if you have only few entry points to your system and you put those entry points into the #if statement, and they are not called, then the final code will not make into the build as well.
    Let us consider this example:
    Code snippet using C preprocessor with #if ENABLE_NTP directive
    If the ENABLE_NTP is 0, and functions NTP_Init, NTP_OnEverySecond and NTP_AppendInformationToHTTPIndexPage are not called anywhere else, then they will not be included into the build, even if they are themselves NOT included in the #ifdef block.
    That being said, if some other NTP-related code is called somewhere in the build, it still will be included in the flash.

    I would strongly suggest for you to make a basic working PR that is self-consistent and let me merge it (after basic testing) and then you will be able to work on futher features in a separate PR.

    For the manual timekeeping (or how is it called), I think you should have a separate #define from ENABLE_NTP, maybe:
    Code: text Expand Select all Copy to clipboard
    
#define ENABLE_LOCAL_CLOCK

    or something like that
    Helpful post? Buy me a coffee.
  • #5 21032907
    max4elektroda
    Level 20  
    I just made PR#1167.
    Please note that as requested the function is disabled, so to see and check it, you would need to change the #defines in
    obk_config.h and compile it again.

    I used ENABLE_LOCAL_CLOCK for basic usage and ENABLE_LOCAL_CLOCK_ADVANCED to also use daylight saving time calculation (requiring ENABLE_LOCAL_CLOCK to be set ;-)).
  • ADVERTISEMENT
  • #6 21034044
    p.kaczmarek2
    Moderator Smart Home  
    So, to confirm, it's ready to merge as it is, right?

    Added after 1 [minutes]:

    Is this fragment always executed, or only if local time keeping is enabled?
    JavaScript code snippet with a function for updating and setting the time.
    Helpful post? Buy me a coffee.
  • #7 21034483
    omniron
    Level 10  

    Best solution to all NTP related issues would be to use an RTC.
    Here I added the DS3231 to a KMC 4AC Power Monitoring unit (30608).
    How to Implement a Clock in OpenBeken Devices Without Using NTP Protocol How to Implement a Clock in OpenBeken Devices Without Using NTP Protocol How to Implement a Clock in OpenBeken Devices Without Using NTP Protocol How to Implement a Clock in OpenBeken Devices Without Using NTP Protocol How to Implement a Clock in OpenBeken Devices Without Using NTP Protocol

    With such an RTC the accurate (a minute in a year) time can be queried without opening the units to the internet or having internet running at all.
  • #8 21035091
    max4elektroda
    Level 20  
    Sorry, I'm quite busy this weekend and will be on a business trip next week, so just a few words:

    I think I didn't make it clear enough: I would really appreciate some feedback before merging it.

    Will it work on other platforms as expected?
    How accurate is the time for some days?

    What would others say on how should daylight saving time be implemented?
    With a table, like in the actual PR? This means, it must be adjusted to the timezone on compile time - or changed to variables, that could be added during the run.
    If we changed to a calculation we would be much more flexible but this will "cost" quite some code/space in the image.

    p.kaczmarek2 wrote:
    Is this fragment always executed, or only if local time keeping is enabled?

    Yes, that's one of the drawbacks I pointed out: Javascript is not inside an "#if" statement.
    If this is a blocking point: No problem, I will move the JS code inside the HTML part, which can be controlled with an #if, but I think it will be next weekend ...

    omniron wrote:

    Best solution to all NTP related issues would be to use an RTC.

    That might be a possibility, if there is enough space in your device and you really know much (or don't care so much ;-)) about the regulations for such devices.
    (Yes, you might argue, that probably most of the devices from aliexpress etc have a "C E" confirmation I would not trust too much...)

    And it depends on your needs:
    Keeping track on energy usage per day, the quality of the used clock is not so much of importance as if you will need it for some switching actions. And: It comes for free (only some bytes of code, it can even save space in an image, if you don't need NTP an only use this approach).
  • #9 21036097
    p.kaczmarek2
    Moderator Smart Home  
    I see, I will try to run it on one of my devices soon, depending on how much time I have. Maybe someone else can also test? @divadiow @DeDaMrAz ?

    max4elektroda wrote:

    p.kaczmarek2 wrote:
    Is this fragment always executed, or only if local time keeping is enabled?

    Yes, that's one of the drawbacks I pointed out: Javascript is not inside an "#if" statement.
    If this is a blocking point: No problem, I will move the JS code inside the HTML part, which can be controlled with an #if, but I think it will be next weekend ...

    It is possible to merge C strings at compile time with C preprocessor so you should be able to separate it anyway


    max4elektroda wrote:

    What would others say on how should daylight saving time be implemented?
    With a table, like in the actual PR? This means, it must be adjusted to the timezone on compile time - or changed to variables, that could be added during the run.
    If we changed to a calculation we would be much more flexible but this will "cost" quite some code/space in the image.

    This is good for a start but maybe we could consider using LittleFS for this purpose. Just assume that there is a file called "DST.bin" and read it as float array (or peek when needed), etc. Then we could upload DST bins for each zone separately...
    Btw how it's done in Tasmota?
    Helpful post? Buy me a coffee.
  • #10 21036106
    divadiow
    Level 34  
    I will flash PR this evening when home onto LN, BL602, BKN and BKT then take them offline for some time. Is this the desired test?

    Added after 13 [minutes]:

    And briefly, off the top of your heads, can OBK be queried programmatically for its current time?
  • #11 21036229
    max4elektroda
    Level 20  
    divadiow wrote:
    I will flash PR this evening when home onto LN, BL602, BKN and BKT then take them offline for some time. Is this the desired test?

    Yes (and no...): The PR is with "disabled" feature in obk_config.h, so the provided images won't show too much.

    divadiow wrote:
    And briefly, off the top of your heads, can OBK be queried programmatically for its current time?

    If I get your question right: That's the basic idea here ;-).
    If I'm not mistaken, actually OBK has "secondsElapsed" as an uptime counter, which I try to "tune" a bit with RTOS ticks, to make it more reliable.
    So its "current time" is its uptime, but if I combine it with the knowledge of the time of device start, the sum will be the "actual time".

    Then we can use "Clock_GetCurrentTime()" (derived from NTP_GetCurrentTime()) to get the time.
    Or did I misunderstand your question??
  • #12 21036296
    divadiow
    Level 34  
    Eek. I think I was trying to be clever with checking the device current time against a known good source at regular intervals and mapping the drift over time. But maybe I just report on the uptime of all devices at the same time after a set period of time (ie at the end)? I don't need to complicate things.
  • #13 21036320
    max4elektroda
    Level 20  
    divadiow wrote:
    Eek. I think I was trying to be clever with checking the device current time against a known good source at regular intervals and mapping the drift over time.

    Ah, yes, its possible to get the local time, e.g.with "STATUS 8". I also used this on a router (with NTP source) to get regulary "reports" on devicetime vs. NTP time like this:
    Code: text Expand Select all Copy to clipboard
    
echo "$(wget -q "http://192.168.0.58/cm?cmnd=STATUS%208" -O -) --  $(date "+%Y-%m-%dT%H:%M:%S")"
{"StatusSNS":{"Time":"2024-04-07T15:05:09"}} --  2024-04-07T15:05:10
  • #14 21036961
    divadiow
    Level 34  
    max4elektroda wrote:
    Yes (and no...): The PR is with "disabled" feature in obk_config.h, so the provided images won't show too much.


    forgive me, how can I know if the new clock is running? ie how can I test?

    startdriver ntp as normal, make sure the device can't reach the ntp server then check local times after xx period of time?

    eli5
  • #15 21037122
    max4elektroda
    Level 20  

    The "clock" as implemented here, is visible on the GUI and also in the answer to a STATUS request.
    But as the defines in obk_config.h are set to 0, the clock would miss initializing.
    (There is no entry "set clock to browser's time" on the bottom of the config menu page and the link on the status page is misleading because the function to actually set the clock is missing).
    If it's ok, I could make the git system build images with this feature enabled.
    Just give me a hint if it would be better to commit a simple change for this to the existing PR or it should be better a "Testing PR", just to provide the images, that can be deleted if no longer needed.
    BTW, in a later version your suggestion would work, if accepted: the time, once got by NTP, could be handled like a manual clock setting and the clock would run, even if NTP is stopped.
  • ADVERTISEMENT
  • #16 21038282
    omniron
    Level 10  

    Would it be possible to use one unit with your amazing addition as NTP server for others on the network?
    Or does each unit have to be programmed as such and synced individually.
  • #17 21038303
    max4elektroda
    Level 20  
    Without any further work (as far as I know there is no NTP server implemented) the answer is: kind of.
    You don't need to set the clock in the GUI, this can be done from remote.
    So any device that can reach an OBK device by IP and has a good working clock can use an HTTP request to it to set the clock. It's not necessarily an OBK device.


    Added after 8 [hours] 58 [minutes]:

    Made a change to the PR so that the features are enabled in obk_config.h
    So it should be easy to use the generated images for some testing:

    Link to artifacts
  • Helpful post
    #18 21038935
    divadiow
    Level 34  
    yay.

    will set some devices up with no internet access, no ntp driver or other drivers running and powersave 0.

    it begins

    Screenshot of a device settings interface with a button labeled Set device clock to browser time.
  • #19 21038948
    max4elektroda
    Level 20  
    Great!
    Thanks, I'm really curious how it's working on other platforms
  • #20 21039030
    divadiow
    Level 34  
    about as close as I can get uptime and clocks as a starting position.

    Screenshot showing technical details of four devices, including online time, software version, device time, and MAC addresses.

    Added after 3 [minutes]:

    starting again. the uptime is weird on refresh, as you can see.

    Added after 6 [minutes]:

    OK. uptime before setting time to browser.

    Screenshot showing technical information about four devices, including their MAC addresses, online time, and software version.

    ah interesting, all 4 were set within 1-2s of eachother. N and T chips online/uptime seem to be put back 1 minute at the time this happens, as below. actual time seems good as a starting point though

    Four sections of device information showing parameters like firmware version, device time, and MAC address.

    Added after 4 [minutes]:

    is the device time seen in the browser the *actual* time reported on the device, in real-time, or does the browser carry on with it's own ticking over mechanism from when the page is loaded/refreshed?

    Added after 2 [minutes]:

    hmm. OK refreshed all 4 within a 1-2s of each other (first refresh since time was first set). something's wrong with N and T

    Screenshot of four different devices showing uptime, device time, and software version information.
  • #21 21039109
    max4elektroda
    Level 20  

    Oh, N and T really look strange.

    The displayed time in GUI is made the same way as it was done with uptime before: The device submits an "initial value" and this value is (with JavaScript) incremented (approximately) every second and this calculated uptime/device clock is shown in the GUI.
  • #22 21039114
    divadiow
    Level 34  
    max4elektroda wrote:
    The device submits an "initial value"
    yes, makes sense. Tx

    Added after 13 [minutes]:

    to illustrate further. second refresh of all.

    Screenshot of an interface showing information about four online devices with different names and MAC addresses.

    LN and BL seem good.

    Added after 26 [minutes]:

    I don't have W800, W600 or XR809 yet I'm afraid
  • #23 21039456
    omniron
    Level 10  

    >>21038303

    HTTP request sounds great!
    I think openbeken firmware can do that?

    Also, from my end a few minutes discrepancy wouldn't matter for things like turning lights on/off.

    I tried that "Link to artifacts" but it went nowhere.

    Could you please, when all finished, make a how-to for us less advanced?

    Just a sample on how to add that feature to existing devices and also a sample how to access that remotely from another device

    that would be really great!!
    Thanks
  • #24 21039505
    max4elektroda
    Level 20  
    I will look into my code to calculate the uptime again to see why it seems to get only about half the time elapsed on that T/N platforms.

    @omniron:
    Sorry, it seems the link to the artifacts is only working if you have a git account and are logged in.
    Which platform do you have?


    Added after 9 [hours] 7 [minutes]:

    I think I found the culprit: I missed the correction factor like
    Code: text Expand Select all Copy to clipboard
    uint32_t ms_to_tick_ratio = 2;

    as defined in
    Code: text Expand Select all Copy to clipboard
    sdk/platforms/bk7231n/bk7231n_os/beken378/os/FreeRTOSv9.0.0/rtos_pub.c
  • #25 21039700
    divadiow
    Level 34  
    LN and BL holding steady still this morning

    Screenshot displaying network device information, online status, and time details.
  • #26 21039949
    max4elektroda
    Level 20  

    Thanks for the input.
    I tried to fix the Beken devices in the PR.
    So if you have some time (and after the last problems are still brave enough ;-)) I would be pleased to hear if the issue with the slow clock is fixed for the N/T devices
  • #27 21039956
    divadiow
    Level 34  
    of course. flashing now

    Added after 40 [minutes]:

    starting positions

    Screenshot with information about several network devices, including names, MAC addresses, online time, and chipsets.

    browser time set

    Screenshot showing information about networked devices, including build time, online status, and MAC addresses.

    13 minutes later - page refresh

    Screenshot showing details of four network devices with information on build time, online duration, device time, MAC address, and short name.

    Added after 1 [hours] 18 [minutes]:

    more time

    Screenshot of a web application control panel showing device details and a system clock.
  • #29 21046552
    divadiow
    Level 34  
    I have a W600 now. Sadly, it's not happy very quickly

    Screenshot displaying device and time information.
  • #30 21046557
    p.kaczmarek2
    Moderator Smart Home  
    Are you testing on W600 with PowerSave or without it?
    Helpful post? Buy me a coffee.
Reply | New topic
Report a violation of the law

Topic summary

The discussion revolves around implementing a local clock in OpenBeken devices without relying on the NTP protocol. The initial approach involved using a variable to track the time since startup, allowing for time calculations based on elapsed seconds. While this method lacks the accuracy provided by NTP, it offers advantages such as independence from network connectivity and reduced resource requirements. Participants provided feedback on code structure, suggested improvements for compatibility, and discussed the potential for using real-time clocks (RTC) for enhanced accuracy. Testing across various platforms, including LN882H, BL602, and W600, was conducted to evaluate the clock's performance and reliability. The conversation also touched on daylight saving time adjustments and the possibility of remote clock synchronization.
Summary generated by the language model.
Popular Topics
ADVERTISEMENT