logo elektroda
logo elektroda
X
logo elektroda

Wemos D1 "Arduino" and DHT11 - a simple weather station with graphs on the website

p.kaczmarek2 4671 17

TL;DR

  • Wemos D1/ESP8266 weather station reads a DHT11 sensor, fetches NTP time, stores measurements, and serves temperature and humidity graphs over a local WiFi web page.
  • The design uses ESP8266WebServer for HTTP, NTPClient for network time, chart.js for plotting, and a circular buffer to keep recent samples in RAM.
  • The history buffer holds 24 * 6 samples, enough for a full day at 10-minute intervals.
  • The graph is built from label, temperature, and humidity arrays, iterating from the oldest sample and skipping empty entries where time equals 0.
  • RAM limits and code organization remain open issues, and the author suggests Ticker, WiFiManager, and checking NTP before the first measurement.
Generated by the language model.
ADVERTISEMENT
Treść została przetłumaczona polish » english Zobacz oryginalną wersję tematu
📢 Listen (AI):
  • Temperature and humidity chart displayed on the ESP8266 Weather Station webpage
    My previous topic about creating charts on Arduino R4 received a greater response than I expected, so now, for a change, I am making a similar project but in a more complete form and on the ESP8266 platform in PlatformIO. Typically, the whole thing will be assembled from ready-made libraries, and I will describe the process in the form of a tutorial. First, I will run the DHT11 temperature and humidity sensor here, then I will show you how you can download the current time from the Internet. Then I will demonstrate how to collect measurements in memory in a circular buffer, and finally display them on a simple web page with a graph, all available within our WiFi.

    First steps
    Some time ago I published a topic about ArduinoOTA and ESP8266 in PlatformIO , I recommend you read it. Here I am assuming that we have OTA running and I will use what I developed earlier. As a reminder - here is my "hello world":
    Code: C / C++
    Log in, to see the code

    In this topic, we try to assemble everything from ready-made elements, so we will also use a ready-made library for the DHT11 sensor. We just need to install it.
    Open the Libraries tab:
    PlatformIO interface with the libraries tab open in Visual Studio Code
    We look for DHT. The DHT library also requires the Adafruit Unified Sensor Library to be added, so we add both libraries. First Unified Sensor:
    Screenshot showing a search for DHT libraries in PlatformIO.
    We add to the project:
    PlatformIO interface with selected Adafruit Unified Sensor library
    We will need to indicate to which project we are adding this library:
    Project dependency addition dialog in PlatformIO
    Added:
    Adafruit Unified Sensor library installation in PlatformIO
    This is how we add both libraries. A keen eye will notice that they have also been added to our platformio.ini:
    PlatformIO library configuration with added Adafruit libraries
    It is also worth noting that there is also an example of its use on the library`s website:
    Screenshot of an example using the DHT_Unified_Sensor library in PlatformIO.
    Let`s integrate it into our code. I perform the measurement when the diode status is switched:
    Code: C / C++
    Log in, to see the code

    At the moment, I only send results via UART. Then we will make a website. Let`s first check if DHT works at all:
    Terminal display showing WiFi connection data and temperature and humidity readings.

    We add a website
    Putting our measurements out into the world is really very simple. A class is used to create a server ESP8266WebServer . First, we include its header:
    Code: C / C++
    Log in, to see the code

    Then we create its instance, the argument is the port on which the server will be created (HTTP is usually port 80):
    Code: C / C++
    Log in, to see the code

    Then we create functions that handle the page data:
    Code: C / C++
    Log in, to see the code

    In the main loop we serve clients:
    Code: C / C++
    Log in, to see the code

    And finally, the function that handles the main page:
    Code: C / C++
    Log in, to see the code

    Result:
    Webpage displaying the text Hello Elektroda! in a browser.
    It`s really very simple. Now we will display the temperature.


    We display the temperature
    To display the temperature on the page we need to do two things:
    - these measurements need to be saved, for simplicity we will put them in global variables
    - they must then be placed in the text of the page, I will use my favorite method to format the string, i.e. a C-style buffer and the sprintf function.
    Global variables themselves:
    Code: C / C++
    Log in, to see the code

    Writing to global variables:
    Code: C / C++
    Log in, to see the code

    Modified page creation function:
    Code: C / C++
    Log in, to see the code

    In this way, the temperature and humidity read from DHT will appear on our screen.

    Network time (NTP)
    NTP stands for Network Time Protocol, a way to obtain time from the network. We also have a ready-made one here, we will use the class NTPClient . We add the appropriate library as before:
    Screenshot displaying the library search interface for NTPClient in PlatformIO.
    PlatformIO interface with NTPClient library open in the code editor.
    Of course, this library also has a ready-made example of use:
    Code: C / C++
    Log in, to see the code

    We still need to somehow obtain time in the form of an inscription. There is a ready-made function for this - getFormattedTime . After integration with our website we get:
    Code: C / C++
    Log in, to see the code

    Result:
    Screenshot displaying weather data on a webpage
    If we are ambitious, we can format the time ourselves as we wish. We have a function for this strftime , which works similarly to spritnf . We also format the time using special tags. Below is a slightly developed example:
    Code: C / C++
    Log in, to see the code



    Better organization
    However, our goal is to collect measurements. Before we implement it, I suggest that you better organize your measurement records. Let`s introduce a structure to represent the measurement. But what should such a structure contain? We will include:
    - temperature (float)
    - humidity (also float)
    - time (here it is best to use the time_t type, which is in fact also a number, I do not recommend writing time as a string, it would be inefficient and inconvenient in further processing)
    Code: C / C++
    Log in, to see the code

    You can then create an instance of this structure globally:
    Code: C / C++
    Log in, to see the code

    and our program will work as before, the only important thing is to save the time at the moment of measurement and not create it at the moment of reading the page.

    Measurement history - circular buffer - part 1
    Now you need to figure out how to store the measurements. You`d like to use a regular table, but what about deleting old measurements? RAM is not infinite.
    There is a simple way to do this - we will use it here circular buffer .
    A circular buffer is an array-based data structure that allows you to store a certain number of data elements, and when the buffer is full, new data begins to overwrite the oldest. As the name suggests - the index of the element to which we write comes full circle here. Appropriate manipulation of the array index (along with the appropriate method of reading and writing) makes it become a circular buffer.
    Let`s first decide on the size of the board - if we want to have a day of measurements, where measurements are taken every 10 minutes, then 24 * 6 will be enough.
    Code: C / C++
    Log in, to see the code

    We also need the index of the last measurement (this index will "circle" around the array, it will loop):
    Code: C / C++
    Log in, to see the code

    A function that adds a measurement (and loops through the index using the modulo operator):
    Code: C / C++
    Log in, to see the code

    And the table clearing function:
    Code: C / C++
    Log in, to see the code

    Of course, this also needs to be included in downloading measurements from DHT:
    Code: C / C++
    Log in, to see the code

    and their display (no chart at the moment):
    Code: C / C++
    Log in, to see the code



    Finally a chart
    Now it`s time to create the chart. We will generate it similarly to the related topic: Drawing charts in HTML on Arduino R4 - statistics, measurements on a mini website .
    I decided to use a library for this chart.js
    On the W3schools website you can find examples of its use:
    https://www.w3schools.com/js/js_graphics_chartjs.asp
    Quoting the example from the website above - it all comes down to plugging our collected values into arrays:
    Sample source code for a line chart in Chart.js
    The above code will generate a chart like this:
    Line chart with three lines in different colors.
    Arrays are characterized by the fact that all elements except the first one are preceded by a comma, which we also have to handle, probably using some if.
    We create three arrays:
    - labels (markings, dates/times here)
    - temperature values
    - humidity values
    Sample solution:
    Code: C / C++
    Log in, to see the code

    The above solution ignores the circular buffer for now!
    Additionally, I break the loop early to skip "empty" measurements.
    Then we need to place the tables generated in this way in the rest of the code generating the chart, which we can take from examples of using this library:
    Code: C / C++
    Log in, to see the code

    Here is the result, for the effect I heated the sensor with a powerful flashlight:
    Graph showing temperature and humidity changes.
    Webpage displaying a temperature and humidity chart.

    Fixes and improvements
    The code in the previous paragraph is not correct. We must respect the circular buffer and ignore empty measurements. The measurement is empty if the time is equal to 0, and we start the buffer iteration from the oldest entry, which we identify as:
    
    (lastSample+1)%SAMPLES_COUNT
    

    the % operator is modulo, the remainder when divided by an integer. This way we start with the oldest entry and end with lastSample.
    Code: C / C++
    Log in, to see the code

    You can still show the result here, but in terms of appearance nothing will change (well, unless we wait longer, then without the correction to the circular buffer the graph could break):
    Temperature and humidity chart on a webpage


    Measurement frequency, cleaning
    Now it is worth separating these 1000 ms of the temperature sampling period into a separate variable. Let`s say:
    Code: C / C++
    Log in, to see the code

    In fact, this entire piece of code should be rewritten, maybe it could be possible to use, for example, a class Ticker , but maybe in the next part.

    View full code:
    Code: C / C++
    Log in, to see the code

    A screenshot of collecting samples over an extended period of time in a slightly modified version of the program:
    Temperature and humidity graph over time



    Summary
    A very simple and pleasant design. Just right to start with ESP. After selecting the appropriate number of samples (we are limited by RAM memory) and sampling frequency, you can have nice graphs showing the temperature and humidity for the last few days.
    Of course, this is just a small example and there is much room for improvement, e.g.:
    - aesthetics (dividing the chart into two units, etc.)
    - code (mentioned Ticker would come in handy)
    - configurability (you could connect WiFiManager to avoid the need to hard-code our SSID and password)
    - it would also be useful to check before the first measurement whether the time from NTP has already been downloaded, if not, skip the measurement
    Perhaps I will deal with this in the next part, but for now I invite you to comment. Have you implemented this type or a similar project at ESP? Or maybe someone will be tempted to develop the seed I provided here and turn it into a full-fledged system?

    Cool? Ranking DIY
    Helpful post? Buy me a coffee.
    About Author
    p.kaczmarek2
    Moderator Smart Home
    Offline 
    p.kaczmarek2 wrote 14408 posts with rating 12345, helped 650 times. Been with us since 2014 year.
  • ADVERTISEMENT
  • #2 21040993
    bialy
    Level 16  
    Posts: 227
    Help: 10
    Rate: 54
    DHT11 used to be an indicator rather than a temperature sensor (it showed whether it was rising or falling ;D).
    How does it work better now?
  • #3 21041002
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    Basically I can only agree. There are also slightly better sensors, for example BME280:
    BME280 sensor module on a purple circuit board with visible solder points and components.
    A beginner can treat this as an exercise and convert the code to a different sensor, basically it will only come down to changing the library, connecting the element, and adding one variable to the measurements (if we want to take pressure into account)
    Helpful post? Buy me a coffee.
  • ADVERTISEMENT
  • #4 21041078
    SylwekK
    Level 32  
    Posts: 2764
    Help: 82
    Rate: 2762
    p.kaczmarek2 wrote:
    There are also slightly better sensors, for example BME280:


    and the SHT3x series measures temp&humi and is praised :)
  • #5 21041318
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    I recognize SHT3X, I sometimes see it in Tuya products:
    https://www.elektroda.com/rtvforum/find.php?q=SHT3X+Tuya

    Interestingly, there are temperature sensors with an "alarm" function, which allows you to put the WiFi module into sleep and wake it up with an alarm:
    MCP9808 I2C temperature sensor module with alarm - startup, communication protocol
    Helpful post? Buy me a coffee.
  • #6 21041790
    Nargo
    Level 23  
    Posts: 495
    Help: 45
    Rate: 202
    SylwekK wrote:
    p.kaczmarek2 wrote:
    There are also slightly better sensors, for example BME280:


    and the SHT3x series measures temp&humi and is praised :)


    My budget replacement for BMP280 is BME280+AHT20 in one module.
    The disadvantages that I noticed after almost 6 months on 4 systems are the difference of one 1`C-1.1`C between AHT and BME and the need for changes in the libraries when compiling Tasmota.
    The advantages include two temperature sensors in one module :) After DHT22(AM2301) started buzzing in the external temperature sensor in the garage.
  • #7 21041948
    mariomario
    Level 18  
    Posts: 707
    Help: 17
    Rate: 139
    p.kaczmarek2 wrote:
    (..)
    "Let`s first decide on the size of the array - if we want to have a day of measurements, where measurements are taken every 10 minutes, then 24 * 6 will be enough."
    (..)


    I have `quick question` , because I was very interested in this project on ESP8266 -> Were there any tests carried out, how many measurements will enter the RAM memory of this ESP8266, let`s say the last 30 full days with the resolution of readings taken every 15 minutes for temperature and humidity readings? I know that there are ready-made solutions, such as the popular Xiaomi thermometers for several PLN, but I prefer something with a web interface, as in the case of this project on ESP..
  • #8 21041976
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    @mariomario, you have information on the Internet how much RAM is on ESP8266, but the question is, are you sure you want to use RAM? So that it disappears after a power loss? Maybe it`s better to use some external chip, for example EEPROM or Flash, you can even desolder these from electronic waste... or, as a last resort, write to flash from ESP8266, but in a clever way, saving erase cycles, see my OpenBeken project for the flash vars code which optimizes flash usage:
    https://github.com/openshwprojects/OpenBK7231...ob/main/src/hal/bk7231/hal_flashVars_bk7231.c

    Additionally, if we were to optimize for memory, we would need to improve the data structure. I write the tutorial simply and without complications, but if we were to reduce the amount of memory:
    Code: C / C++
    Log in, to see the code

    you could lose a lot of weight. Now this is 12 bytes (16 if time_t is 64 bit). What resolution and temperature range do we really want?
    What humidity resolution?
    I think that instead of two floats (8 bytes), I would pack it in 3 bytes... probably not in 2, but 9 bits for humidity and the rest for temperature...
    time_t (you would need to check whether it is 32-bit or 64-bit) could also be reduced, for example by counting seconds or tens of seconds from January 1, 2024. Or even minutes, because is accuracy down to the second necessary?

    If you want, I can work on something similar in the next part, or we can think about it together.
    Helpful post? Buy me a coffee.
  • ADVERTISEMENT
  • #9 21041977
    bialy
    Level 16  
    Posts: 227
    Help: 10
    Rate: 54
    In the option as above, approximately 46kb of RAM will be used for 30 days. You can optimize the capacity because 16 bytes per measurement is a lot of data.
  • ADVERTISEMENT
  • #10 21042000
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    Every 15 minutes a reading gives us 4 readings per hour, or 96 per day.
    30 days is 30*96 = 2880 readings
    Indeed, if this time_t is 64-bit, with a measurement structure of size 16 bytes we get 46kB, as @bialy wrote

    However, I think that it could be reduced by at least half, i.e. 8 bytes per measurement, then it would be 23kB.
    Helpful post? Buy me a coffee.
  • #11 21042013
    mariomario
    Level 18  
    Posts: 707
    Help: 17
    Rate: 139
    p.kaczmarek2 wrote:
    you have information on the Internet how much RAM is on ESP8266, the only question is whether you really want to use RAM? So that it disappears after a power loss? Maybe it`s better to use some external chip, for example EEPROM or Flash, you can even desolder these from electronic waste... or, as a last resort, write to flash from ESP8266, but in a clever way, saving erase(..) cycles

    I don`t know about other forum members, but in my opinion a project like this would be nice, which after entering the web interface shows the current (current) temperature and humidity reading along with the current time (read as above from the NTP server), and e.g. on the dashboard, at the bottom there is a graph with all the collected temperature and humidity data in a circular table (including those e.g. 30 days ago with a resolution of every 15 minutes).
    Data recording, if it fits in RAM, would be sufficient because the ESP would have to be powered from the power supply anyway (because it consumes a lot of power). :) ), but maybe it is possible to somehow add functionality ("save to Flash" checkbox) that would save data from the web-interface to Flash every 24 hours, for example, one large record from the entire day from RAM -> to the built-in Flash (on the ESP8266 board), then there would be 1 record per day and possibly the data from the last 24 hours would be lost.



    p.kaczmarek2 wrote:
    (..)What resolution and temperature range do we really want?
    What humidity resolution?
    (..)I think that instead of two floats (8 bytes), I would pack it in 3 bytes... probably not in 2, but 9 bits for humidity and the rest for temperature...
    time_t (you would need to check whether it is 32-bit or 64-bit) could also be reduced, for example by counting seconds or tens of seconds from 01/01/2024. Or even minutes, because is accuracy down to the second necessary?

    Temperature with one decimal place, as well as humidity, because these sensors may have good resolution, but the (real) accuracy is a bit of a lottery (I used to play with different sensors and each one shows something different...). The time in the DD-MM-YYYY HH:MM format would also be sufficient for most people.



    p.kaczmarek2 wrote:
    If you want, I can work on something similar in the next part, or we can think about it together.

    In my free time, I would love to join such a project
  • #12 21042317
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    mariomario wrote:

    I don`t know about other forum members, but in my opinion a project like the one that after entering the web interface would be cool

    This means that you would also need to connect WiFiManager so that you do not have to enter network data.

    mariomario wrote:

    on the same dashboard, at the bottom there is a graph of all collected temperature and humidity data in a circular table (including those e.g. 30 days ago with a resolution of every 15 minutes).

    I can`t estimate it in my memory, but you should consider whether you want to send all 2880 readings to the browser at once, or maybe make a less accurate "last 30 days" chart and the option to enter more detailed charts for a given day.


    mariomario wrote:
    it would save data to Flash every 24 hours, for example, one large record from the entire day from RAM -> to the built-in Flash (on the ESP8266 board), then there would be 1 record per day and possibly the data from the last 24 hours would be lost.

    Are you sure you need to save flash so much?
    For example, W25Q80 withstands a minimum of 100,000 deletion cycles (4KB sector):
    Fragment of Winbond memory specification regarding endurance and data retention.
    Assuming that there is a reading every 15 minutes a day, i.e. 96 readings, and that you exhaust one entire sector, you will exceed the limit after 100000/96 = 1041 days
    However, if you use the solution I linked earlier , which saves erase cycles, then assuming that the measurement structure is 16 bytes, you improve the situation by 4096/16 = 256 times.
    So after 266496 days you will reach the minimum number of erase cycles... after 729 years. This assumes that you tire out one sector, but you can separate them, e.g. 16... then 729 increases even 16 times.


    mariomario wrote:

    Temperature with one decimal place, as well as humidity, because these sensors may have good resolution, but the (real) accuracy is a bit of a lottery (I used to play with different sensors and each one shows something different...).

    You still need to specify the temperature range because -20°C to 40°C is different than -40°C to 100°C. Although I know that I`m exaggerating a bit with this packing, considering today`s flash capacities it`s a bit of an art for art`s sake.
    Helpful post? Buy me a coffee.
  • #13 21042544
    mariomario
    Level 18  
    Posts: 707
    Help: 17
    Rate: 139
    p.kaczmarek2 wrote:
    mariomario wrote:

    I don`t know about other forum members, but in my opinion a project like the one that after entering the web interface would be cool

    This means that you would also need to connect WiFiManager so that you do not have to enter network data.

    This would definitely improve the experience of using such a thermometer :) (the only question is how much memory will be left for the reading data?)


    p.kaczmarek2 wrote:
    mariomario wrote:

    on the same dashboard, at the bottom there is a graph of all collected temperature and humidity data in a circular table (including those e.g. 30 days ago with a resolution of every 15 minutes).

    I can`t estimate it in my memory, but you should consider whether you want to send all 2880 readings to the browser at once, or maybe make a less accurate "last 30 days" chart and the option to enter more detailed charts for a given day.

    Even without the optimizations discussed above, we have up to 45KiB of data to transfer from ESP to the browser. These are probably not large amounts of data. Such a chart can be generated on a smartphone or older PC without any problems.


    p.kaczmarek2 wrote:
    mariomario wrote:
    it would save data to Flash every 24 hours, for example, one large record from the entire day from RAM -> to the built-in Flash (on the ESP8266 board), then there would be 1 record per day and possibly the data from the last 24 hours would be lost.

    Are you sure you need to save flash so much?(..)
    It would then be possible to save the data to Flash every hour, and then only this one-hour section would be lost, i.e. only 4 measurements.


    p.kaczmarek2 wrote:
    mariomario wrote:

    Temperature with one decimal place, as well as humidity, because these sensors may have good resolution, but the (real) accuracy is a bit of a lottery (I used to play with different sensors and each one shows something different...).

    You still need to specify the temperature range because -20°C to 40°C is different than -40°C to 100°C. Although I know that I`m exaggerating a bit with this packing, considering today`s flash capacities it`s a bit of an art for art`s sake.

    With today`s RAM and Flash memory capacities, it is better to do it as conveniently as possible without worrying about individual data bits. Temperature range -40C to 100C with display up to 1 decimal place, I think it will satisfy most potential users
  • #14 21042575
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    mariomario wrote:

    Even without the optimizations discussed above, we have up to 45KiB of data to transfer from ESP to the browser. These are probably not large amounts of data.

    I haven`t played much with advanced JavaScript and I`m not at the top in this language, but I rather thought that the measurements were sent to the browser as text, not as binary data. So, for example, this JavaScript array:
    Code: Javascript
    Log in, to see the code

    Here we have 4 bytes (because it`s ASCII) for the measurement, four because I`m including a comma. But, for example, with timestamps it is much worse, at the moment when I write unix epoch time is 1712916348, and that is already 10 characters...
    Well, the fact is that after counting how many bytes in ASCII there will be, the result is similar to these 16 binary bytes.

    Additionally, you can think not about a static website, but about a website written in Javascript communicating with the ESP via the measurement API. This means that first, if we have a summary, e.g. for 7 days, we ask ESP about hourly averages (which ESP will calculate itself). When displaying measurements from 7 days, you don`t need resolution down to 15 minutes.

    Although I think I saw that Javascript also parses binary files. The bot definitely "knows":
    @ElektrodaBot show an example of downloading from the Internet via GET and parsing the binary file in Javascript. Load the 16-bit integer and 32-bit integer types from it.
    Helpful post? Buy me a coffee.
  • #15 21042577
    ElektrodaBot
    AI OpenAI ChatGPT4
    Posts: 8733
    Help: 350
    Rate: 919
    p.kaczmarek2 wrote:
    show an example of downloading from the Internet via GET and parsing the binary file in Javascript. Load the 16-bit integer and 32-bit integer types from it.
    Hello!

    I will be happy to show you how you can download a binary file from the Internet using the GET method and parse it in JavaScript to extract 16-bit integer and 32-bit integer values from it. I have prepared a step-by-step process for completing this task for you. Let`s try to do it together!

    ### Step 1: Downloading the binary file

    To download the binary file from the Internet, we will use the Fetch API, which is built into most modern web browsers. The following code shows how to do this:

    Code: Javascript
    Log in, to see the code


    In this code, the `getBinaryFile` function takes as an argument the URL of the file you want to download. It uses the Fetch API to perform an HTTP GET request. If the request is successful, the response is processed as an `ArrayBuffer`, which is a type used to store binary data in JavaScript.

    ### Step 2: Parsing the binary file

    After downloading the binary data, we need to parse it appropriately to extract the values we are interested in. For this purpose, we will use the DataView object, which allows the interpretation of binary data stored in `ArrayBuffer`.

    Below you will find an example function that shows how to process an `ArrayBuffer` and read a 16-bit and 32-bit integer from it:

    Code: Javascript
    Log in, to see the code


    In this code, `dataView.getInt16(0, true)` reads a 16-bit integer starting from position 0 in the buffer, using Little Endian byte order. Similarly, `dataView.getInt32(2, true)` reads a 32-bit integer starting from position 2 because a 16-bit number takes up 2 bytes.

    ### Step 3: Connecting everything

    Now that we have the functions for fetching and parsing data, we can use them together to download a binary file from the internet and parse it for values:

    Code: Javascript
    Log in, to see the code


    Make sure you change `http://example.com/plik.bin` to the URL pointing to the actual location of the binary file you want to download and parse.

    I hope this information is helpful to you. If you have any questions or need further assistance, please feel free to ask. Good luck with your project!
    You can call me in any topic by writing @ElektrodaBot. Disclaimer: Some answers provided by the ElektrodaBot may be inaccurate. Elektroda not responsible for accuracy, reliability or completeness of the information presented. All ElektrodaBot responses should be verified by the user. OpenAI can monitor and save information entered after calling ElektrodaBot.
  • #16 21042590
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    @mariomario, I think you`re right that there`s no point in packing bit by bit. I`m exaggerating a bit out of habit, because I have created/am making systems where I fight for every bit, etc.

    If you were to offer a binary measurement API, you could simply copy the readings from memory to the GET response without converting them to ASCII text. This way we would avoid a lot of sprintf and similar operations on the ESP side.

    But still, out of curiosity...

    @ElektrodaBot Write a Javascript parse of a DataView object where the first 10 bits are the value A, the next 12 bits are the value B, and the next 18 bits are C. Suggest ways on how I can process this and split the values into separate variables
    Helpful post? Buy me a coffee.
  • #17 21042595
    ElektrodaBot
    AI OpenAI ChatGPT4
    Posts: 8733
    Help: 350
    Rate: 919
    p.kaczmarek2 wrote:
    Write a Javascript parse of a DataView object where the first 10 bits are the value A, the next 12 bits are the value B, and the next 18 bits are C. Suggest ways on how I can process this and split the values into separate variables
    Welcome back!

    Parsing the complex data format from a `DataView` object requires a special approach because we will be working at the bit level. In your case, we are dealing with a total of 40 bits of data, where the first 10 bits are the A value, the next 12 bits are the B value, and the last 18 bits are the C value.

    First, let`s see how we can split this data into separate variables:

    ### Step 1: Preparing to work with bits

    JavaScript does not have direct support for single-bit operations at the `DataView` level. For this reason, we will first have to load the data in larger blocks (e.g. 16-bit or 32-bit) and then use bitwise operations to extract the values we are interested in.

    ### Step 2: Read data and extract values

    Assuming our 40 bits of data are packed into the first 5 bytes of the buffer, we can do it as follows:

    Code: Javascript
    Log in, to see the code


    ### Step 3: Explain the action

    1. We read 32-bit data – this allows us to easily access the first 32 bits (4 bytes). `getUint32` reads values as unsigned numbers, which is important when working with bits.
    2. We read the fifth byte – allows access to the last 8 bits of data.
    3. A value extraction – we use the `&` (AND) operation with the `0x3FF` mask (which in binary consists of 10 lit bits) to keep only the 10 least significant bits.
    4. B-value extraction – here we first shift the bits to the right by 10 places to get rid of the bits corresponding to A, and then we apply the `&` operation again, this time with a mask for 12 bits (`0xFFF`).
    5. C value extraction – requires a bit more work. We shift the data from the first 32 bits 22 places to the right to leave only the most significant bits that do not belong to values A and B. We shift the value from the 5th byte to the left 6 places and OR (`|`) to add 2 missing bits, giving a total of 18 bits of data per C value.

    ### Summary

    This is how we can split binary data into specific, smaller bit values in JavaScript. The key is to use bitwise operations such as AND, OR, and bit shifts left and right to manipulate and extract data from larger binary blocks.

    If you have any questions or need further clarification, please feel free to let me know. I`d be happy to assist!
    You can call me in any topic by writing @ElektrodaBot. Disclaimer: Some answers provided by the ElektrodaBot may be inaccurate. Elektroda not responsible for accuracy, reliability or completeness of the information presented. All ElektrodaBot responses should be verified by the user. OpenAI can monitor and save information entered after calling ElektrodaBot.
  • #18 21042604
    p.kaczmarek2
    Moderator Smart Home
    Posts: 14408
    Help: 650
    Rate: 12345
    I see that in JS it basically looks like what I would do in C.
    Helpful post? Buy me a coffee.
📢 Listen (AI):

Topic summary

✨ The discussion revolves around creating a weather station using the Wemos D1 (ESP8266) and the DHT11 sensor, with a focus on displaying temperature and humidity data on a web interface. Participants share insights on sensor alternatives, such as the BME280 and SHT3x series, which offer improved accuracy and additional features. The conversation also addresses memory management for storing measurements, suggesting the use of external storage like EEPROM or Flash to retain data across power losses. Various strategies for optimizing data structure and transmission to the web interface are discussed, including the potential for a binary measurement API to enhance efficiency. The feasibility of saving data to Flash and the implications of memory usage for long-term data collection are also explored.
Generated by the language model.

FAQ

TL;DR: This FAQ shows Wemos D1 mini beginners how to build a Wi‑Fi weather station that stores 144 samples per day, while one expert calls DHT11 a “trend indicator.” It solves the full path from OTA and NTP time to circular-buffer history and Chart.js graphs on a local ESP8266 web page. [#21040300]

Why it matters: The thread turns a simple ESP8266 demo into a practical blueprint for local monitoring, memory planning, and future upgrades.

Sensor / option What the thread says Best fit in this project
DHT11 Easy starter part, but treated more as a trend indicator than a precise sensor Learning, quick prototype
BME280 Explicitly called a slightly better sensor Better all-round upgrade
SHT3x Praised for temperature and humidity measurement Quality temp+RH alternative
BME280 + AHT20 module Reported as a budget replacement with two temperature sensors Low-cost experiment, redundancy
DHT22 / AM2301 Mentioned as an alternative; one user reported buzzing in an outdoor sensor Drop-in alternative with caveats

Key insight: The most valuable design choice is not the graph. It is storing readings in the right structure and reading the circular buffer in the correct order, from the oldest valid sample to the newest.

Quick Facts

  • The tutorial sizes history as 24*6, which equals 144 samples for one day at 10-minute intervals. [#21040300]
  • For 30 days at 15-minute intervals, the thread calculates 2880 readings total. [#21042000]
  • A Measurement with float temperature, float humidity, and time_t timestamp is estimated at 12 bytes or 16 bytes if time_t is 64-bit. [#21041976]
  • With a 16-byte record size, 2880 readings use about 46 kB RAM; cutting records to 8 bytes drops that to about 23 kB. [#21042000]
  • One flash-endurance example uses a W25Q80 with a minimum 100,000 erase cycles per 4 KB sector; the thread argues that wear-managed logging can stretch practical lifetime dramatically. [#21042317]

How do I build a simple weather station on a Wemos D1 mini (ESP8266) with a DHT11 sensor, OTA updates, and a web page in PlatformIO?

Build it by combining ready-made ESP8266 libraries in PlatformIO. 1. Connect Wi‑Fi, start ArduinoOTA, and initialize DHT_Unified on pin D3. 2. Read temperature and humidity, then store them in a Measurement record with a timestamp. 3. Start ESP8266WebServer on port 80 and serve an HTML page with current values or a graph. The thread’s working example also blinks D13 and samples every 1000 ms. [#21040300]

What is a circular buffer, and how does it help store temperature and humidity history on an ESP8266 with limited RAM?

A circular buffer stores a fixed number of readings and overwrites the oldest when full. "Circular buffer" is an array-based data structure that stores a limited number of elements, reuses the same memory, and wraps the write index with modulo arithmetic. On this ESP8266 project, lastSample++ followed by % SAMPLES_COUNT keeps RAM usage predictable. That matters because history can grow quickly: even 30 days at 15-minute intervals reaches 2880 readings. [#21040300]

How can I use ESP8266WebServer to show current DHT11 temperature, humidity, and NTP time on a local web page?

Use ESP8266WebServer server(80);, register server.on("/", HTTP_GET, handleRoot);, and call server.handleClient() in loop(). In handleRoot(), format HTML with sprintf, inserting temperature, humidity, and either timeClient.getFormattedTime() or a strftime string. The thread shows a page that returns HTTP 200 and displays all three values on one local page after Wi‑Fi connects. [#21040300]

What is NTPClient, and how do I use it on ESP8266 to fetch and format the current time from the Internet?

NTPClient is the library the project uses to fetch network time over UDP. "NTPClient" is a time-synchronization library that queries an NTP server, returns current network time, and exposes both epoch values and formatted strings for easy display. In the thread, it is created with WiFiUDP ntpUDP; NTPClient timeClient(ntpUDP, "pool.ntp.org");, then started with timeClient.begin() and refreshed by timeClient.update(). [#21040300]

Why is the DHT11 often considered more of a trend indicator than an accurate temperature sensor, and what practical limits should I expect?

The thread treats DHT11 as good for direction, not precision. One participant says it used to be “an indicator rather than a temperature sensor,” meaning it shows whether values rise or fall more reliably than it gives exact readings. In practice, expect it to work for a beginner graphing project, but expect limited trust in absolute accuracy and be ready to upgrade if measurement quality matters. [#21040993]

Which sensor is better for an ESP8266 weather station: DHT11, BME280, SHT3x, AHT20, or DHT22?

BME280 or SHT3x are presented as better choices than DHT11 for a serious ESP8266 weather station. The author explicitly names BME280 as “slightly better,” and another participant says the SHT3x series is praised. A combined BME280+AHT20 module is described as a budget option, though one user saw a 1.0–1.1°C difference between its two temperature readings. DHT22 is mentioned, but one outdoor sensor reportedly started buzzing. [#21041790]

How do I draw temperature and humidity graphs on an ESP8266 web interface using Chart.js and data collected from a DHT sensor?

Generate three JavaScript arrays on the ESP8266: labels, temperatures, and humidities, then inject them into a Chart.js line chart. The thread builds HTML with a <canvas> element and loads Chart.js from cdn.jsdelivr.net. Each label is a formatted timestamp, while the two datasets plot temperature and humidity with different colors. This gives a graph directly in the browser, using only local ESP8266 data and one HTTP page. [#21040300]

Why does a Chart.js graph break when I read samples from a circular buffer in the wrong order, and how do I iterate from the oldest entry correctly?

The graph breaks because the buffer is not a simple linear history after wraparound. Once the array fills, the newest sample may sit in the middle, so reading from index 0 upward mixes old and new data. The thread fixes this by starting from the oldest entry, calculated as (lastSample+1)%SAMPLES_COUNT, then iterating SAMPLES_COUNT times and skipping records where timestamp == 0. That preserves chronological order. [#21040300]

How many ESP8266 RAM bytes would 30 days of temperature and humidity measurements every 15 minutes use, and how can I estimate it from the Measurement struct size?

For 30 days at 15-minute intervals, you need 2880 readings, and the thread estimates about 46 kB if each Measurement is 16 bytes. The math is simple: 30*96 = 2880, because 15-minute logging gives 4 readings per hour and 96 per day. Then multiply record count by struct size. If the struct shrinks to 8 bytes, the same history drops to about 23 kB. [#21042000]

What are the best ways to reduce measurement storage size on ESP8266: using smaller structs, packed values, lower timestamp resolution, or binary API transfers?

The best reductions come from shrinking the struct first, then lowering timestamp precision, then avoiding text transfer. The thread suggests packing temperature and humidity into about 3 bytes instead of two floats, and replacing full time_t with seconds, tens of seconds, or minutes counted from 2024-01-01. It also notes that a binary API can copy raw readings directly to the HTTP response, avoiding sprintf and ASCII expansion on the ESP8266. [#21041976]

How should I store long-term weather history on ESP8266 so data survives power loss: RAM, built-in flash, external EEPROM, or another flash chip?

Use flash or external nonvolatile memory if you need data after power loss. The thread warns that RAM disappears on outage, then suggests built-in flash, external EEPROM, or even another flash chip. It also argues that flash wear is manageable: with a W25Q80 rated for at least 100,000 erase cycles per 4 KB sector, careful wear-saving writes can make hourly or periodic logging practical for very long periods. [#21042317]

Why might sending 2880 readings from ESP8266 to the browser as JavaScript text arrays be larger than expected, and what alternatives are more efficient?

Text arrays are larger than many people expect because every number and timestamp becomes ASCII characters plus commas. The thread notes that a value like 19.5 already takes about 4 text bytes, while a Unix timestamp like 1712916348 takes 10 characters before separators. For 2880 readings, that overhead adds up quickly. More efficient options are a binary measurement API, summary views such as hourly averages, or separate detailed charts per day. [#21042575]

How can I create a binary measurement API on ESP8266 and parse the response in JavaScript with ArrayBuffer and DataView?

Return raw bytes from the ESP8266 instead of JavaScript text, then parse them in the browser with fetch(), arrayBuffer(), and DataView. The thread explicitly proposes copying readings from memory straight into the GET response to avoid sprintf work. It also shows JavaScript that reads binary data with getInt16, getInt32, and even bit-packed fields. That approach cuts formatting overhead and gives tighter control over bandwidth. [#21042590]

What role would WiFiManager play in an ESP8266 weather station, and how would it improve setup compared with hard-coded SSID and password?

WiFiManager would remove the need to hard-code MY_SSID and MY_PASS in the firmware. The thread recommends it for a more complete product, especially if other users will install the weather station. That improves first-time setup and makes the device easier to move between networks without rebuilding and reflashing the code. It also aligns with the thread’s goal of turning a tutorial into a fuller system. [#21042317]

How should I redesign the sampling loop on ESP8266 so measurement timing is cleaner than using delay(1) and a counter, for example with Ticker?

Replace the delay(1) plus incrementing counter with a timer-driven design. The author already separates timing into g_sampleIntervalMS = 1000 and says the loop should be rewritten more cleanly, suggesting Ticker as the next step. A cleaner design keeps ArduinoOTA.handle(), server.handleClient(), and timeClient.update() responsive while sampling on schedule. That avoids fragile timing based on loop speed and manual counter increments. [#21040300]
Generated by the language model.
ADVERTISEMENT