Monitoring Solar Power Over WiFi

solar panel

India is rapidly switching to renewable energy sources. It’s not just the government that’s making the switch but people as well have been switching to solar roof tops for their power requirements. In our society, 30-40% houses have set up solar panels on their roofs. Most of these houses do not pay at all to the MSEB but on the other hand contribute surplus to the grid.

The one that is setup at my home - shown in picture above - has 2.27 kWh capacity. Over the past 3-4 months I have been monitoring the output of these panels and they have been on average producing 220-270 kW per month. There is around 70-100 units of surplus which is fed into the grid every month. I have seen output of about maximum 11.6 kWh in a single day. Rest of days it fluctuates in 6-8 kWh range. There are many parameters affecting output like weather, panel temperature, dust on the panels. I do not have any control over the weather but rest two parameters can be tweaked to improve effeciency of these panels. But before deploying measures to improve effeciency I need more data to estimate if these measures are worth deploying.

Currently, only way to monitor output of these panels is by checking the inverter manually. This is not useful at all because to perform any kind of proper analysis I would need data from these panels on per minute basis. This is not feasible at the moment. So I needed a system that will continuously log the panel output to a central database. Logging to central system has more advantages: I can not only do a thorough analysis but also get power stats over the Internet. Let’s build this system 😃

Components

The sensor must be clipped on phase

  1. SCT-013-030: A non-invasive current sensor with a clip-on. I did not wanted to meddle with 230V mains supply, hence this sensor. You can clip it on any wire and it will generate output voltage corresponding to the current flowing from the wire.
  2. Arduino Uno: I had this sitting in my closet. It has an ADC onboard which can be used to measure output of the current sensor.
  3. ESP01 WiFi module: This module can directly transmit data over WiFi network and it was the cheapest one that I found. Also, it seems it can be easily interfaced with Arduino.
  4. 3.3V power supply: ESP01 needs a regulated 3.3v power supply
  5. A raspberry pi: This is used for capturing the data shared by ESP01 and as a data logger. Eventually this will also be used as a server to broadcast captured data over Internet.

A Note on ESP01

I am using Micropython firmware for ESP01. Stock AT firmware should also work. I chose Micropython because I was facing issues communicating with Arduino over serial. Although I later found out that the issue was because of power supply. ESP01 strictly needs 3.3V[1]. The issue went away once it was powered by a dedicated 3.3V power supply instead of Arduino’s 3.3V supply. I spent lot of time resolving this issue. 😅

Interfacing Arduino With ESP01

The interfacing here is for serial communication between Arduino and ESP01. If you want to flash or load programs on ESP01 using Arduino you will have to change wiring. Refer this note for wiring, flashing and loading programs on ESP01.

Below is the wiring between UNO and ESP01 for serial communication. You can choose different pins for RX & TX in your code.

    UNO — ESP8266
    RX — 11 
    TX — 10 
    GND — GND 
    3.3V — VCC 
    3.3V — CH_PD

The configuration of Arduino environment is out of scope of this article. Arduino IDE makes it very easy to load programs on Arduino.

Serial I/O and Broadcast

Serial write with Arduino
Here’s listing of code that writes Hello from Arduino to ESP01

#include <SoftwareSerial.h>
#define RX 10
#define TX 11
SoftwareSerial esp8266(RX, TX);
void setup()
{
    esp8266.begin(115200);
}
void loop()
{
    esp8266.println("Hello from Arduino");
    delay(3000);
}

Serial read from ESP01
ESP SoC has 2 serial I/Os. UART 1 is write only and UART 0 can do perform both read and write. That is why we have to use UART0 for communication with Arduino. But there’s a catch with UART0, it is used by micropython REPL for loading programs. The only way to communicate with Arduino over serial is to detach UART0 from REPL. Doing this would disable the functionality to load or edit programs until it is re-attached[2].

uos.dupterm(None, 1)            # detach UART from REPL. 
uart = UART(0, 115200)          # UART 0 is available for our program
if uart.any():
    message = uart.readline()   # read data from serial input
uos.dupterm(uart, 1)            # re-attach UART with REPL before end of program

Broadcast over WiFi
Now that the data is read from serial it’s time to broadcast it over WiFi using below function

def broadcast(message: bytes):
    cs = socket(AF_INET, SOCK_DGRAM)
    cs.sendto(message, ('192.168.0.255', 7007))

The socket is a UDP socket as selected by SOCK_DGRAM parameter. We are choosing to send the message over broadcast address 192.168.0.255 so that we can receive it on any device on network. Listening to the message on a Linux system is as easy as running the command

# listen continuosly for messages on udp socket port 7007
nc -kul 7007

I would recommend that you run the serial read and broadcast operation on your ESP01 before you load the actual program to confirm everything is working properly.

Measuring Current

We can measure AC output voltage of CT sensor using ADC on Arduino. The signal first needs to be shifted above 0V in order to meet requirements of ADC. We use a voltage divider circuit as shown below to shift the analog signal by 2.5V so that its lower peak always stays above 0V. Note that our CT sensor has output of 0-1V.

We will be using EmonLib from Open Energy Monitor to sample and measure the RMS current. Here’s a code listing to read current from A0 pin with EmonLib.

EnergyMonitor emon1;
void setup()
{
  Serial.begin(9600);
  emon1.current(1, 30); //specify pin and calibration
}
void loop()
{
  double I = emon1.calcIrms(1480); 		//number of samples to take 
  Serial.println("cu value=" + String(I));	//write data to esp in a InfluxDB line format
  delay(3000);
}

emon1.current(1,30) is used to choose ADC pin and for calibration. For the sensor I am using, it is 30. You can find out more about calibration calculation here.

Data Logging And Dashboarding

For logging we are going to use timeseries database InfluxDB. InfluxDB integrates well with Grafana which we will be using for dashboarding. Both InfluxDB and Grafana will be installed on Raspberry Pi. Installation of both application is pretty straightforward. You can refer the guides included in notes below[3].

Ingesting Data With InfluxDB Line Protocol

The fastest way of inserting a record in InfluxDB is through it’s CLI. Here’s an example:

$ influxdb
> CREATE pmon
> USE pmon
> cu value=2.2
> cu value=2.5

First we create database pmon and then insert values 2.2 and 2.5 as current(cu) in it. You can see inserted records along with timestamps using show measurements command. InfluxDB supports a lot of interfaces REST, UDP, etc,. Since we have already implemented the broadcast over UDP with ESP01 we will be using UDP interface of InfluxDB but first we need to enable UDP interface before we can start ingesting data over it. You can enable the interface in influxdb.conf. This file on exist at /etc/influxdb/influxdb.conf on raspbian. In the udp section add

[[udp]]
  enabled = true
  bind-address = "0.0.0.0:7007"		# port address to listen on
  database = "pmon"
  retention-policy = ""
  batch-size = 2						# write to database after collecting this many measurements

There are more options in the config file which you can explore as per your requirement.

Dashboard with Grafana

Integrating Grafana with InfluxDB is pretty easy. Grafana can be accessed from browser for its configuration. Just point your browser to your Pi IP with 3000 port and that should open Grafana. You will have to go through some basic configuration steps and then from the settings panel you can add InfluxDB as data source. That’s it. You can start creating panels for the data that has been recorded.

Here’s Grafana dashboard for my home setup

Creating Grafana Panels

InfluxDB queries are fairly easy to understand so I will just put the queries behind each panel that I have used

  1. Current Panel
    SELECT value FROM "cu"
  2. Power Panel
    SELECT value*252 FROM "cu"
    The voltage at my home averages around 252V. Since, P=VxI. Power generated = 252*value.
  3. Energy Generated Hourly in Wh panel
    SELECT integral("power")/3600 FROM (SELECT value*252 AS power FROM "cu") WHERE $timeFilter GROUP BY time(1h) fill(null)
    Energy generated is the area under the power curve. Hence, the integral. We are calculating energy generated on per hour basis.
  4. Today’s generation panel
    SELECT integral("value")*252/3600000 FROM (SELECT value FROM "cu") where time > now()-12h
    Same as energy generated calculation but here we do for last 12 hours.
  5. Monthly energy generated panel
    Same as today’s generation panel but calculated over period of 30d
  6. Carbon panels
    Carbon generated per kWh is called as carbon factor for the grid. For India, it is 0.7 and thus Energy(E)*0.7 gives us the carbon removed from atmosphere.

Final Words

I am very happy with this setup and it’s been very reliable so far even though I have done all of this on a bread board! The current readings are pretty accurate above 1A. There is deviation in power and energy measurements but this is because we are assuming constant voltage while in reality it fluctuates quite a bit. Maybe I will add voltage sensor to this!

Repository

Both micropython and arduino code is uploaded at https://github.com/quaintdev/EnergyMonitor

Notes:

  1. There are instances when people have ran ESP01 chip on 5V. This is most probably because of how different manufactures design their chips. The ESP8266 SoC on board ESP01 needs 3.3V.
  2. Our program is expected to communicate with Arduino continously and that is why we cannot re-attach the REPL. In the final code listing I have added delay of 10 seconds before detatching REPL. This gives us enough time to load new programs on ESP if required. If we do not do this then we loose the ability to load programs on ESP until we re-flash the firmware.
  3. Install grafana on Raspberry PI
    Install influxdb on Raspberry PI
  4. CT sensor interfacing with Arduino