Using ec3k with Raspberry Pi

The gnuradio based ec3k software allows you to receive data from the Voltcraft Energy Count 3000 sold by Conrad. This device is also sold under different brand names such as “Velleman NETBSEM4” and “La Crosse Techology Remote Cost Control Monitor – RS3620”.

These sets contain plugs and a little display device that looks a little like a remote control. The display device can be used to receive and display the data from the plugs. Each plug sends out a data packet each 5 seconds, the data that is sent contains the following information:

• The unique ID for the plug (a 4 character hex number)
• The current power usage (in Watts)
• The maximum usage of the device (in Watts) since the start of the measurements
• The total energy usage (in Wh) since start of the measurements
• The time that the device was ‘on’
• The total time since the start of the measurements

Receiving this information using a computer allows you to plot nice graphs of your energy usage per device / plug, giving great insight in energy usage of different devices in your household.

Using the ec3k software on a regular x86 or x64 based Linux device is quite straightforward, however using it on an ARM processor based Linux device like the Raspberry Pi turned out to have some quirks. The main thing is that the Gnuradio software is not (yet) available as a package in the repositories of the Linux variants for the Raspberry and that there is an issue when compiling the Gnuradio sources as-is. That’s why I wrote this little blog to help out anyone who wants to use this setup.

As all software mentioned in this blog is still evolving always first have a look at the requirements as mentioned for ec3k at the github page or the pypi page.

The basic steps are

• Download and install a Raspberry Pi image with Gnuradio pre-installed
• Download and install the ec3k software
• Fix the Gnuradio installation by downloading Gnuradio and compiling a small part of it
• Start the ec3k software for testing
• Adjust the ec3k_recv script
• Write a script to send the data received to your favorite dashboard software

Please note that I am doing nothing new or special here, I’m just using software that others have kindly made available to the public.

Download a Raspberry Pi image with Gnuradio

Gareth Hayes created a Raspberry Pi image for Software Defined Radio which includes Gnuradio. It can be downloaded using the Torrent mentioned in his article. It’s a 3.75 Gb download so it can take some time depending on your Internet speed.

Install the Raspberry Pi image on an SD-card

After you have downloaded the Raspberry Pi image, you can install it on an SD-card using your favorite software, I used Win32 Disk Imager for this. Please use an SD-card of at least 8Gb so you will have some spare room for software and data. Put the SD-card in your PC, just start Win32 Disk Imager, choose the downloaded file as “Image File”, and the drive for your SD-card as “Device”. Then press “Write”.

If you don’t have a spare SD-card and you want to preserve the content of your current Raspberry Pi SD-card, then choose a backup location in “Image File” and press “Read” first. This will save your current SD-card image so you can restore it later.

Once the image is on the SD-card, boot your Pi with it. It is easiest if you connect the Ethernet port of your Raspberry to your home network (assuming a DHCP server is running locally). This way the Pi will automatically obtain an IP address. (you can determine it’s IP address by having a look at the DHCP administration of your local router (or other DHCP server), it will register with the name “raspberry”). You can login to the Pi via SSH using the username ‘pi’ and password ‘raspberry’.

Note that the installed image file does not have any free space left so we can’t add anything to it. We first have to extend the partition to the full size of the SD-card. You can do this using standard tools like fdisk and  resize2fs, Basically the steps are: throw away the Linux partition and create it again with the same “first sector numer”, but now let fdisk fill in the “last sector number” which defaults to the end of the disk / SD-card. Then use resize2fs to extend the filesystem within the larger partition. This goes like:

pi@raspberrypi% sudo fdisk /dev/mmcblk0

Command (m for help): p

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b5098

 Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 7872511 3874816 83 Linux

Command (m for help): d
Partition number (1-4): 2

Command (m for help): p

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b5098

 Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)

Command (m for help): n
Partition type:
 p primary (1 primary, 0 extended, 3 free)
 e extended
Select (default p): p
Partition number (1-4, default 2): 2
First sector (2048-15523839, default 2048): 122880
Last sector, +sectors or +size{K,M,G} (122880-15523839, default 15523839): 
Using default value 15523839

Command (m for help): p

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b5098

 Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 15523839 7700480 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

pi@raspberrypi% sudo shutdown -r 0
Broadcast message from root@raspberrypi (pts/0) (Tue Oct 28 09:43:20 2014): 
 The system is going down for reboot NOW!

Now login again and extend the filesystem

pi@raspberrypi% sudo resize2fs /dev/mmcblk0p2
resize2fs 1.42.5 (29-Jul-2012)
Filesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/mmcblk0p2 is now 1925120 blocks long.

Now we have a raspbian image with Gnuradio and some free space on it.

Download and install the ec3k software

Once logged in to the Pi and assuming you have Internet access you can download the ec3k software, unpack and install it. Please first have a look at https://pypi.python.org/packages/source/e/ec3k and verify that you are downloading the most recent version (which was ec3k-1.1.1 at the time of writing, note that the original  ec3k-1.1.0 does not work together with Gnuradio 3.5.7).

wget https://pypi.python.org/packages/source/e/ec3k/ec3k-1.1.1.tar.gz
tar xvzf ec3k-1.1.1.tar.gz
cd ec3k-1.1.1/
sudo python setup.py install
python setup.py test # this is optional

If everything was OK you should now be able to run the ec3k_recv script, so let’s try:

pi@raspberrypi ~ $ ec3k_recv
linux; GNU C++ version 4.6.3; Boost_104900; UHD_003.004.002-0-unknown

gr-osmosdr v0.1.3-4-g42c66fdd (0.1.4git) gnuradio 3.7.5git-0-g2bb2b314
built-in source types: file fcd rtl rtl_tcp uhd hackrf rfspace
Using device #0 Generic RTL2832U OEM
Using 16 buffers of size 262144.
usb_open error -3
Please fix the device permissions, e.g. by installing the udev rules file rtl-sdr.rules

FATAL: Failed to open rtlsdr device.

Trying to fill up 1 missing channel(s) with null source(s).
This is being done to prevent the application from crashing
due to gnuradio bug #528.

Using Volk machine: generic
VOLK: Error allocating memory (posix_memalign: 22)
Segmentation fault

So not everything seems to be OK. There are still 2 issues to be solved:

• Installing udev rules for rtl-sdr
• Fix the Gnuradio VOLK memory allocation issue

Fixing the issues

For installing the udev rules, please follow these instructions on instructables.com. Please use the ID’s as found by lsusb for your rtl-sdr device. Use the command

sudo /etc/init.d/udev restart

to restart udev.

Fixing the Gnuradio VOLK memory allocation issue takes a little bit more time: First download and unpack the Gnuradio source:

wget http://gnuradio.org/releases/gnuradio/gnuradio-3.7.5.tar.gz
tar xvzf http://gnuradio.org/releases/gnuradio/gnuradio-3.7.5.tar.gz

We have to apply the fix as mentioned here.

So obtain the patch, and apply it:

wget http://gnuradio.org/redmine/attachments/download/821/0001-volk-Fix-volk_malloc-when-alignment-is-1.patch
patch -p1 < *volk*.patch

Now build the volk part of Gnuradio and install it:

mkdir build
cd build
cmake -DENABLE_DEFAULT=Off -DENABLE_VOLK=True -Dhave_mfpu_neon=0 ..
make
sudo make install

Use it (and fix another issue…)

Now we should be able to use the ec3k_recv script. I however noticed that the ec3k_recv as installed indeed runs, shows output about Noise levels steadily increasing (with -dB numbers going from about -40 dB up to -16 db in my case (due to auto-gain), however it would never show any valid output.

If you experience the same, don’t despair: It turns out that the default frequency that is being used in the ec3k software (868.402 MHz) doesn’t work for my dongle. At first I fiddled around a little with the frequency and found 868.24 MHz to be working quite decently. However when using another almost identical dongle it again didn’t play well with that adjusted frequency. I got curious and wrote a little script to ‘scan’ several frequencies around 868.3 MHz, I put the results in a stacked bar-graph for my dongles. I found the results quite surprising, I expected to see some kind of “standard distribution”, however the results were quite different:

So it seems like there are 4 regions in the frequency spectrum around 868.3 MHz which are sensitive to receiving the ec3k data packets with gaps in between them where the dongle seems almost deaf to the signals. I’m not yet quite sure what causes this. It might be because of the used modulation (FSK), the way the decoding software is setup, the dongle/tuner/antenna used or any combination of these. Anyone who is more experienced with radio technology, DSP and/or SDR is welcome to comment on this phenomenon.

Anyhow: this clearly shows that setting the right frequency for your dongle is of utmost importance if you want to receive most data packages from these plugs. I will cleanup the code of the scanner script a little and try to have it incorporated in the ec3k software. I’m still having some doubts about the results of the test and finding the best frequency because of the effects of using auto-gain during the measurement.

Another way to find the right frequency is to hook-up your dongle to a Windows system, Install SDR# or another SDR package that can do a waterfall plot (a.k.a scrolling spectrum plot). It seems like the best frequencies are just besides the center frequency for the dongles (where there is no ‘red’ color, this also depends on the settings of SDR#).

You can fix the frequency in ec3k.py at line 275 (for version 1.1.1, this may be different for a new version of ec3k). If you write your own client program (like ec3k_recv) then it is best you also provide this frequency using an extra ‘freq=868.xxxe6’ parameter in the call to ec3k.EnergyCount3K. This way you can later update the ec3k software without having to modify its source code.

Just have a go at it and you will probably find a frequency that is suitable for your plugs.

Optimize it

Because the Raspberry Pi has a quite modest processor and ec3k uses Guradio in a quite powerful way, cascading several Gnuradio blocks together, CPU usage will be quite high. Even so high you will typically get so called overflows (the dreaded “OOOOO” lines from Gnuradio) which mean that the software can’t keep up with the data that is flowing in.

To alleviate things a little we can use an optimized capture program written in C instead of in Python, This will place a lesser burden on the CPU minimizing overflows. This can be done as follows:

wget https://www.tablix.org/~avian/am433/am433-0.0.5.tar.gz
tar xvzf am433-0.0.5.tar.gz
cd am433-0.0.5/am433/
make
sudo cp capture /usr/local/bin

Now ec3k_recv will use the C program capture instead of the somewhat less efficient capture.py program. You should see less O’s now 🙂

Feed the data to a dashboard

This will be handled in a next edition of this blog.

Relevant Links

• https://github.com/avian2/ec3k – Github page for ec3k
• https://pypi.python.org/pypi/ec3k – Python package for the ec3k software
• http://garethhayes.net/gnu-radio-for-raspberry-pi/ – A Raspberry Pi image with Gnuradio
• http://gnuradio.org/ – Homepage of the Gnuradio project
• http://www.instructables.com/id/rtl-sdr-on-Ubuntu/step3/Setup-udev-rules/ – Adding udev rules
• https://www.tablix.org/~avian/blog/archives/2012/07/energycount_3000_part_2/ – Avian’s blog, the author of ec3k