Using ec3k with Raspberry Pi

The gnuradio based ec3k software allows you to receive data from the Voltcraft Energy Count 3000 sold by Conrad. This device is also sold under different brand names such as “Velleman NETBSEM4” and “La Crosse Techology Remote Cost Control Monitor – RS3620”.

volctraft-ec3000velleman-netbsem4

These sets contain plugs and a little display device that looks a little like a remote control. The display device can be used to receive and display the data from the plugs. Each plug sends out a data packet each 5 seconds, the data that is sent contains the following information:

  • The unique ID for the plug (a 4 character hex number)
  • The current power usage (in Watts)
  • The maximum usage of the device (in Watts) since the start of the measurements
  • The total energy usage (in Wh) since start of the measurements
  • The time that the device was ‘on’
  • The total time since the start of the measurements

emoncmsReceiving this information using a computer allows you to plot nice graphs of your energy usage per device / plug, giving great insight in energy usage of different devices in your household.

Using the ec3k software on a regular x86 or x64 based Linux device is quite straightforward, however using it on an ARM processor based Linux device like the Raspberry Pi turned out to have some quirks. The main thing is that the Gnuradio software is not (yet) available as a package in the repositories of the Linux variants for the Raspberry and that there is an issue when compiling the Gnuradio sources as-is. That’s why I wrote this little blog to help out anyone who wants to use this setup.

As all software mentioned in this blog is still evolving always first have a look at the requirements as mentioned for ec3k at the github page or the pypi page.

The basic steps are

  • Download and install a Raspberry Pi image with Gnuradio pre-installed
  • Download and install the ec3k software
  • Fix the Gnuradio installation by downloading Gnuradio and compiling a small part of it
  • Start the ec3k software for testing
  • Adjust the ec3k_recv script
  • Write a script to send the data received to your favorite dashboard software

Please note that I am doing nothing new or special here, I’m just using software that others have kindly made available to the public.

Download a Raspberry Pi image with Gnuradio

Gareth Hayes created a Raspberry Pi image for Software Defined Radio which includes Gnuradio. It can be downloaded using the Torrent mentioned in his article. It’s a 3.75 Gb download so it can take some time depending on your Internet speed.

Install the Raspberry Pi image on an SD-card

After you have downloaded the Raspberry Pi image, you can install it on an SD-card using your favorite software, I used Win32 Disk Imager for this. Please use an SD-card of at least 8Gb so you will have some spare room for software and data. Put the SD-card in your PC, just start Win32 Disk Imager, choose the downloaded file as “Image File”, and the drive for your SD-card as “Device”. Then press “Write”.

If you don’t have a spare SD-card and you want to preserve the content of your current Raspberry Pi SD-card, then choose a backup location in “Image File” and press “Read” first. This will save your current SD-card image so you can restore it later.

Once the image is on the SD-card, boot your Pi with it. It is easiest if you connect the Ethernet port of your Raspberry to your home network (assuming a DHCP server is running locally). This way the Pi will automatically obtain an IP address. (you can determine it’s IP address by having a look at the DHCP administration of your local router (or other DHCP server), it will register with the name “raspberry”). You can login to the Pi via SSH using the username ‘pi’ and password ‘raspberry’.

Note that the installed image file does not have any free space left so we can’t add anything to it. We first have to extend the partition to the full size of the SD-card. You can do this using standard tools like fdisk and  resize2fs, Basically the steps are: throw away the Linux partition and create it again with the same “first sector numer”, but now let fdisk fill in the “last sector number” which defaults to the end of the disk / SD-card. Then use resize2fs to extend the filesystem within the larger partition. This goes like:

pi@raspberrypi% sudo fdisk /dev/mmcblk0

Command (m for help): p

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b5098

 Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 7872511 3874816 83 Linux

Command (m for help): d
Partition number (1-4): 2

Command (m for help): p

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b5098

 Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)

Command (m for help): n
Partition type:
 p primary (1 primary, 0 extended, 3 free)
 e extended
Select (default p): p
Partition number (1-4, default 2): 2
First sector (2048-15523839, default 2048): 122880
Last sector, +sectors or +size{K,M,G} (122880-15523839, default 15523839): 
Using default value 15523839

Command (m for help): p

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b5098

 Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 15523839 7700480 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

pi@raspberrypi% sudo shutdown -r 0
Broadcast message from root@raspberrypi (pts/0) (Tue Oct 28 09:43:20 2014): 
 The system is going down for reboot NOW!

Now login again and extend the filesystem

pi@raspberrypi% sudo resize2fs /dev/mmcblk0p2
resize2fs 1.42.5 (29-Jul-2012)
Filesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/mmcblk0p2 is now 1925120 blocks long.

Now we have a raspbian image with Gnuradio and some free space on it.

Download and install the ec3k software

Once logged in to the Pi and assuming you have Internet access you can download the ec3k software, unpack and install it. Please first have a look at https://pypi.python.org/packages/source/e/ec3k and verify that you are downloading the most recent version (which was ec3k-1.1.1 at the time of writing, note that the original  ec3k-1.1.0 does not work together with Gnuradio 3.5.7).

wget https://pypi.python.org/packages/source/e/ec3k/ec3k-1.1.1.tar.gz
tar xvzf ec3k-1.1.1.tar.gz
cd ec3k-1.1.1/
sudo python setup.py install
python setup.py test # this is optional

If everything was OK you should now be able to run the ec3k_recv script, so let’s try:

pi@raspberrypi ~ $ ec3k_recv
linux; GNU C++ version 4.6.3; Boost_104900; UHD_003.004.002-0-unknown

gr-osmosdr v0.1.3-4-g42c66fdd (0.1.4git) gnuradio 3.7.5git-0-g2bb2b314
built-in source types: file fcd rtl rtl_tcp uhd hackrf rfspace
Using device #0 Generic RTL2832U OEM
Using 16 buffers of size 262144.
usb_open error -3
Please fix the device permissions, e.g. by installing the udev rules file rtl-sdr.rules

FATAL: Failed to open rtlsdr device.

Trying to fill up 1 missing channel(s) with null source(s).
This is being done to prevent the application from crashing
due to gnuradio bug #528.

Using Volk machine: generic
VOLK: Error allocating memory (posix_memalign: 22)
Segmentation fault

So not everything seems to be OK. There are still 2 issues to be solved:

  • Installing udev rules for rtl-sdr
  • Fix the Gnuradio VOLK memory allocation issue

Fixing the issues

For installing the udev rules, please follow these instructions on instructables.com. Please use the ID’s as found by lsusb for your rtl-sdr device. Use the command

sudo /etc/init.d/udev restart

to restart udev.

Fixing the Gnuradio VOLK memory allocation issue takes a little bit more time: First download and unpack the Gnuradio source:

wget http://gnuradio.org/releases/gnuradio/gnuradio-3.7.5.tar.gz
tar xvzf http://gnuradio.org/releases/gnuradio/gnuradio-3.7.5.tar.gz

We have to apply the fix as mentioned here.

So obtain the patch, and apply it:

wget http://gnuradio.org/redmine/attachments/download/821/0001-volk-Fix-volk_malloc-when-alignment-is-1.patch
patch -p1 < *volk*.patch

Now build the volk part of Gnuradio and install it:

mkdir build
cd build
cmake -DENABLE_DEFAULT=Off -DENABLE_VOLK=True -Dhave_mfpu_neon=0 ..
make
sudo make install

Use it (and fix another issue…)

Now we should be able to use the ec3k_recv script. I however noticed that the ec3k_recv as installed indeed runs, shows output about Noise levels steadily increasing (with -dB numbers going from about -40 dB up to -16 db in my case (due to auto-gain), however it would never show any valid output.

If you experience the same, don’t despair: It turns out that the default frequency that is being used in the ec3k software (868.402 MHz) doesn’t work for my dongle. At first I fiddled around a little with the frequency and found 868.24 MHz to be working quite decently. However when using another almost identical dongle it again didn’t play well with that adjusted frequency. I got curious and wrote a little script to ‘scan’ several frequencies around 868.3 MHz, I put the results in a stacked bar-graph for my dongles. I found the results quite surprising, I expected to see some kind of “standard distribution”, however the results were quite different:

ec3k_distribution

So it seems like there are 4 regions in the frequency spectrum around 868.3 MHz which are sensitive to receiving the ec3k data packets with gaps in between them where the dongle seems almost deaf to the signals. I’m not yet quite sure what causes this. It might be because of the used modulation (FSK), the way the decoding software is setup, the dongle/tuner/antenna used or any combination of these. Anyone who is more experienced with radio technology, DSP and/or SDR is welcome to comment on this phenomenon.

Anyhow: this clearly shows that setting the right frequency for your dongle is of utmost importance if you want to receive most data packages from these plugs. I will cleanup the code of the scanner script a little and try to have it incorporated in the ec3k software. I’m still having some doubts about the results of the test and finding the best frequency because of the effects of using auto-gain during the measurement.

Another way to find the right frequency is to hook-up your dongle to a Windows system, Install SDR# or another SDR package that can do a waterfall plot (a.k.a scrolling spectrum plot). It seems like the best frequencies are just besides the center frequency for the dongles (where there is no ‘red’ color, this also depends on the settings of SDR#).

sdr-sharp-ec3k

You can fix the frequency in ec3k.py at line 275 (for version 1.1.1, this may be different for a new version of ec3k). If you write your own client program (like ec3k_recv) then it is best you also provide this frequency using an extra ‘freq=868.xxxe6’ parameter in the call to ec3k.EnergyCount3K. This way you can later update the ec3k software without having to modify its source code.

Just have a go at it and you will probably find a frequency that is suitable for your plugs.

Optimize it

Because the Raspberry Pi has a quite modest processor and ec3k uses Guradio in a quite powerful way, cascading several Gnuradio blocks together, CPU usage will be quite high. Even so high you will typically get so called overflows (the dreaded “OOOOO” lines from Gnuradio) which mean that the software can’t keep up with the data that is flowing in.

To alleviate things a little we can use an optimized capture program written in C instead of in Python, This will place a lesser burden on the CPU minimizing overflows. This can be done as follows:

wget https://www.tablix.org/~avian/am433/am433-0.0.5.tar.gz
tar xvzf am433-0.0.5.tar.gz
cd am433-0.0.5/am433/
make
sudo cp capture /usr/local/bin

Now ec3k_recv will use the C program capture instead of the somewhat less efficient capture.py program. You should see less O’s now 🙂

Feed the data to a dashboard

This will be handled in a next edition of this blog.

Relevant Links

Advertisements

21 gedachten over “Using ec3k with Raspberry Pi

  1. Pingback: Getting data from Voltcraft Energy Count 3000 on your computer | Batilan Blog

  2. Awesome writeup! I was about to do the same until i found your blog. Sadly i did not manage to decode one single packet from my ec3000 plugs so far… everything else seems to be ok but i just see the noise levels increasing as you described it also. I also followed your advice and tried to optimize the frequency setting using the sdr windows tool but still no success. Do you have any hints for me? How long do you run the ec3d_recv script on average until you see a frist output from one of the plugs?

    Like

    • update: i played around a litte with the frequencies and also changed the self.noise flag to -40. and bingo! packets are comming in 🙂

      next step for me would now be to write a script as you did to determine my optimal frequencies and the forward the data to emoncms. is your script already somewhere on github or simlilar? i do not care if the code is not clean.

      Like

  3. Hi! I have also found your blog and I have setup everything as you did. Unfortunately I had no luck so far to get any packages decoded. I see the transmissions in SDR# on my windows machine, but I could not find any frequency for the ec3k-script that matched. Do you have some more hints for me? Thanks in advance.

    Like

    • Hi Stefan,

      Are you using the optimized C capture program? I found this doesn’t always work reliably. If you have the capture program in your PATH please remove it and try the python based capture.py instead (ec3k will use ‘capture.py’ when there is no C-based ‘capture’).

      If this doesn’t help, what kind of output do you see? On which Raspberry Pi model are you running the ec3k software?

      Like

      • Hi! Thank you for your answer. I was using an original Raspberry Pi model. I followed you advice and disabled the C program. But without it I am getting a lot of OOOOs from GnuRadio. The SDR device is actually working fine, e,g, when I am using rtl_fm.
        I have now installed GnuRadio and ec3k on a Pi 2. But I am getting the same result, I am seeing only the line “Noise level: -17.8 db” (the level starts at about -40 and it ends at about -16 db or -17 db.
        I have tried adjusted the frequencies according to the waterfall analysis I was doing SDR# on my Windows machine. I have tried a lot of frequencies and also changed the self noise level according to the post before. But no luck so far.
        Although my background is electrical engineering, I am more an IT guy. Could it be that the noise is to much to detect the signals? The SDR# waterfall shows that my base noise level is about -30db to -25db.

        Like

  4. Hi,
    Now there are some debian packages available for gnuradio, in the jessie release. I’ve followed the first part of http://www.rs-online.com/designspark/electronics/eng/blog/taking-the-raspberry-pi-2-for-a-test-drive-with-gnu-radio-2 and it worked all pretty out-of-the-box.

    Next part is indeed finding the right frequencies. I’ve just played around, but my actions were not very research worthy. I did see that you have a script to find the right frequencies, shared with s4k-IO above. May I receive it as well?

    Keep up the good work!

    Like

    • Meanwhile I’ve found the best frequency for my setup. I’ts 868.185Mhz. Now I receive at least once per minute an update which I write to a file. Then I’ve written a munin-plugin which reads these files and so it generates graphs.

      My current ec3k_recv is here: http://pastebin.com/fTyedG85

      Like

  5. Hi,
    good news. I have changed my USB receiver to a E4000 model (from NooElec) and this works much better. The measurements of two sensors are now popping up. The hardware of my other receiver was obviously not good enough to receive a clean signal.
    I am using a Raspberry Pi 2 with the software setup that decramy is describing. I will now have to find out the frequencies of my others sensors.
    Next task will be to find out how to bring the data to emoncms.
    Thank you for your help!

    Like

    • Hi Stefan,

      Thanks for letting me know your experiences! Nice you could still find an E4000 tuner based SDR dongle as these seem to be getting pretty rare these days. Personally I have had bad experiences with Fitipower FC0013 in combination with ec3k and rtl_433. Although I must say I was able to manually decode the signal from the FC0013 radio using Audacity. My suspicion is that using different gain settings might improve things for the FC0013, however setting the gain factors for the different amplifiers from librtlsdr seems to be limited. As my experience with the cheapest and smallest Rafael Microelectronic, Inc R820t based tuners was great I’m currently only using these for my experiments.

      Like

  6. Thanks for the work and the article. Do you know if the Voltcraft Energy count 3000 measures the power consumption of computers correctly? I have the Voltcraft Energy monitor 3000 and that one displays active power, apparent power and cos phi.

    Like

    • Hi Jonas,

      Sorry for the late reply. The ec3k seems to measure power consumption quite well, also for lower power devices like Raspberry. Also it keeps track of total power consumption (not only active power). I do not (yet) have the tools to verify accuracy of the measurements, but I have not yet detected any obvious error readings.

      I should update this article some time as these days it has become easier to install stuff. If I only had enough time….

      I hope this helps.

      Like

  7. Hi, would it be possible to take a look at your scanner script? I have three hubs and the ec3k_recv script doesn’t receive any responses with the default configured frequency.

    Like

    • By the way: what type of SDR dongle are you using? Note that dongles using R820T and E4000 tuners typically work better than FC0013 based dongles for this piece of software. I don’t think there is anything really wrong with the FC0013, but it seems the gain control is not optimal for this kind of application

      Like

  8. Hi!
    Thanks for the great work and great step-by-step instruction!
    I’d like to use the ec3k for features like sending a what’s-app message when the washing machine is done (when power consumption goes down) in my openhab system, or starting hot water circulation when the dishwasher starts.
    Unfortunately I don’t get it working…
    I get the increasing dB-numbers from -35dB to -18dB, but no useful output.
    One key problem might be that I don’t really know what I’m doing when it comes to python…
    I tried to modify the frequency as described above, but I think I’m modifying the wrong file – I modified ec3k.py at line 275.
    Even if I completely delete ec3k.py and start ec3k_recv it gives me the same output.
    Do I need to compile anything? Or modify another file? How are the scripts linked to each other?

    Another problem could be that I did not use the recommended Raspbian image, but installed GNU radio on my pi according to this instruction:
    http://www.rs-online.com/designspark/electronics/deu/blog/taking-the-raspberry-pi-2-for-a-test-drive-with-gnu-radio-2

    And a 3rd source of problems could be that if I do lsbusb, it gives me Bus 001 Device 005: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T.
    Should be 2832 insted of 2838, right?
    of Course I used 2838 in /etc/udev/rules.d/20.rtlsdr.rules

    This is my device:
    http://www.ebay.de/itm/291640205336

    I’l be grateful for any ideas how to get this thing running…
    Best Regards,
    Christoph

    Like

    • I just uploaded my scanner script that might be useful for you. I belive this is similar to what Batilan also used and it will also create a graph similar to the one in this post. Please note that the script with the default settings will take a while to run since it scans the spectrum in small intervals. It works find on my Raspberry Pi using the Raspian GNU Radio packages (i.e. not the dedicated image).
      The script is available here: https://github.com/molobrakos/ec3kscan

      Like

Geef een reactie

Vul je gegevens in of klik op een icoon om in te loggen.

WordPress.com logo

Je reageert onder je WordPress.com account. Log uit / Bijwerken )

Twitter-afbeelding

Je reageert onder je Twitter account. Log uit / Bijwerken )

Facebook foto

Je reageert onder je Facebook account. Log uit / Bijwerken )

Google+ photo

Je reageert onder je Google+ account. Log uit / Bijwerken )

Verbinden met %s