Wrapping up MATLAB/LabView Problem

So here is some final thoughts on the problem of compiling MATLAB code into C shared libraries to use with LabView; before I forget about it all now that I decided to stop worrying about it:

- The first problem is that Matlab uses a custom data type called MxArray, and one needs to create a wrapper function that takes in standard data types (doubles, integers, etc.) and calls the appropriate Matlab compiled functions with the arguments passed as MxArrays.

- After creating the wrapper, you can re-compile it all to end up with a .dll and a .h file.

- These files can be used with the function library node sub-vi in LabView, and the appropriate functions should show up. However...

- The first function to call is one that loads the global MCR (Matlab Complie Runtime or something similar), which should only be stopped after one finishes using any Matlab libraries.

- Then one can call the set of initialization functions for each matlab function complied into the shared librarry. After initializing, one can call the excecuting function, and lastly the terminate function. The initalization/termination funciton basically create a new local MCR instance for each one.

- The last step is to call the global termination, to stop the global MCR.

- This all sounds fine, until you start implementing it. It is just a PITA to get it working.

PHP server setup status

I've been trying to get the server running with the energy data visualization code I've been working on. it has not yet been a success, but here's my progress so far:

I could not get the installed version of php working with apache, though i added the necessary directives. There was no module binary for php in the apache/modules directory, but I found one elsewhere (in the php installation dir). However, that module threw some error about "garbled module" when I tried to restart the apache httpd server (using apachectl).

Next plan: install a fresh version of php which will be able to compile the module from source. Downloaded php 5, but it didn't like apache's apxs library. Then I downloaded php 4, removed the currently installed version of php 4 (using RPM) and tried to build the module. It complained that if I didn't specify a path to the mysql directory, other apache modules might not be able to access the mysql database. This might be a problem, but it's one for another day. I compiled the module, and was able to restart apache and get a php file to load properly - yeah!

However, the database code did not work; it just threw an error about not being able to connect to localhost as daemon@localhost with no password. When I explicitly included the link_identifier in the mysql_select_db function, I instead got this error:
"Client does not support authentication protocol requested by server; consider upgrading MySQL client"

I suspect that the final solution may include removing the RPM versions of apache and mysql and compiling everything from scratch. Or move the project off of cee-landscaper and onto sensor2.andrew and see if the default installation is wprking any better over there...

Compiling Matlab code into Shared Libraries

It's been a long time since our last post. Many things have happened since, and even though we intended to use this blog as a progress report log, I am starting to realize that it will be more similar to a "Lessons Learned" kind of log, at least during the early stages.

Today I spent a few hours trying to get some .m files working in LabView by compiling them into .dll shared libraries, and importing them using the new Shared Library Import Wizard that came out on v8.2 of LabView. Read lots of forum threads, and spent some time staring at the command line prompt.

So far, I've managed to compile the code by using the following:

mcc -a neededmatfiles.mat -W lib:libraryName -T link:lib filestocomiple.m

This results in a .dll and some other necessary files.

It turns out, however, that the choice of compiler (which can be configured using "mbuild -setup" is an important one. The Lcc-win32 compiler that comes with Matlab doesn't seem to be doing the work, and I've read on some forums that the Visual Studio 7.0 compiler is our best bet. So now I'm looking for a computer with it, since the "free" student version (DreamSpark) is not so easy to get.

Update (06/06/08): I might have found the solution here:
http://forums.ni.com/attachments/ni/170/7347/1/dllmatlablabview.pdf

Google Alerts and Current Transducers

I have a Google Alert for many different things. One of them is set to look for electricity monitoring devices or news. Most of the time it brings up automotive related things, but many others it shows links to interesting webpages.

Today it brought up this not so good article "How To Find The Best Green Gadgets", which in itself has a link to the Efergy smart meter.

Looking at simple technologies like this, that are only measuring current, and pairing it up with the harmonic analysis we are starting to perform on the current waveforms, I start to believe that there is hope for doing good classification of appliances with this simple to acquire data.

Categorization algorithms, take 1

With data in hand (66 events over 20 minutes from one data set with RMS volts for normalization purposes) we started post-processing the data. This will convert the raw values (of power, reactive power, etc.) to more useful metrics (of delta power, delta reactive power, etc.).

Step one was to map the human-tagged events to more precise timestamps based on transitions in real power. We took the set of all events and selected the nearest timestamp with a change in real power of greater than 5 watts, provided it was less than 4 seconds away from the event. This did a nice job of locating the events at a more precise time relative to the actual change in power, since the light switch and LabView click are not quite synchronized.

Next we wanted the changes in real power at each one of these event times. The sample rate in this data set is 20 Hz (50 ms intervals) so it's not helpful to look at inter-sample changes (though the rate of change might be useful, this method is both susceptible to noise and also will fail to capture the full jump from off-power to on-power). Instead, we discovered (with a little trial-and-error) that averaging the power from all samples in the interval 4 seconds before the event and calling that the before-power, then subtracting the after-power averaged from the 4 seconds after the event gave reasonable intervals. It was able to absorb relatively short start-up transients, but the microwave has a transient with a large spike, then a more gradual descent down to the steady operating power. This resulted in a start-up power change of +300W but a turn-off power change of -90W; reliable for comparing among on-events, but not so good for pairing on-events and off-events.

While none of the appliances we have been testing so far have similar real power consumption, we noticed that the difference between the high and low states of the fan was about 10W, similar to the difference between the on and off states of the CF bulb. We hoped that using reactive power as a secondary axis would help to differentiate those two events, as it did for Hart (1992). However, when we applied the same transformations to the normalized reactive power metrics, we found little correlation between delta-Q values within events of a given appliance transition (eg. fan changing from high to low). In fact, for the incandescent lightbulb on-to-off transition we saw +3W twice and -12W once! Investigating this oddity (and remember that such a resistive load should effect no change in reactive power at all) we discovered that the -12W delta Q transition occurred during a microwave on-cycle. This produced about 40VAR fluctuation in reactive power, so the changes we were seeing were entirely due to the microwave noise, not the light turning off.

For harmonics we are starting to filter the extracted tones to exclude frequencies outside of +/-15Hz of the 1st, 3rd, and 5th harmonics. Then we will look at the normalized 3rd and 5th (e.g., the ratio of the Nth harmonic to the 1st, or fundamental frequency) and the changes to those values around the event times. So far it is clear that some of the harmonics drop out entirely during periods that are visually correlated with certain appliance events. Hopefully the harmonics will help us to filter out some of the noise as it has helped others to isolate continuously variable loads.

More begets Onzo

More Associates has worked with the UK Design Council on energy use visualizations, and done some damn cool stuff. They have recently spun off Onzo, which is about to commercialize a home appliance for the same purpose. 

We should give them a second look. And a call. They might have some hardware for us to play with.

EnerSure Installation Tips

After Ethan's last post explaining the details of the installation process, I thought it would be a good idea to summarize some of the lessons we've learned so far.
So here's a summary of what we learned from the new EnerSure installation on the 123 wing of Porter Hall:

• Besides physical space and easy access to the individual circuits in the electric panel, here are the other most obvious requirements for installation:


• Network connection (a cable from the EnerSure unit to a network outlet)


• Power supply (a power outlet for a three-prong adapter that powers the unit)


• Make note of the MAC address of the Digi Connect ME serial to ethernet adapter that comes in the processor module of the EnerSure.


• Make sure there are enough data gathering modules for all the circuits that will be measured from the panel.


• Configure the jumpers in the data gathering modules according to the phases in which the circuits are in the panel. Also set the jumpers that number the different data gathering modules accordingly.


• Do the last three steps before assembling all the modules together. Only after making sure it is all correct, you can put them together.


• Test the unit before mounting it. Run the latest version of TPConfig over the network, setup the unit via this software (ModBus address, CT type, circuit phases, etc).


• Run your logging daemon (in our case thermd) and make sure it works (also before mounting the unit and connecting the CTs). The latest version of thermd allows us to specify the ModBus address for the unit, which was a source of problems.


• Label all the CTs with numbers on both ends of the wire.


• Mount the EnerSure, connect all the cables, and power it up.


• Listo.

Bringing the 123 wing Enersure online

After a long installation process, the new Enersure electricity meter has finally connected to the circuit breaker panel in the Porter Hall 123 wing. This involved installing a new steel box over the breaker panel, connecting the two enclosures with conduit (inside the cinder block wall), mounting the Enersure in the new box, pulling CT cables through the conduit, wiring the voltage transformer into a 3-pole breaker (since it's a 3-phase panel), and installing a 110V outlet outside the box (for the Enersure's power supply). We estimate that this took over 100 hours of electrician labor: once the box was installed (in October) it took Joe--often working with Dave, his assistant--the better part of a week to get everything wired up.

One installation issue that might be re-addressed for the next panel is how the Enersure unit is connected to the back of the box. The Enersure is mounted on threaded posts: mostly male/female, but some f/f are available to reverse the "polarity" of the stack. In this way, it's possible to attach the top board with phillips-head screws and still have female thread on the bottom of the stack. For this installation, Joe mounted threaded posts to the back of the box, then secured two rails to the posts with fender washers and nuts. Before securing the rails to the posts, he screwed the Enersure to the rails from the back. It's now possible to remove the Enersure-on-rails assembly by removing four nuts, then unscrew the rails from the Enersure. However, the screws used to attach the Enersure to the rails were the wrong thread pitch, and I believe that they were originally installed by force--effectively re-tapping the threads in the bottom posts! In the future (if we use this mounting system again) we should be sure that there are enough of the correct screws to attach the Enersure without damaging any of the mounting hardware.

Once those pieces were all physically installed, we started wiring up the Enersure. This can be a bit of a challenge if one forgets a step before moving on, as each board must be physically configured before the next board is attached to the top of it. The MAC address is labeled on the ethernet port, but must be recorded before the next board is attached. Each metering board (the first/bottom board is the CPU and communication board) must have a jumper set to indicate its order in the stack, then jumpers for each channel to indicate the phase of the circuit they are measuring. Finally, the boards are all connected to a single ribbon cable. I think that the Enersure is implicitly grounded by the way it's connected to the box, which is grounded, but if that's not adequate we should add a ground wire to one of the mounting posts.

Then it was time to try communicating with the Enersure over the ethernet cable that had been run through the ceiling into Scott's office. Though the MAC address of the device had been registered with CMU's netreg system and it had been assigned a static IP address, the default DHCP settings on the Enersure were not adequate to get it working on the network. It turned out to be necessary to first connect to it with the TP Config (www.bitbox.co.uk/trendpoint), but first it's necessary to run the Digi Port software (which establishes a virtual serial port over a crossover ethernet cable, or through a router in our case). This allowed us to indicate that it should communicate via the ethernet connection, not the serial port. It also configured the board with the correct phase for each channel, as well as indicating which channels were coupled on 2- and 3-phase breakers. We did see some errors on the OEM calibration screen, indicating that (for most of the channels) there is a cct or wiring error; I need to record the exact wording, but I'm hoping that it's a matter of reversed CTs and we will also be able to detect that in the data where there are channels with >0 Watts but 0 amps.

Getting the ethernet card working on the network took still more steps. First I logged into the built-in server on the Enersure and changed the DHCP settings to use a static IP address instead. Since the subnet that it was assigned to is configured a little funny (subnet mask of 255.255.240.0), I guessed the router address wrong. I finally figured it out by spoofing the MAC address on my MacBook Pro, choosing to connect to the ethernet port (en0) using "DHCP with static IP" and then recording the router IP and subnet that it was assigned, finally transferring those to the Enersure. Note that this was a nasty mistake to make, because I wasn't able to connect to the Enersure again as easily once I had changed the IP address to the wrong one; I had to configure the router between the laptop and the Enersure to the router address I had entered on the Enersure, then change the laptop to be on that same (bogus) subnet.

Now that the Enersure is finally working on the campus network, I'm not sure it's possible to use TP Config to adjust the setting as easily; I wasn't able to get the Digi Port software to see the device. If necessary, I will have to restore the Enersure to using DHCP and re-connect it directly to the Windows laptop in order to play with those settings.

However, at this point I'm simply going to try to get the thermd config file to see the new Enersure and get some data in the database. From there we should be able to see which CTs need to be flipped and then the data collection can begin in earnest!

VisualWattch from the Interactive Institute

The Interactive Institute in Sweden is developing (as a concept or in practice is not clear to me) interfaces that promote energy-as-a-service. The brief description claims that the Wattch project

will "allow [users] to see, understand, monitor and control their domestic consumption." They have a short video (in Swedish, but easily understood) showing an energy monitor as a sort of EKG on an iPhone. These guys are hip, and they have been into this stuff for a while. I'm not sure how well it motivates users, but it looks good. They have a bunch of cool designs as part of their Static! project, and some of the designs--like the power-aware cord--look like they might actually go into production. Other work in a similar vein is happening through their Aware project, such as a light switch that looks more aesthetically pleasing in the "off" position.

Blueroof Research Cottage

Last Tuesday we (Ethan, Lucio and myself) visited the Blueroof Technologies model research cottage.

Our goal was to find out what the new house they're building is going to look like, specially in terms of the electric layout. We intend to deploy a general purpose data acquisition device to constantly monitor the current and voltage at the main feed of the house, and from there disaggregate the energy consumption of individual electrical appliances.

The fact that these houses have a massive deployment of sensors is attractive to us, because this will serve as a ground truth against which we can compare the effectiveness of our disaggregating algorithms. The scenario is similar to that of a blind man learning to see the world through his hands. Each time he makes a prediction of what the object he is holding is, he can rely on other people's judgment to improve his learning process.

IBACOS meeting

Mario and I met with Eric Helton from IBACOS to discuss possible collaboration. They are building a "lab house" in Pittsburgh (breaking ground around the end of the year) in which we might be able to install some electricity monitoring equipment. They are also building a lighting testbed in their lab which will have a number of different lighting fixtures, and we might be able to gather signatures from those fixtures.

CommonSense ToolKit

CSTK: open source tools for sensor communication, sensor data processing and visualization. Doesn't seem to have been updated much since 2004-2006, but Dr. Kristof Van Laerhoven appears to be the current manager of the code. Several papers describe its design intention, features, and examples. I've downloaded it and will see if I can get it running. One criticism of this toolbox is that it is missing support for sending commands to a node and receiving more complex data types from a sensor. Perhaps these can be added to the code base?