SAWA – Real-Time Sensor Platform

 

Experience: http://linux-utils.blogspot.com/2011/01/arduino-remote-control-outlet.html

Platform features, summary

Web application

- create networks
- create sensors
- create actuators
- add members to a network
- call functions or commands in actuators
- define policies
- define rules
- define notifications
- browse networks and sensors in Google Maps
- Visualize data charts, in average or raw mode
- delete and unsubscibe XMPP nodes
- check event log
- export sensor data in multiple formats
- publish sensor data via HTTP POST
- see users in system (admin users)
- edit system configuration online (admin users)
- check services status (admin users)
- add images to network and sensor galleries
- add properties to sensors
- select chart type

sox_tools

sox_tools is a utility command line XMPP client built for interacting with an XMPP server. Most of its functions is related to PubSub protocol (XEP 0060). The tool help explain each option as shown.

Available options:

	--user			Select user from config

	listen			Listens for pubsub events
	commands         	Listens for commands and call apropriate action in actuators
	publish			Publish to a pubsub node
	publish_cmd		Publish a command to a pubsub node
	last_item	    	Get last item from a pubsub node

	create			Creates a pubsub node
	subscribe	    	Subscribe to a pubsub node
	unsubscribe		Unsubscribe from a pubsub node
	add_member		Add a member to a pubsub node
 	subscriptions		List pubsub subscriptions
	affiliations		Lists pubsub affiliations
	delete			Deletes a pubsub node

	create_user		Creates a XMPP user
	delete_user		Deletes a XMPP user

Screenshot of database

Wine monitoring

“To gain scientific understanding into the vinification process of Madeira wine, we intend to use sensors to study various parameters of the process (e.g. Ethanol, pH, total acid, volatile acid, density, fixed acidity, ash,), as well as actuators to control the environment (e.g. temperature, humidity, etc.)”

Search for sensors

One practical use case was wine monitoring. Due to our limited knowledge in electronics, we looked towards sensors with easy interfacing, to be able to connect in Arduino. Here is a list of  measurable parameters that can be interesting for wine analyses, however, many of these parameters are manually analysed in the lab, using heavy equipment.

After some search, we found some experiences using gas sensors and other sensor, which helped us to make a decision to order or not.

The main criteria for selecting sensor for this experience was :

- easy interfacing
- no maintenance
- affordable

Measure in liquid	Order	Use
pH	Yes	(+) Seems reasonable
Temperature	Yes	(+) Reasonable
Color	No	(-) Expensive(-) Specific HW and SW
Dissolved Oxygen	No	(-) Expensive(-) Specific HW and SW(-) Maintenance
Volatile acidity	No	(-) No sensor
Total acidity	No	(-) No sensor
Oxidation	No	(-) No sensor
Ethanol	No	(-) Expensive(-) Specific HW and SW(-) Maintenance
SO2	No	(-) No sensor

Measure in gas	Order	Use
CO	No	(+) Reasonable(-) Less important in wine
CO2	Yes	(+) Reasonable
Alcohol	Yes	(+) Reasonable
Relative Humidity	Yes	(+) Reasonable

pH – pH probe from Vernier

Temperature – Temperature sensor with steel head, from Seeed Depot

CO2 – Parallax CO2 module with mg811 c02 gas sensor, from Parallax

Alcohol – MQ3 gas sensor, from SparkFun

RH – Relative Humidity sensor, from SparkFun

Testing sensors

After the arrive of ordered sensors, we started to test them.

Building probe

The probe construction was the most difficult part to do, because of excess of wiring and lack of space inside the probe. Three attempts were made to build a stable probe (float with equilibrium in the wine), but it was very difficulty to wire and use all sensors along with pH probe in the same module. And in the end the probe was not floating :’(.

Building probe with success

After the previous experience we came with a more simple solution, use 2 modules, one for pH and temperature, and the other for gas sensors, the last attached to the top of the wine container.  The upper module has 2 separated parts, the top one contains the micro-controller (Arduino) , DC jack and RF transmitter an receiver. The bottom part contains the c02, alcohol and relative humidity sensors.

Installation

We have the probe uploading sensor data every minute, (RF 433MHz module). The sensor gateway (Arduino +  ethernet shield + RF receiver module), receives this information and sends via HTTP POST to server, which then is published using XMPP protocol.

EOF =)

Last update to system

New updates to system, and to sawa wiki

WEB
removed flexigrid
added better notifications
added raw chart
changed all files to the right names (networks, sensors, etc)
better way to upload sensor data via HTTP
added sensor gallery
options to select data parameters on exporting data
edit configuration
services status
user list

DATABASE
added support for messages, events, sensor properties, custom rules and policy groups

SERVICES
added support for date rules and database history rules
added events logging support

Storage statistics, update rates

We have 327.735 records in database, with total size of 27.1 Megabits, which gives an average of 10.84 bytes per record in database.

3552051.2 (bytes)/327735 (records)  = 10.84bytes per record

So when selecting the upload rate for sensors, we should keep in mind this values.

Time to fit one GigaByte
Interval	Records	days		years
Second	1	1146.65		3.14
	10	114.66		0.31
	100	11.47		0.03
	1000	1.15		0
Minute	1	68798.82		188.49
	10	6879.88		18.85
	100	687.99		1.88
	1000	68.8		0.19
Hour	1	4127929.15		11309.39
	10	412792.92		1130.94
	100	41279.29		113.09
	1000	4127.93		11.31

If you have a sensor with the upload rate at 1 seconds, in 3.14 years the sensor data will take 1GB of space

Reliability/Performance tests

in the past week i have been testing sensors for the wine monitoring use case, since our pH probe broke, i am waiting for a new one to arrive and a humidity sensor. This probe will have: c02, alcohol, RH (relative humidity), temperature and pH sensors. Maybe in next week i will create a post about the probe. Until then, i have returned to my report, starting with tests. For now i completed Reliability and Performance tests for publishing data and notifications.

Results:

Since the fail rate was to high in POST, i found the problem was related to C command line tool “sawa_publish”, which publishes the POST data to XMPP.

pthread_mutex_lock.c:87: __pthread_mutex_lock: Assertion `mutex->__data.__owner == 0′ failed.

After looking at google, i did not find any usefull/clear solution, so i decided to simplify the publish function and repeat the same process when fails.

- added a new publish function

—- removed variable and data arrays

—- added repeat process when fails, up to 3 times

	function publish2()
	{

		$this->form_validation->set_rules('USERNAME', '', 'trim|required');
		$this->form_validation->set_rules('USERPASSWORD', '', 'trim|required');
		$this->form_validation->set_rules('NODE', '', 'trim|required');
		$this->form_validation->set_rules('VARIABLEID', '', 'trim|required|numeric');
		$this->form_validation->set_rules('DATA', '', 'trim|required');

		if($this->form_validation->run()){

			$res=0;
			$count=0;

			while(!$res && $count <=2){
				$res=$this->sox_wrapper->publish(
								$this->input->post('USERNAME'),
								$this->input->post('USERPASSWORD'),
								$this->input->post('NODE'),
								-1,
								-1,
								$this->input->post('VARIABLEID'),
								$this->input->post('DATA')
				);
				$count++;
			}

			if($res)
				echo "OK\n";
			else
				echo "FAIL\n";

		}else{
			echo "FAIL\n";
		}

	}

Bash script example to upload data via HTTP POST (useful for resource limited devices)

#!/bin/bash

# settings
USERNAME=user123@hci.uma.pt/scripts
USERPASSWORD=pass123
NODE=testnode
VARIABLEID=23

URL="http://dev.hci.uma.pt/sawa/xmpp/publish2"
DATA=1

for ((  i = 1 ;  i <= 10;  i++  ))
do
  DATA=$i;
  curl -X POST --silent --data-binary "USERNAME=$USERNAME&USERPASSWORD=$USERPASSWORD&VARIABLEID=$VARIABLEID&NODE=$NODE&DATA=$DATA" $URL
done

Updated table with new code changes

Intelligent Mail Box

What about you being  notified when you have mail ? by email or sms, or other type…

I set up this intelligent mail box with 2 sensors and one transmitter.

Magnetic door switch $2.00

Mail sensor ( i invented it , works 1000 times better than distance sensor, or other you may use).

RF transmitter Kit for $4.90, you only need the transmitter for this.

Arduino/Compatible from $23

Comparing the size of RF transmitter module with an old Nokia

To connect any switch,  im using these “2pin plugable terminal module” sold by Seeed Depot, which is very useful.

Here is the mail sensor, basically its a simple switch, with two separated metal parts, they are very flexible in order to make contact when a new mail touches.

Mail box, red would be better

I pasted one part of magnetic door sensor to the box, and the magnet part on lock, so when we turn the key to open door, this magnet will go up.

Adding the box with arduino, and connected sensors.

Here we can see mail sensor attached to mail box and a external power supply.

Programming arduino..

Creating a new policy one server, this policy has only one filter, when new_mail value equals 1 then the action is triggered, which in this case was a SMS notification.

New mail arrived!, check out the video

Video
Video