Trustwave sold a Man-in-the-Middle certificate

06:25 PM No comments

Trustwave sold a subordinate root certificate to a private company, enabling this company to issue unlimited SSL certificates for any domain/hostname (further informations at: http://blog.spiderlabs.com/2012/02/clarifying-the-trustwave-ca-policy-update.html).

I state this “selling the security” – therefor I can’t trust any more in certificates of Trustwave. I think every CA with these business practices should be removed from any products. Please vote for the removal of the Trustwave certificate from Mozilla products at: https://bugzilla.mozilla.org/show_bug.cgi?id=724929

Read OS X binary .plist with Java

11:30 AM No comments

With OS X 10.2 Apple introduced a more space-efficient binary plist-format. With OS X 10.4 this new binary format became the default format. The underlaying format is a simple XML-format (introduced with OS X 10.0), which was easy to read and parse from Java.

The new format must be parsed in a special way: either by using Apple’s Property List Editor, or by using the Apache Commons Configuration (which API is supporting the Property List format). In addition there is a new project of Daniel Dreibrodt, who is maintaining a JAVA project to read the binary plist format. It is licenced under GPL3 and can be found at:

http://code.google.com/p/plist/

Update: There is a new project on sourceforge that should be able to read and write the .plist-format. Up to now I haven’t tested it, but I want to provide the link here too: Property List Library

Categories: Java, Mac OS X Tags: , , , , , ,

Run Installer (.pkg or .mpkg) without user dialog (silentmode) from console on OS X

09:23 AM No comments

Very often, software for OS X is provided as .pkg or .mpkg bundles (the difference is, that .mpkg bundles are more customizable than the .pkg-installers). If you want to install them without clicking through the GUI-user-dialog, you can do this by running the installer from the console in silent mode:

sudo -S installer -verbose -pkg your_installer_file.pkg -target /

The installer will run without showing something on the GUI. You will have to enter your system-administrators password, if the installer needs the rights. You can (but shouldn’t) give the password with the command line (so you can install .pkg in an automated way):

echo <password> | sudo -S installer -verbose -pkg your_installer_file.pkg -target /

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Attention: Your password will be stored among other files in the bash_history!

You shouldn’t use this unless you really know what you are doing!

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

Cluster Filesystem for FreeBSD – GFS, OCFS2, …?

09:45 AM 6 comments

To make it short: There is absolutely NO real Cluster-Filesystem (like GFS or OCFS2) for FreeBSD at present. Also other projects for distributed filesystems like GlusterFS, PVFS or DRBD are not ported to FreeBSD, or the ports are very old.
Since I was in the need to have four identical data-filesystems (which have to be in sync just seconds after the upload), I wrote a little work-around for rsync, using the FreeBSD audit-system. The idea to use the audit-system for triggering the rsync I got from Luke Marsden, who is monitoring filesystem activity with audit_control and some python-scripts.

Install & Configure Event Audit Support

First of all, the audit_system must be activated and configured. The event-auditing is part of FreeBSD and has to be compiled into the kernel.

Add the following line to your kernel configuration:

options AUDIT

Then rebuild and reinstall your kernel as described in the FreeBSD Handbook

After this, add the following line to your /etc/rc.conf

auditd_enable=”YES”

The next step is, to configure the audit-system: Open the file /etc/security/audit_control and change the config to:

dir:/var/audit
flags:fc,fd,fw
minfree:20
naflags:lo
policy:cnt
filesz:0

That’s all for now. You can now start the audit-system by either calling

/etc/rc.d/auditd start

or by rebooting your system.

Installing & Running rsync

If rsync isn’t already installed on your system, you may do this by using the ports:

cd /usr/ports/net/rsync
make
make install

Installation of rsync should be no issue.

Next step is, to set an alternative path to your data-directory, using a symbolic link (I’ll explain later why).

ln -s /path/to/your/data/ /alternative_data_path/

Now we have to configure rsync to run as daemon. Therfor we create (or change) the config for rsync: /etc/rsyncd.conf

max connections = 5
log file = /var/log/rsync.log
timeout = 30

[shareName]
comment = Name of this “Rsync mount”
path = /alternative_data_path/
read only = no
list = yes
uid = validUser
gid = validGroup
hosts allow = ,
hosts deny = *

To start the rsync daemon, you have to call:

/usr/local/bin/rsync –config=/etc/rsyncd.conf –daemon

It is perhaps a good idea to monitor the rsyncd with the daemontools to make sure, the rsync-service is always available (you have then to run it with the –no-detach option).

The rsync-audit-script

#!/usr/bin/perl
##
# This software is published under the Apchae 2.0 licenses.
# You may obtain a copy of the License at
# 
#   http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
#   
#   Author: Erik Scholtz 
#   Web: http://blog.elitecoderz.net
###
# We are strict, cauz we are elitecoderz!
use strict;
use threads qw(yield);
use threads::shared;
use Thread::Semaphore;
 
# No caching
$|=1;
 
################
# Configuration
my $debug = 1;																			# 0/1 to enable logging to the console or disable
my $path = '/path/to/your/data/';				# Path to sync
my @cmds;																				# Syncer commands that should be executed
$cmds[0] = '/usr/local/bin/rsync -raz --progress --size-only /path/to/symboliclink/data/<!--target--> rsync:///shareName/<!--target-->';
$cmds[1] = '/usr/local/bin/rsync -raz --progress --size-only /path/to/symboliclink/data/<!--target--> rsync:///shareName/<!--target-->';
$cmds[2] = '/usr/local/bin/rsync -raz --progress --size-only /path/to/symboliclink/data/<!--target--> rsync:///shareName/<!--target-->';
 
###############################################################################
# DO NOT CHANGE ANYTHING BELOW THIS LINE, UNLESS YOU KNOW WHAT YOU ARE DOING! #
###############################################################################
 
###
# Set Threads yield
threads->yield();
 
# SetUp some thread-shared variables
my $commands :shared;
$commands = &share([]);
 
my $run :shared;
$run = &share({});
$run->{'status'} = 1;
 
my $sema :shared;
$sema = &share({});
 
# Local array where all syncer-threads are stored
my @threads;
 
# Create a thread for each syncer
my $maxid = -1;
for (my $i=0;$i<=$#cmds;$i++) {
	print "Starting syncer $i\n" if $debug;
	$sema->{$i} = Thread::Semaphore->new(0);
	my $syncer = threads->create('syncJob',$run,$sema,$i,$commands,$path,$cmds[$i],$debug);
	push(@threads,$syncer);
	$maxid = $i;
}
 
# Create the Checker thread, which cleanup the jobs and ensures the function of all syncers
$sema->{'checker'} = Thread::Semaphore->new(0);
my $syncer = threads->create('JobChecker',$run,$sema,$commands,$maxid,$debug);
push(@threads,$syncer);
 
# Create the audit thread
print "Starting audit\n" if $debug;
my $auditthread = threads->create('audit',$run,$sema,$commands,$path,$maxid,$debug);
print "Waiting for audi to terminatet\n" if $debug;
$auditthread->join();			# If the audit-thread gets joinable, we have to terminate everything
 
# Terminate all threads and cleanup 
$run->{'status'} = 0;
while ($#threads >=0 ) {
	my $worker = shift(@threads);
	print "Shutdown of syncer ...\n" if $debug;
	$worker->join();
}
print "Shutdown clean completed\n" if $debug;
exit(0);
 
########################################################################################################################################
########################################################################################################################################
 
####################################################################################################
# audit thread
sub audit {
	my $r = shift;
	my $sp = shift;
	my $c = shift;
	my $p = shift;
	my $m = shift;
	my $d = shift;
 
	print "       audit started ...\n" if $d;
	# open listener on the audit device
	open(STATUS, "/usr/sbin/praudit /dev/auditpipe |") || die "can't fork: $!";
	while (<STATUS>) {
		my $line = $_;
		last if ($line eq '' || $r->{'status'}<=0);		# Terminate if audit terminated
		if ($line =~ /path,$p(.+)/) {					# Check if the changed file is in the observed path
			my $file = $1;
			print "Change detected on file: $file\n" if $d;
			my $hash :shared;							# Create a command for the syncers
			$hash = &share({});
			$hash->{'file'} = $file;
			$hash->{'status'} = '';
			$hash->{'time'} = '';
			for (my $j=0;$j<=$m;$j++) {					# init job done charta
				$hash->{$j} = 'no';
			}
			if (1) {
				lock($c);
				push(@{$c},$hash);
				print "Added new job for $file\n" if $d;
			}
			for (my $j=0;$j<=$m;$j++) {					# wakeup syncers
				$sp->{$j}->up();
			}
			$sp->{'checker'}->up();
		}
	}
	close STATUS || die "audit not closed correctly: $! $?";	
	print "       audit terminated ...\n" if $d;
	return(0);
}
 
####################################################################################################
# syncer thread
sub syncJob {
	my $r = shift;
	my $sp = shift;
	my $id = shift;
	my $c = shift;
	my $p = shift;
	my $e = shift;
	my $d = shift;
 
	print "       syncer $id started ...\n" if $d;
 
	while ($r->{'status'}>0) {
		if ($#{$c}>=0) {													# if there are any jobs to be done
			for (my $j=0; $j<=$#{$c}; $j++) {
				next if ($c->[$j]->{$id} eq 'ok');							# if my job is already done skip this job and check next
				my $file = $c->[$j]->{'file'};
				if (-e $p.$file)  {											# check if the file is existing
					$c->[$j]->{$id} = 'working';							# mark this job as being worked on
					my $dif = 1;
					while ($dif>0) {										# check if the file is in upload and changes size within 1,5 secs
						print "Checking Filesize ...\n" if $d;
						my $ssize = -s $p.$file;
						sleep(1.5);
						my $eesize = -s $p.$file;
						$dif = $eesize - $ssize;
						print "Checking Filesize $ssize - $eesize = $dif\n" if $d;
					}
					my $cm = $e;
					$cm =~ s/<!--target-->/$file/g;
					system($cm);											# rsync to other server
				}
				lock($c);
				$c->[$j]->{$id} = 'ok';										# mark job as done for me
			}
		}
		$sp->{$id}->down();
	}
	print "       syncer $id terminated ...\n" if $d;
	return(0);
}
 
####################################################################################################
# checker thread that checks if all jobs are done
sub JobChecker {
	my $r = shift;
	my $sp = shift;
	my $c = shift;
	my $m = shift;
	my $d = shift;
 
	print "       checker started ...\n" if $d;
 
	while ($r->{'status'}>0) {
		while ($#{$c} >= 0 && $r->{'status'}>0) {
			print "    Checker loop ...\n" if $d;
			my $rem = 0;
			foreach my $job (@{$c}) {						# loop through all jobs
				my $mem = 'ok';
				for (my $j=0;$j<=$m;$j++) {					# check job done charta
					if ($job->{$j} eq 'no') {				# job not handled
						$mem = 'no' if ($mem ne 'working');	# job not handled (may never override a job in progress state)
					} elsif ($job->{$j} eq 'working') {		# job in progress (always overrides not handled)
						$mem = 'working';
					}
				}
				# Job not completed
				if ($mem eq 'no') {							
					if ($job->{'time'} eq '') {				# Set timestamp to know, how long this job is already waiting
						$job->{'time'} = time;
					} else {								# Job already got a timestamp
						my $watch = time - $job->{'time'};
						print "Job age: $watch\n" if $d;
						if (time - $job->{'time'} > 300) {	# Job has waited for more than 5 minutes. terminate program
							print "TIME FOR JOB EXCEEDED - shutting down syncer";
							$r->{'status'} = 0;
							for (my $j=0;$j<=$m;$j++) {		# wakeup syncers
								$sp->{$j}->up();
								$sp->{'checker'}->up();		# wakeup ourself
							}
						}
					}
				} elsif ($mem eq 'working') {				# job in progress - just actualize the timestamp
					$job->{'time'} = time;
				} else {
					$job->{'status'} = 'complete';			# job is completely done and is marked for being removed
					$rem = 1;
				}
			}
			# Job to remove available
			if ($rem > 0) {
				lock($c);									# lock the command-queue
				my @arr;
				for (my $j=0;$j<=$#{$c};$j++) {				# store all not handled jobs / drop completed jobs
					my $ex = shift(@{$c});
					if ($ex->{'status'} ne 'complete') {
						push(@arr,$ex);
					}
				}
				for (my $j=0;$j<=$#arr;$j++) {				# put all stored (not finished) jobs back into the command queue
					push(@{$c},$arr[$j]);
				}
			}
			print "    Checker reloop ...\n" if $d;
			sleep(1);
		}
		print "    Checker sleeping (".$#{$c}.")...\n" if $d;
		$sp->{'checker'}->down();
	}
 
	print "       checker terminated ...\n" if $d;
	return(0);
}

This script does the whole magic: It listens via the audit-system for files changed or added and then uses rsync to sync the file to the other systems. And here we come to the part why we need a symbolic link to the data directory: when the script uses rsync to copy a file to a second system, the audit-system of this second system will notify the script there about this change. So the script on the second system would start to copy the file back to the first system and so on. So if you do not use a symbolic link for the rsync, you will create an endless loop of copy and recopy-processes!

Installation and configuration of the script is quite easy

Copy this script to each system that should be kept in sync with the others. I recommend to observe this script via daemontools too. Then edit the script on each system as shown below:

$debug can be set to 0 (for no debug output) or 1 (for debugging output).
$path should be set to the physical path of your data.

For each system that should be kept in sync add the following line. Please keep in mind to increase the number in the square-brakets ($cmds["number"]) by 1 in each line:

$cmds[0] = ‘/usr/local/bin/rsync -raz –progress –size-only /path/to/symboliclink/data/ rsync:///shareName/‘;

Some last important informations:

Before changing anything on your system, make sure you have a complete backup of your system! The usage of this script and howto is at your very own risk. So if you suffer any data-losses by using this howto or the script you can not hold me responsible for this.

To get close to a “realtime sync”, the script starts an own thread for each volume to keep in sync. So you need to have a perl-installation that is thread-enabled.

org.mortbay.thread.Timeout. exception on new GWT webapplication with instantiations gwt designer or windowbuilder pro

10:30 AM 16 comments

When creating a new WebApplication with GWT SDK 2.0.3 and App Engine SDK 1.3.3 in eclipse using the Instantiations WindowBuilder Pro or GWT Designer, the following error occors when trying to launch the “GWT application” (google calls this “Development Mode”).

Exception in thread “main” java.lang.NoSuchMethodError: org.mortbay.thread.Timeout.<init>(Ljava/lang/Object;)V
at org.mortbay.io.nio.SelectorManager$SelectSet.<init>(SelectorManager.java:306)
at org.mortbay.io.nio.SelectorManager.doStart(SelectorManager.java:223)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:39)
at org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:303)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:39)
at org.mortbay.jetty.Server.doStart(Server.java:233)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:39)
at com.google.gwt.dev.shell.jetty.JettyLauncher.start(JettyLauncher.java:543)
at com.google.gwt.dev.DevMode.doStartUpServer(DevMode.java:421)
at com.google.gwt.dev.DevModeBase.startUp(DevModeBase.java:1035)
at com.google.gwt.dev.DevModeBase.run(DevModeBase.java:783)
at com.google.gwt.dev.DevMode.main(DevMode.java:275)

I could not figure out what is happening and why this exception is raised, but I found a workaround for this:

Instead of using the App Engine SDK 1.3.3 on creation of the new Web Application Project, select the App Engine SDK 1.3.1. Create the new project, then right-click the project-folder and select the Properties. In Google>App Engine you can now switch from App Engine 1.3.1 to App Engine 1.3.3 and your project will run cleanly under the new engine.

select new appengine in the properties

select new appengine in the properties



Categories: eclipse, GWT Tags: , , ,