Whitemice Consulting

Whither Samba?

2012-01-17T16:10:00.000-05:00

"What's up with Samba?" .... "Is Samba4 for real?" ... "Is Samba4 ever going to be released?" These questions have been common for several years now. Now we have answers thanks to this excellent article/interview on LWN.

Streaming the BBC World Service

2012-01-03T07:37:00.001-05:00

Every leap year the american public radio service becomes obsessed with providing non-stop glossy and shallow coverage of the various presidential candidates, campaigns, and caucases. They report endlessly on various polling results [even though the validity of such polls is in question according to credible statisticians and demographers]. Fortunately there is the BBC World Service which offers an alternative for english speaking listeners exhausted by caucus mania. The appropriate links are not easy to find - but they are there. The MP3 link under "World Service English (Internet Schedule)" works perfectly when added to Banshee's radio playlist.

Querying Connectivity

2011-12-12T07:00:00.000-05:00

You're application almost always needs to know if there is a working network connection. This is typically handled by placing the connection attempt in a try...catch block. That works, but can be slow, and it means the UI can't really adapt to the level of current connectivity. A much better solution is to query the NetworkManager [used by every mainstream distribution] via the System D-Bus for the current connectivity. This method is used by many applications from GNOME's Evolution to Mozilla's Firefox - but it doesn't seem to get much press coverage. So here is a simple example to query connectivity via Python [assuming NetworkManager 0.9 or later]:

#!/usr/bin/env python
import dbus

NM_BUS_NAME       = 'org.freedesktop.NetworkManager'
NM_OBJECT_PATH    = '/org/freedesktop/NetworkManager'
NM_INTERFACE_NAME = 'org.freedesktop.NetworkManager'
NM_STATE_INDEX = {  0: 'Unknown',
                   10: 'Asleep', 
                   20: 'Disconnected',
                   30: 'Disconnecting',
                   40: 'Connecting',
                   50: 'Connected (Local)',
                   60: 'Connected (Site)',
                   70: 'Connected (Global)' }

if __name__ == "__main__":
    bus = dbus.SystemBus()
    manager   = bus.get_object(NM_BUS_NAME, NM_OBJECT_PATH)
    interface = dbus.Interface(manager, NM_INTERFACE_NAME)

    state = interface.state()
    if state in NM_STATE_INDEX:
        print('Current Network State: {0}'.format(NM_STATE_INDEX[state]))
    else:
        print('Network Manager state not recognized.')

FYI: if you search the interwebz for the NetworkManager API specification ... every search engine will send you to the wrong place; either just wrong or to the documentation of an older version of the API. The current API specification is here.

GNOME3 Journal Extension

2011-12-12T06:12:00.000-05:00

Now that's what I'm talking about! A new extension just showed up on extensions.gnome.org that adds a "Journal" tab to the already awesome GNOME3 overview. It integrates with Zeitgeist to provide access to recently or heavily used categories of items - sort of like "Recent" but all grown up and with college smarts. And installing it is as easy as clicking "On" [assuming you have Zeitgeist already installed].

Journal tab in Overview

A very handy addition that adds to the same concept provided by the gnome-activity-journal [which is packaged for openSUSE, BTW].

Really? A SHMMAX of 36MB

2011-12-10T09:20:00.001-05:00

I was running some tests on OpenGroupware Coils on my new HP workstation - and PostgresSQL seemed to be huffing-and-puffing like an overweight guy trying to run up stairs. Huh. What is the first thing a PostgreSQL administraor always checks? The shared_buffers parameter [in /var/lib/pgsql/data/postgresql.conf]; the default package typically sets this value to some absurdly small value resulting in dreadful performance. This default value is the principal reason the absurd notion that "MySQL is faster than PostgreSQL" got traction. Sure enough - the default value is 24MB! Yikes. So I changed that setting to a more reasonable 512MB.

shared_buffers = 512MB

Then I tried to restart PostgreSQL, and it flopped.

$ service postgresql start
redirecting to systemctl
Job failed. See system logs and 'systemctl status' for details.

Looking in /var/log/messages I see:

Dec 9 19:27:23 workstation postgresql[7288]: Starting PostgreSQL2011-12-09 19:27:23 EST FATAL: could not create shared memory segment: Invalid argument
Dec 9 19:27:23 workstation postgresql[7288]: 2011-12-09 19:27:23 EST DETAIL: Failed system call was shmget(key=5432001, size=76685312, 03600).

Huh. I remember an error message like that from the RedHat 6.x & 7.x days when the kernel's default settings regarding System V IPC resources where really low. Back then you possibly had to go look at defines in header files for these values. Fortunately we now have sysctl. And what does sysctl tell us?

$ sysctl kernel.shmmax
kernel.shmmax = 33554432

What?!?! That value is in bytes - so the limit is ~32MB?! This is on a 64-bit installation with 8GB of RAM. That makes no sense at all. Time to change that to something reasonable:

$ sysctl -w kernel.shmmax=1000000000

And now "service postgresql start" succeeds. The ipcs command which reports System V IPC resources allocated in the system shows that the PostgreSQL engine has allocated the expected buffer pool:

$ ipcs
------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status      
0x0052e2c1 1212416    postgres   600        572383232  4

Once you change a sysctl setting in a helpful way the change has to be added to the /etc/sysctl.d/ so that it gets re-applied when the machine reboots; otherwise our PostgreSQL instance is going to fail to start next time. Create a file in that directory with a name such as postgresql.conf containing a single line of -

kernel.shmmax = 1000000000

The files in /etc/sysctl.d/ are processed after distribution and package defaults are applied. It is also possible to edit /etc/sysctl.conf; however manual changes to that file may get overwritten by system management tools. So use the /etc/sysctl.d/ strategy to override distribution defaults.
Now, back to work.

"Secret" extensions.gnome.org links

2011-12-09T06:12:00.000-05:00

The new extensions.gnome.org is a fabulous idea and well-executed concept. But one issue is that is it isn't easy to visualize what is new; and the content of the site is growing rapidly. As it turns out there is a secret link to sort extensions by the order they were added: https://extensions.gnome.org/?sort=recent For all the RSS user's out there the site also provides an RSS feed: https://extensions.gnome.org/rss/ Enjoy!

Note: Seriously people - this site is barely two weeks old. Have some patience. It seems doubtless that the obviously inspired developers will add the "obviously" needed features in due time

TortoiseHg Packages

2011-12-07T07:11:00.000-05:00

It turns out some kind fellow has packaged TortoiseHg for openSUSE 12.1; you can find the packages in the home:tzotsos:vcs repo. The repo provides not just the Tortoise GUI but the nautilus extensions as well. The only item the packages seem not to provide is a .desktop file for starting the thg application - without a .desktop file GNOME3 doesn't know the application exists. You can either create the required file using alacarte or by hand; if by hand open the [new] file at ~/.local/share/applications/tortoise.desktop and paste in:

#!/usr/bin/env xdg-open

[Desktop Entry]
Version=1.0
Type=Application
Terminal=false
Icon[en_US]=/usr/share/pixmaps/tortoisehg/icons/thg_logo_92x50.png
Exec=/usr/bin/thg
Name[en_US]=Tortoise Hg
Comment[en_US]=Mercurial Client
Name=Tortoise Hg
Comment=Mercurial Client
Icon=/usr/share/pixmaps/tortoisehg/icons/thg_logo_92x50.png

Now GNOME3 will know about the thg application and you can launch it via the application search as well as add it to your launcher.

Enabling the RabbitMQ Management Plugin

2011-12-05T07:39:00.001-05:00

Prior to 2.7.x version of RabbitMQ it was necessary to manually install the plug-ins that provided the management interface [as well as their dependencies]. Now in the 2.7.x series the management interface plug-in and related dependencies are included - but not enabled by default. The management plug-in must be toggled into the enabled state using the new rabbitmq-plugins command. Enabling a plug-in will automatically enable any other plug-ins that the specified plug-in depends on Whenever you enable or disable a plug-in you must restart the sever.
If you have a brand new 2.7.x instance installed, turn on the plug-in with:

service rabbitmq-server stop
rabbitmq-plugins enable rabbitmq_management
service rabbitmq-server restart

When you performed the rabbitmq-plugins command you should have seen the following output:

The following plugins have been enabled:
mochiweb
webmachine
rabbitmq_mochiweb
amqp_client
rabbitmq_management_agent
rabbitmq_management

You management interface at TCP/55672 should be available. The initial login and password are "guest" and "guest". You want to change those.

Idjit's Guide To Installing RabbitMQ on openSUSE 12.1

2011-12-03T17:21:00.003-05:00

The RabbitMQ team provides a generic SUSE RPM which works on openSUSE 11.x, openSUSE 12.1, and I presume on the pay-to-play versions of SuSE Enterprise Server. About the only real dependency for RabbitMQ is the erlang platform which is packages in the erlang language repo. So the only real trick is getting the RabbitMQ package itself [from this page]. Then install and start is as simple as:

zypper in http://download.opensuse.org/repositories/devel:/languages:/erlang/openSUSE_12.1 erlang
zypper in erlang
wget http://www.rabbitmq.com/releases/rabbitmq-server/v2.7.0/rabbitmq-server-2.7.0-1.suse.noarch.rpm
rpm -Uvh rabbitmq-server-2.7.0-1.suse.noarch.rpm
service rabbitmq-server start

Now you probably want to do some configuration and provisioning using the rabbitmqctl command; but your RabbitMQ instance is up and running.

Using gedit to make a list of values into a set.

2011-12-01T09:00:00.000-05:00

gedit is awesome; the flexibility of the tool continues to impress me. One problem I'm frequently faced with is a list of id values from some query, or utility, or e-mail message... and I want to do something with them. So, for example I have:

10731
10732
10733
10734
10735
10736
10737
10738
10739

but what I need is those id values as a sequence such as for use in an SQL IN expression or to assign to a Python set or list. What I want is:

(10731,'10732', '10733', '10734', '10735', '10736', '10737',
'10738', '10739')

Reformatting a few numbers by hand isn't too hard - but what if I have a list of hundreds of id values? The answer, of course, is provided by gedit. Under Tools -> Manage External Tools the user can build filters that can be applied to documents and have the results returned to gedit. If I create a new external tool that accepts as input the "Current document" and as output has "Replace current document" then gedit will replace the contents of the current document with the results of the filter [pretty obvious; and if it doesn't work I can always Ctrl-Z]. The body of the filter can be any script - a Python script is perfectly valid. Like this hack:

#!/usr/bin/env python
import sys

iteration = 0
line_length = 0
text = sys.stdin.readline()
while (text !=  ''):
  text = text.strip()
  if (len(text) > 0):
    if (iteration == 0):
      sys.stdout.write('(')
    else:
      sys.stdout.write(', ') 
    if (line_length > 74):
      sys.stdout.write('\n ')
      line_length = 0
    if (len(text) > 0):
      sys.stdout.write('\'{0}\''.format(text))
    line_length = line_length + len(text) + 4
    iteration = iteration + 1
  text = sys.stdin.readline()
sys.stdout.write(')')  
sys.stdout.flush()

The current document becomes the standard-input for the script and the standard-output of the script will replace the current document. The above hack reads in a list of lines and returns them as a set enumeration nicely wrapped to 80 characters per line. External tools are saved under names; for this one I saved it as "IN-Clause-Filter".
Now that I've setup the external tool every time I paste a list of id values into gedit I can simply select Tools -> External Tools -> IN-Clause-Filter and my list is instantly turned into a set enumeration.

Samba-VirusFilter 0.1.3 Released

2011-11-28T05:15:00.002-05:00

Samba-VirusFilter (the heir to samba-vscan) version 0.1.3 was released a few days ago [download]. Samba-VirusFilter supports ClamAV, F-Secure, and Sophos. A sample ClamAV configuration snippet for your smb.conf looks like:

vfs objects = svf-clamav
svf-clamav:scan on open = yes | no
svf-clamav:scan on close = yes | no
svf-clamav:max file size = (bytes, default 10^8)
svf-clamav:min file size = (bytes, default 10)
svf-clamav:infected file action = nothing | quarantine | delete
svf-clamav:quarantine directory = directory
svf-clamav:quarantine prefix = string
svf-clamav:infected file command = command
svf-clamav:scan error command = command

If you use a quarantine directory don't forget to make sure the Samba daemon will always have sufficient access to put files in there.

Overlooked Content From 2011

2011-11-27T09:59:00.002-05:00

Yes, it is almost 2012.
Looking back for overlooked content I'm compelled to mention the 2011 annual "LDAPCon" conference focused on directory services and LDAP; the organizers/hosts have made available some excellent videos and papers [index page]. These are an unrivaled source of information on the current state and future of directory services.
The same is also true of the annual SambaXP conference in relation to all things Samba and CIFS [2011 content index].
Podcast content in general tends to be lite and aimless fare; those sources provide some meat for the meal.

All those SQLite databases...

2011-11-27T09:11:00.000-05:00

Many current application use the SQLite database for tracking information; this includes F-Spot, Banshee, Hamster, Evolution, and others. Even the WebKit component uses SQLite [you might be surprised to discover ~/.local/share/webkit/databases]. It is wonderfully efficient that there is one common local data storage technique all these applications can use, especially since it is one that is managable using a universally known dialect [SQL]. But there is a dark-side to SQLite. Much like old Dbase databases it needs to be vacuumed. And how reliably are all those applications providing their little databases with the required affection? Also, do you trust those lazy developers to have dealt with the condition of a corrupted database? If an application hangs, or is slow, or doesn't open... maybe that little database is corrupted?

Aside: As a system administrator for almost two decades I do not trust developers. They still put error messages in applications like "File not found!". Argh!

On the other hand SQLite provides a handy means of performing an integrity check on databases - the "PRAGMA integrity_check" command. I've watched a few of these little databases and discovered that (a) they aren't often all that little, and (b) manually performing a VACUUM may dramatically reduce their on-disk size. Both these facts indicate that developers are lazy and should not be trusted.

Note: in at least one of these cases the application has subsequently been improved. Developers do respond rather quickly when offered a blend of compliments spiced with bug reports. No, I'm not going to name offending applications as that is too easily used as fodder by nattering nabobs. And even the laziest Open Source developer is working harder than their proprietary brothers.

In light of this situation my solution is a hack - a Python script [download] that crawls around looking for SQLite databases. First the script attempts to open the database in exclusive mode, then it performs an integrity check, and if that succeeds it performs a vacuum operation. Currently it looks for databases in "~/.local/share" [where it will find databases managed by application appropriately following the XDG specification], "~/.cache", "~/.pki", "~/.rcc", and "~/.config".
Download the script and run it. Worst thing that happens is that it accomplishes nothing. On the other hand it might recover some disk space, improve application performance, or reveal a busted database.

gnome-tweak-tool

2011-11-25T08:54:00.000-05:00

GNOME3 simplified many things. In the process some settings and preferences got removed from the primary user interface. A side-effect of that is a fair number of BLOG posts like "Dude where’s my settings?". Many of these BLOG posts are informative; they explain how to get to the preference value via dconf either by the command line gsettings tool or the GUI dconf-editor [which replaces GNOME2's gconf-editor]. Both of those are good methods. Knowing how to use gsettings in case of an epic-fail situation is a very useful skill to have. On the other hand - most of the settings discussed in these posts can be easily tweaked using the appropriately named gnome-tweak-tool. gnome-tweak-tool provides a friendly GUI to a variety of maybe-you-shouldn't-mess-with-this-but-here-we-are kind of preferences. The gnome-tweak-tool package is available in the standard repositories for openSUSE 12.1

GAJ, Zeitgeist, & openSUSE 12.1

2011-11-25T06:57:00.000-05:00

In openSUSE 12.1 the GNOME Activity Journal and Zeitgeist data hub are only a package install away -

zypper in gnome-activity-journal

Now the GNOME Activity Journal is available; an excellent productivity tool. Hopefully more data providers will appear soon.

Ctrl-Alt-Shift-V ... Pasting Happiness

2011-11-25T06:25:00.000-05:00

If you cut or copy text from an application [especially a web browser] and then paste it into LibreOffice what you often get is formatted text, or at least some approximation of the text's original formatting. This is awful. When using LibreOffice appropriately all formatting is managed via the excellent support for styles. All you want is the text - and nothing about the text. I've used the brute force solution in the past of bouncing my cut-n-paste through gedit. Until I discovered Ctrl-Alt-Shift-V. Yes, all four keys at once. Ctrl-Alt-Shift-V is un-formatted paste [it pastes nothing but the text]. Pasting happiness!

TortoiseHg & openSUSE 12.1

2011-11-23T06:59:00.000-05:00

TortoiseHg is an excellent GUI for working with code from a Mercurial [aka Hg] repository. You can just download the source and run "./thg". The previous version of TortoiseHg used PyGTK and pretty much just-worked. Recently they switched to using Qt4. Only once you install Qt4 support for Python the GUI still doesn't run - it fails with an error:

ImportError: No module named Qsci

What is "Qsci"? "Qsci" is short hand for Qscintilla which is a Qt port of the Scintilla editing component. This is provided to Python as the PyQt4.Qsci module. This module is installed separately from the main Qt4 module in a package called "python-qscintilla" [notice a pattern where this feature of Qt4 is referred to by at least three names?]. Once you've run down this quirky daisy-chain of names it is just:

zypper in python-qscintilla

and now thg runs.

Converting M4B's to MP3

2011-11-01T18:15:00.002-04:00

I ended up with some M4B audio files; these are "MPEG v4 system, iTunes AAC-LC" files. In order to reliably manage these files along with every other audio file [all of which are MP3] the simplest solution is just to convert them to MP3. In order to accomplish that I dumped them back out to WAV using mplayer and re-encoded them to MP3 using lame. Both lame and mplayer are available for openSUSE from the PacMan repositories, so you can easily install them via zypper.

mplayer "filename.m4b" -nojoystick -ao pcm:file=tmp.wav
lame -b 128 -q1 tmp.wav tmp.mp3

The "-nojoystick" option for mplayer isn't required but it prevents mountains of output about mplayer being unable to read the joystick device [most likely due to the fact that I don't have a joystick]. I left the bit-rate for lame at 128 since there is no point in re-encoding a file at a higher bit-rate than the originally encoded file - these files are mono-channel human speech not high-fidelity audio.

"samba-vscan" Is Dead, Long Live "samba-virusfilter"!

2011-11-01T05:15:00.001-04:00

Noticed an interesting message on the Samba-Technical list today. The Samba VFS module "samba-vscan" which has long been used to build integrated malware detection into Samba is no longer supported for the 3.6.x series. SATOH Fumiyasu has been patching samba-vfs for 3.3 and up through 3.5; but with 3.6.1 he has created a new module named samba-virusfilter. His message is here.

Where and what is /var/run/named?

2011-10-31T13:05:00.000-04:00

# service named start
Starting name server BIND checkproc: Can not stat /var/run/named/named.pid: Too many levels of symbolic links
- Warning: /var/run/named/named.pid exists! start_daemon: Can not stat /var/run/named/named.pid: Too many levels of symbolic links
done

Eh? Somehow I messed up wherever /var/run/named is supposed to be. This happened when changing a root-jail DNS server to a non-jailed server. After toggling the NAMED_RUN_CHROOTED value to "no" in /etc/sysconfig/named starting named complains about this [this named is meant to integrate with Samba4]. Seems strange. Once you try to restart named after this change /var/run/named is automatically created as a directory - but it doesn't work. This fix is to stop named and create the correct symbolic link:

ln -s /var/lib/named/var/run/named /var/run/named

Not sure how this situation happens; but now the fix/gotcha is here for the search engines to crawl.

Reformatting an iPod

2011-10-31T06:59:00.000-04:00

I have an iPod whose content seems to have gone wonky. Delete's fail, play-lists won't sync, etc... I wanted to start-over. Unfortunately the iPod itself doesn't have any useful "reset" feature. According to the interwebs you need Apple's iTunes application in order to reformat an iPod. Alright, time to find another solution; and the winner is:

mkfs.vfat -F 32 -I -n "iPod Name" /dev/sdb1

A good old-school reformat. After reformatting and resetting [hold down select and play for 15 seconds] the device is back to its original state.

Implementing Queue Expiration w/RabbitMQ

2011-10-31T06:47:00.000-04:00

The latest versions of RabbitMQ support a feature where idle queues can be automatically deleted from the server. For queues used in an RPC or workflow model this can save a lot of grief - as the consumers for these queues typically vanish leaving the queue behind. Over time these unused queues accumulate and consume resources on the server(s). If you are using pyamqplib setting the expiration on a queue is as simple as:

import amqplib.client_0_8 as amq
connection = amq.Connection(host="localhost:5672", userid=*, password=*, virtual_host="/", insist=False)
channel = connection.channel()
queue = channel.queue_declare(queue="testQueue", durable=True, exclusive=False, auto_delete=False, arguments={'x-expires': 9000})
channel.exchange_declare(exchange='testExchange', type="fanout", durable=False, auto_delete=False)
channel.queue_bind(queue="testQueue", exchange='exchange')

Now if that queue goes unused for 9 seconds it will be dropped by the server [the value is in milliseconds]. So long as the queue has consumers it will persist, but once the last consumer has disconnected and no further operations have occurred - poof, you get your resources back.

Changing Terminal Services License Mode

2011-10-21T06:29:00.000-04:00

You are provisioning a Window 2008R2 server for remote desktop service; you've configured the terminal services license manager in one mode [ device | user ]. But when you receive the license documentation you discover that the CALs purchased were for the other mode. Then the Windows terminal license server manager tells you to change the mode of the license server.... but there is no obvious way to change the mode [because Windows is user-friendly!]. One option is to go old-school - hack the registry!.

First, stop the license server. Then in regedit change the value of the "LicensingMode" key in the "HKEY_LOCAL_MACHINE \ System \CurrentControlSet \ Control \ Terminal Server \ RCM \ LicensingCore" collection. A value of "2" indicates per device licensing and a value of "4" indicates a value of per user licensing. Then reboot.

Finding Address Coordinates using Python, SOAP, & the Bing Maps API

2011-10-13T10:09:00.002-04:00

Bing maps provides a SOAP API that can be easily accessed via the Python suds module. Using the API it is trivial to retrieve the coordinates of a postal address. The only requirement is to acquire a Bing API application key; this process is free, quick, and simple.

import sys, urllib2, suds

if __name__ == '__main__':  
    url = 'http://dev.virtualearth.net/webservices/v1/geocodeservice/geocodeservice.svc?wsdl'
    
    client = suds.client.Client(url)
    client.set_options(port='BasicHttpBinding_IGeocodeService')
    request = client.factory.create('GeocodeRequest')

    credentials = client.factory.create('ns0:Credentials')
    credentials.ApplicationId = 'YOUR-APPLICATION-KEY'
    request.Credentials = credentials

    #Address
    address = client.factory.create('ns0:Address')
    address.AddressLine = "535 Shirley St. NE"
    address.AdminDistrict = "Michigan"
    address.Locality = "Grand Rapids"      
    address.CountryRegion = "United States"
    request.Address = address

    try:        
        response = client.service.Geocode(request)    
    except suds.client.WebFault, e:        
        print "ERROR!"        
        print(e)
        sys.exit(1)

    locations = response['Results']['GeocodeResult'][0]['Locations']['GeocodeLocation']
    for location in locations:        
        print(location)

If you need to make the request via an HTTP proxy server expand the line client = suds.client.Client(url) to:

    proxy = urllib2.ProxyHandler({'http': 'http://YOUR-PROXY-SERVER:3128'})\
    transport = suds.transport.http.HttpTransport()
    transport.urlopener = urllib2.build_opener(proxy)
    client = suds.client.Client(url, transport=transport)

The results will be Bing API GeocodeLocation objects that have an Longitude and Latitude properties. Note that you may receive multiple coordinates for an address as there are multiple mechanism for locating an address; the method corresponding to the coordinates is a string in the CalculationMethod property of the GeocodeLocation objects.

Suppressing SNMP Connection Messages

2011-08-04T07:33:00.000-04:00

You have, of course, done the responsible sys-admin thing and setup an NMS (be it ZenOSS, OpenNMS, Nagios, whatever...). Then there is the concommitant action of configuring SNMP services on all the relevant hosts. All is good. But running SNMP on several distributions churns out the log messages; when you go to the logs to research a problem you have to filter out and sort through thousands upon thousands of pointless messages like:

Aug 1 16:08:38 flask-yellow snmpd[1976]: Connection from UDP: [192.168.1.38]:52021
Aug 1 16:08:38 flask-yellow last message repeated 24 times

Argh. Detail logging is good, but pointless noise is not. The solution isn't very well documented but you can bring this to a stop.

Step 1.) Make sure you have net-snmp 5.3.2.2 or later. This should not be a problem as even RHEL5/CentOS5 provide this version via update.

$ rpm -q net-snmp
net-snmp-5.3.2.2-9.el5_5.1

Step 2.) Edit /etc/sysconfig/snmpd.options or your system's equivalent making sure you do not pass the "-a" option to the SNMP daemon. The "-a" option enables the logging of the source IP addresses of all incoming requests. If you want to know about these kind of events iptables and ulog are more reliable and efficient methods for capturing that information.

# OPTIONS="-Lsd -Lf /dev/null -p /var/run/snmpd.pid -a"
OPTIONS="-Lsd -Lf /dev/null -p /var/run/snmpd.pid"

Step 3.) Edit the /etc/snmpd/snmpd.conf verifying you have the dontLogTCPWrappersConnects directive set to 1 (true). Add this directive to the configuration file if it is not specified.
Step 4.) Restart the SNMP service.

Now when you go to look into the log files it is again possible to hear the breeze, the singing of the birds, and the distant growling of that guy from Kazakstan who is trying to crack your SSH daemon.

Changing Your FreeNode Password

2011-05-17T19:00:00.000-04:00

There is no shortage of documentation for common IRC operations like registering your nick and managing channels. But I had quite the time figuring out the syntax for changing my password; turns out it's obvious:

/ns set password new_password

There is no password confirmation - so make sure you type it correctly.

On a related note, for anyone new to IRC, it is possible to bind multiple nicks to your FreeNode account. Just switch to the nick you want to bind and then issue the "group" command.

/nick alternate_nick
/msg nickserv group

You can now authenticate as this alternate nick using the same password as that of your master nick.

A Fortnight With GNOME3

2011-05-17T07:33:00.010-04:00

A fortnight with GNOME3

I'm a skeptic of "revolutionary" change. Most [all?] revolutionary changes result in epic-fail; everyone who has been in IT for more than a decade knows this. And there have been no shortage of predictions that GNOME3 will face this same fate. KDE endured this storm recently with version 4. Anyone unfortunate enough to be on the openSUSE user lists remembers the swarm of incessant nattering nabobs that was kicked up when the distro switched from KDE3 to KDE4. But change seems to be in the wind: KDE with version 4, Canonical deciding to hoe a proprietary row with Unity, and GNOME developers finally launching GNOME3.

I've been to several GNOME3 talks at Ohio LINUX given by GNOME team members; I've seen it demonstrated and I understood, at least vaguely, the ideological premise. As a hard-core groupware guy the idea of focusing on the actual workflow of the user was like a marketing pitch designed with people like me as the target audience. But ideology is a dangerous master; ideologues usually end up skidding face-first across the rough ground of reality. Take away the maximize and minimize buttons? Remove, or even refactor, the task bar? Get rid of the system / notification / whatever-junk tray? Do that and the hue and cry will be so loud you'll never get a chance to explain your ideology [as though anyone but you cared in the first place].

But now it seems that hue and cry will fall on deaf ears; there isn't much of a refuge. Canonical, KDE, and GNOME - everyone is moving. So it is time to move. I installed GNOME3. I bit-the-bullet and used it.

Now a fortnight later... I like it. It is better. Performance is improved and common operations are smoother. Doing things like navigating applications, which was before a combination of launchers, menus, and third-party components like GNOME-Do, is significantly more intuitive. I should say: they are more intuitive once you get over the habituation of how you were doing it before. I can see the designer's intent. With GNOME2 my desktop usage was efficient but I'd made it that way; my desktop was different than every other GNOME2 desktop [and conversely their desktops were different from mine]. Installing a modern LINUX distribution like openSUSE 11.4 gives me 99.44% of the applications I need; but I still have to go about making the desktop configuration efficient for me with launchers, etc... No more with GNOME3. GNOME3 provides the functionality of GNOME-Do and other GNOME2 third-party enhancements and drops the cruft I was always, unknowingly, fighting against.

Yes, some changes seem a bit arbitrary, like removal of the maximize and minimize buttons [you can turn them back on BTW, using the very nice GNOME Tweak Tool]. But how often do I really minimize anything anymore? Something close to never.

[UPDATE#1: I should have included a link to sloshy's very helpful post "How To Tweak GNOME 3 To Your Needs". So I've now rectified that error. Please note that I don't actually tweak GNOME3 much or install the various GNOME Shell extensions which are available. I recommend you really give GNOME3 vanilla an honest try. His post "10 Things I Love About GNOME 3" is also an interesting read - and helps explain some of the GNOME3 ideology. I should also point out that there are at least two ways to maximize a window in vanilla GNOME3 so the removal of maximize button seems reasonable. Minimization in GNOME3, lacking a task bar, is awkward - so removal of the minimize button in order to discourage the behavior seems reasonable as well. As I said originally: I never minimize anyway.]

Yes, the task bar is gone. This I was certain I would notice. But after a few days I didn't. The much improved Alt-Tab (application switch) and Alt-` (window switch) is far more productive than the task bar [which required use of the mouse]. Do I need a list to remind me of which applications are running? No.

Marking an application as a favorite, creating the GNOME3 equivalent of a launcher, is intuitive. This is a big improvement over trying to find an application in the menus and then awkwardly dragging it to some empty space on the toolbar. In GNOME3 it is also possible to drag an application icon into a specific workspace to start it on that workspace - which is nothing short of elegant.

There are also long needed improvements. GNOME has always had excellent screen-capture capability. GNOME3 now provides integrated screencast capture. The need for third-party tools like GNOME-Do for launching and the tracker applet for searching have been eliminated. It is all built in, as it should be.

So after my fortnight I look at overwhelmingly negative articles and I wonder... what desktop environment are they talking about? Because I don't see their criticisms in GNOME3. Perhaps they are booting it up and just test-driving it for a few hours? That would certainly be frustrating. But quotes like "No matter how you look at GNOME Shell ... you are going to do a lot of clicking" is just incorrect. I do far less "clicking" in GNOME3. By the end of that article I don't recognize the DE he is talking about; it certainly isn't GNOME3.

None of this is to say that there aren't valid criticisms of GNOME3.

Ideologically the emphasis on making a one-size-fits-all DE is misguided. Talking about one DE for both my large-screen i7 laptop and a small low-power mobile device does not make sense. Comparing some facebook/gmail jockey's usage of a tablet to someone doing real work is nonsensical. But I have confidence that ideology will be tempered by the reality of these very different use-cases. GNOME Shell seems flexible enough to accommidate both; thus being the same while being different.

Technically, there are some dot-zero kind of warts. The network manager interface isn't nearly as robust or as feature complete as the excellent interface provided in GNOME2; the absence of VPN support is particularly painful. Not all applications currently support startup-notification so dragging a launcher to a specific workspace doesn't always work.

[UPDATE#2:Our web developer pointed out that VPN support is now available in the the openSUSE 11.4 GNOME3 Network Manager. Sure enough - it works. The only option I don't see is how to enable proxyarp for a PPTP connection. That is one of the biggest negatives taken care of; now I can't think of a compelling reason not to recommend GNOME3.]

Maybe GNOME3, most specifically because of its weak Network Manager, isn't ready for your desktop quite yet. But surf on over to the excellent GNOME3 website and take a look. Go into GNOME3 with an Open mind and I think you'll discover that you like it.

Automated Backup of IOS Router Configuration

2011-05-17T06:08:00.001-04:00

Who hasn't had the experience of remoting to a router and making a configuration change... and not saving that change. Inevitably that is the weekend that facility will experience a power outage long enough to deplete their UPS. And then you get that dreaded text message from NetOps that a facility is down. Argh! Fortunately Cisco IOS 12.x and later supports a cron like service known as "kron". One of the handiest uses for kron is to configure automatic backup of the router's configuration to a TFTP server.

kron occurrence backup at 0:00 Thu recurring
 policy-list backup
!
kron policy-list backup
 cli write
 cli show running-config | redirect tftp://192.168.1.38/brtgate.config
!

This creates a batch of commands named "backup" [where in typical ISO fashion everything is referred to as a "policy"] that will be executed every Thursday morning. This batch commits the running configuration to flash memory ["cli write"] and copies the running configuration to the specified TFTP server ["cli show running-config | redirect tftp://192.168.1.38/brtgate.config"]. The rather odd looking use of "redirect" is because the IOS "copy" command is interactive and interactive commands cannot be run via "kron".

Remember that the file on the TFTP server has to exist, even if zero sized, and be world writable; otherwise the redirect will fail with a permission denied error.

Compression & Decompress Of A Stream

2011-04-22T07:48:00.001-04:00

So far in Python I had not found a good method / module for performing compression and decompression of data as streams; most tools required files to be compressed which has some obvious limitations. But then I saw a mention of pyLZMA roll by. It supports compression and decompression of streams using the Lempel–Ziv–Markov chain algorithm. The license of the module is LGPL-2.1; not MIT, but at least it is "Lesser" GPL'd. I've taken it for a spin and it seems to successfully compress and decompress all the data I've thrown at it (remember to always checksum your data).

import pylzma, hashlib

# Calculate the SHA checksum for our input file
i = open('Brighton.jpg', 'rb')
h1 = hashlib.sha1()
while True:
    tmp = i.read(1024)
    if not tmp: break
    h1.update(tmp)
h1 = h1.hexdigest()
print 'Input SHA Checksum: {0}'.format(h1)
    
# Compress the input file (as a stream) to a file (as a stream)
o = open('compressed.lzma', 'wb')
i.seek(0)
s = pylzma.compressfile(i)
while True:
    tmp = s.read(1)
    if not tmp: break
    o.write(tmp)
o.close()
i.close()

# Decomrpess the file (as a stream) to a file (as a stream)
i = open('compressed.lzma', 'rb')
o = open('decompressed.raw', 'wb')
s = pylzma.decompressobj()
while True:
    tmp = i.read(1)
    if not tmp: break
    o.write(s.decompress(tmp))
o.close()
i.close()

# Check the decompressed file
i = open('decompressed.raw', 'rb')
h2 = hashlib.sha1()
while True:
    tmp = i.read(1024)
    if not tmp: break
    h2.update(tmp)
h2 = h2.hexdigest()
print 'Result SHA Checksum: {0}'.format(h2)
if (h1 == h2): print 'OK!'

Of course a JPEG file doesn't compress much, but that makes it an even better test case.

block_dump logging

2011-04-20T18:15:00.001-04:00

There are lots of tools for studying the systems use of CPU and memory, but I/O is generally harder to track down. A useful trick is available via the block dump. Setting the value to "1" turns on block access logging to the kernel ring-buffer [aka dmesg] and a value of "0" turns it back on. This means it can be turned on by a simple:

echo "1" > /proc/sys/vm/block_dump

This logs the accesses to the block storage as:

[ 2032.934178] postmaster(11528): READ block 5058592 on dm-3 (16 sectors)
[ 2032.934200] postmaster(11528): READ block 5058624 on dm-3 (32 sectors)
[ 2032.934240] postmaster(11528): READ block 3172800 on dm-3 (16 sectors)
[ 2032.945328] banshee-1(11267): dirtied inode 1051864 (banshee.db-journal) on dm-0
[ 2032.945336] banshee-1(11267): dirtied inode 1051864 (banshee.db-journal) on dm-0
[ 2033.042671] python(11518): READ block 9017928 on dm-2 (32 sectors)
[ 2033.055771] python(11518): dirtied inode 267260 (expatbuilder.pyc) on dm-2
[ 2033.055808] python(11518): READ block 9017960 on dm-2 (40 sectors)
[ 2033.412972] nautilus(11078): dirtied inode 410492 (dav:host=127.0.0.1,port=8080,ssl=false) on dm-0
[ 2033.413001] nautilus(11078): READ block 50855560 on dm-0 (40 sectors)
[ 2033.431011] nautilus(11078): dirtied inode 410596 (dav:host=127.0.0.1,port=8080,ssl=false-ab9de673.log) on dm-0
[ 2033.431044] nautilus(11078): READ block 50855736 on dm-0 (64 sectors)
[ 2034.221831] jbd2/dm-2-8(386): WRITE block 21261800 on dm-2 (8 sectors)
[ 2034.221887] jbd2/dm-2-8(386): WRITE block 21261808 on dm-2 (8 sectors)

Handy.

Printing Via LPR

2011-03-17T06:34:00.001-04:00

If you have a Python app, or almost any kind of app, the accepted manner for printing is to use some kind of subprocess to invoke some command-line utility to submit the print job. Of course this requires that the underlying subsystems are aware of printers [and thus run a printer subsystem]. It also assumes the name of the command-line utility, the permissions are adequate to execute it, and all manner of other things. To put it simply: this is terrible! Why does my web server, workflow server, etc... need to run a print service? Why can't my environment just support submitting a job to an actual print server? Good question! To answer this OpenGroupware Coils now has a simple implementation of the LPR/LPD protocol. This code is under the MIT/X11 license so you are free to copy-and-paste it into your own application. Sending a job to the server is as simple as:

f = open('Draft.ps', 'rb')
pr = LPR('crew.mormail.com', user='adam')
lpr.connect()
lpr.send_stream('cisps', 'test job', f, job_name='my awesome job')
lpr.close()
f.close()

This sends the contents of the file "Draft.ps" to the printer "cisps" on the LPD server "crew.mormail.com" with user name "adam", job name "my awesome job" and job file name "test job".

The code for the LPR class can be found here.

Configuration of NTP for Samba4

2010-12-08T13:51:00.001-05:00

In an Active Directory domain, which is a Kerberos domain, time-synchronization is critical. If a workstation or servers clock drifts to much from the clock of the KDC neither users or services will be able to participate in the domain. But Active Directory clients [such as Windows 7] require an MS-SNTP server that cryptographically signs the response. This means that the NTP server shipped with almost all current distributions cannot provide time services to these devices - support for this interoperability was added in NTP 4.2.6 (see NTP Bug#1028). Appropriately versioned NTP packages are available for openSUSE 11.3 from namtrac's repository and will be included in openSUSE 11.4 (see openSUSE Bug#657194)..
With a good version of NTP the required NTP configuration is:

ntpsigndsocket /opt/ad/samba4/var/run/ntp_signd/
restrict default mssntp

Samba4 will create a socket which NTP can use to have its responses signed - the location of this socket is what is indicated by the NTP directive "ntpsigndsocket". Note that this is a directory path to where NTP will expect to find a domain socket named "socket" (not intuitive!). It is Samba which creates and listens to this socket; NTP uses the socket as a consumer and only opens the socket when it needs signing services. The default location where Samba will initially create this socket is controlled via the "ntp signd socket directory" directive in Samba's "smb.conf". In most cases this won't be set explictely. You can verify the default value using the Samba "testparams" command with the "--verbose" option which will list both defined directives and the default values of undefined directives [including "ntp signd socket directory"].
Running the NTP daemon as "strace ntpd -n" will keep ntpd in the foreground and allow you to see the files and sockets that it opens and closes. Once properly configured issue the "w32tm /resync /rediscover" option on a Windows 7 AD member and you should receive the response:

Sending resync command to local computer
The command completed successfully.

On the server you will see (in the strace output) and exchange between NTP and Samba4 via the signing socket. It will look something like:

recvmsg(21, {msg_name(-1734344)={...},
msg_iov(17832102453714092032)=0x14ffe5894c, msg_controllen=120259084288,
msg_control=0x848, msg_flags=MSG_DONTROUTE|MSG_PROXY}, 0) = 68
recvmsg(21, 0xffe5891c, 0)              = -1 EAGAIN (Resource
temporarily unavailable)
clock_gettime(CLOCK_REALTIME, {1291827449, 637483079}) = 0
socket(PF_FILE, SOCK_STREAM, 0)         = 4
connect(4, {sa_family=AF_FILE,
path="/opt/ad/samba4/var/run/ntp_signd//socket"}, 110) = 0
write(4, "\0\0\0@", 4)                  = 4
write(4, "\0\0\0\0\0\0\0\0\1\0\0\0P\4\0\0\34\v\21\360\0\0\0\0\0\0002\337
\177\177\1\0"..., 64) = 64
read(4, "\0\0\0P", 4)                   = 4
read(4, "\0\0\0\0\0\0\0\3\0\0\1\0\34\v\21\360\0\0\0\0\0\0002\337\177\177
\1\0\320\2527c"..., 80) = 80
sendto(21, "\34\v\21\360\0\0\0\0\0\0002\337\177\177\1\0\320\2527c>\360
\220\303\320\2527\201\3478 \330"..., 68, 0, {sa_family=AF_INET,
sin_port=htons(123), sin_addr=inet_addr("10.66.77.102")}, 16) = 68
close(4)

If that doesn't happen verify that both Samba4 and NTP have access to the socket; this may include permissions issues as well as SELinux / AppArmour policies. Once it works just run NTP normally - you have working secure time synchronization.

Manually Adding an ACL To An Object

2010-12-08T09:15:00.002-05:00

In OpenGroupware the ACLs applied to an object are stored in the "object_acl" table. If, for example, I want to add the list, view, read, write, and administer privileges for the team 11,530 for object 1,6829,810 the correct SQL to execute is:

INSERT INTO object_acl
(object_acl_id, sort_key, action, object_id, auth_id, permissions)
VALUES (nextval('key_generator'), 0, 'allowed', 16829810, 11530, 'lvrwa')

The important points are:

Use the "key_generator" sequence to assign the "object_acl_id" value. This is the object id of the ACL itself; all object ids are assigned from the key_generator sequence.
The value of "sort_key" is always 0. This value isn't actually used for anything.
The value of "action" must be either "allowed" or "denied". In most cases "allowed" is what you want in order to grant access.
"object_id" is the object id of the object to which the ACL is applied in contrast to "auth_id" is the context to which the privileges, specified in "permissions", are either granted [if "action" is "allowed"] or revoked [if "action" is "denied"]. The value of "auth_id" should be the object id of an account or a team.
The permissions string is always lower case. Permission flags are documented in WMOGAG.

The ACLs in "object_acl" are the primary access control mechanism for all entities excepting Projects and Appointments.

BOM Squad

2010-11-30T14:55:00.000-05:00

So you have a lovely LDIF file of Active Directory schema that you want to import using the ldbmodify tool provided with Samba4... but when you attempt the import it fails with the error:

Error: First line of ldif must be a dn not 'dn'
Modified 0 records with 0 failures

Eh? @&^$*&@ &@^&*@ ^@&*^@&* It does start with a dn: attribute!
Once you cool down you look at the file using od, just in case, and you see:

0000000 o ; ? d n : sp c n = H o r d e -

You've been bitten by the BOM! But even opening the file in vi you can't see the BOM because every tool knows about the BOM and deals with it - with the exception of anything LDIF related.
The trick is to break out dusty old sed and remove the BOM -

sed -e '1s/^\xef\xbb\xbf//' horde-person.ldf > nobom.ldf

And double checking it with od again:

0000000 d n : sp c n = H o r d e - A g o

The file now actually starts with a dn attribute!

Samba4 & PHPLDAPAdmin

2010-11-26T11:19:00.000-05:00

Samba4 includes a PHP LDAP admin configuration sample. Unfortunately it doesn't match up to the current version of PHP LDAP Admin. The configuration you want to put into config/config.php is:

$servers->newServer('ldap_pla');
$servers->setValue('server','name','Samba4 AD Server');
$servers->setValue('server','host','ldapi://%2Fopt%2Fad%2Fsamba4%2Fprivate%2Fldapi');
$servers->setValue('login','auth_type','session');
$servers->setValue('login','attr','dn');

Then you should get an PHP LDAP Admin login screen for your shiny Samba4 AD DSA. For the AD uninitiated the DN of the Administrator's account is "CN=Administrator,CN=Users,DC=ad,DC=mormail,DC=com" (for example, if you AD domain is "ad.mormail.com"). Using that DN and the domain administrator's password you should be able to login.
Another trick is to put that LDAPI LDAP URI into the /etc/openldap/ldap.conf file so you can use the OpenLDAP LDAP CLI utilities [ldapsearch, ldapadd, ldapmodify, etc...] provided by your distribution.

Python Exceptions and "auto" Instantiation

2010-11-26T09:50:00.003-05:00

Interesting thread on the python-list as to what happens regarding raising exceptions. The low-down: If you "raise ExceptionClass" and instance of that ExceptionClass is always created.

For example:

try: raise KeyError
except KeyError: pass

From the documentation:

raise evaluates the first expression as the exception object. It must be either a subclass or an instance of BaseException. If it is a class, the exception instance will be obtained when needed by instantiating the class with no arguments.

Some exception classes require arguments so the naked raise will fail with a TypeError.

raise UnicodeDecodeError
Traceback (most recent call last):
File "", line 1, in
TypeError: function takes exactly 5 arguments (0 given)

Seems to me it is clearer to just do the raise with instance syntax:

try:
raise KeyError()
except KeyError as e:
pass

SQLAlchemy & Upcoming Birthdays

2010-11-05T07:09:00.000-04:00

OpenGroupware Coils uses SQLAlchemy as it's ORM. One of the desired features was a Logic command that efficiently returns contacts with upcoming birthdays. In raw SQL this query would be very simple to write - but how to do it in SQLAlchemy? The answer: "sql.expression.extract" which will create an expression column equivalent to EXTRACT. With EXTRACT it is possible to compare to the year-of-day represented by a date. The Python code looks like:

db = self._ctx.db_session()
# Get the current day-of-year
doy = datetime.today().timetuple().tm_yday
# Deal with year wrap-around
floor = doy - 2
if (floor < 1): floor +=365
ceiling = doy + 14
if (ceiling > 365): ceiling -= 365
# Create a field that is the SQL expression DOY(Contact.birth_date)
orm_doy = sql.expression.extract('doy', Contact.birth_date)
# Create the query
query = db.query(Contact).filter(and_(sql.expression.between(orm_doy, floor, ceiling),
                                      Contact.birth_date != None,
                                      Contact.is_account == self.accounts,
                                      Contact.status != 'archived'))

In this example "self.accounts" is an attribute with a value of 0 or 1.

Ohio LinuxFest 2010

2010-09-16T07:34:00.000-04:00

Last weekend (Sept. 10th - 11th) was Ohio LinuxFest 2010 - and it was the biggest one yet. The growth each year of this conference is impressive; both in attendees and vendors. While the website is persistently void of detailed information[1] this is a great conference for users of Open Source software and I recommend everyone attend if at all possible. While once a Saturday only conference it seems to have expanded, without much fanfare, to include Friday. This year there was a free all-day training session by Zenoss on Friday that I would have desperately loved to attend but was announced too late to make that possible. Next year I hope to be there for the Friday sessions. We (I and a sys-admin from Kalamazoo) did make if for the pre-party however - which is also something not-to-miss; it is a great opportunity to hobnob with other users, including some well-known luminaries. The nearly unlimited drink-tickets makes that even more entertaining.

Saturday's conference track began with Stormy Peters' excellent keynote on "Who is stealing your desktop?" about the dangers of cloud computing and the, almost stealth, proprietary services that so many users have come to rely on. It raises the question if Open Source "won" only to then give up and accept yet another hegemony of corporations. This same sentiment was echoed in Phil Robb's [of HP] talk "Irrelevance of the desktop".

Then it was on to regular part of the conference. This is where I have one serious gripe about OLF: the topic selection. It seems so "LINUX Journal". They aren't "bad" topics, but it seems a lot of sessions are spent on promotional, evangelistic, or quite esoteric fare. I'm generally opposed to almost all forms of "evangelism" so I'll admit I'm biased - but talking about promoting Open Source or how Open Source is better or the history of Open Source at an Open Source conference... doesn't this seem like preaching-to-the-choir and self-congratulation? The very first year I went to OLF I learned about what was new with PostgreSQL [from a PostgreSQL developer!], about Apache mod_rewrite, about SMTP grey listing, etc... really solid detailed stuff. I'd sit down and the presenter would dive right in. I still have the notes from that conference and I still use all that information in my daily job. That is lacking in recent OLF years where I come away with only a couple of actually useful bits of information. I still go because it is simply fun and the feel you get for the direction of certain projects and vendors is valuable in its own way.

Probably the best presentation was David Nalley's (ke4qqq) talk on SELinux. SE has always seemed like a black box - a black box to promptly be disabled! Only a few sentences into his talk and David had knocked the sides of that box. Straight-to-the-point and beautiful. Over time I will now proceed to re-enable SELinux at least on my DMZ hosts. There was some overlap between this an Deral Heiland's "And Now They Own You" - which could also have been named: "PEOPLE! FIREWALL EGRESS TOO!".

Oh, and Asterisk 1.8 will support IPv6! Another side-effect of OLF is to install a deep sense of Asterisk envy in those saddled with lousy / proprietary phone systems. CalDAV integration with the PBX... really... why didn't someone thing of that feature before? How simply spectacular. Oh, wait, I have Nortel phone systems... @*(&$&*(@*(& &*(@^$&*@^* @&^@$@!

[1] Even now the schedule page lists the "Pre-Party" details "TBD" (to-be determined). Fortunately a quick Twitter search discovered where it was.

XFS & Inodes

2010-09-16T06:17:00.001-04:00

There was an interesting thread on the opensuse list this week; and regular readers will note how rare an event that is [even if you filter out the KDE-users-who-hate-KDE and the IPv6-is-irrelevant-malcontents]. It had to do with the XFS filesystem and inodes. At least it was interesting to me as I recall facing the exact same problem several years ago. While, unlike traditional UNIX filesystems, XFS doesn't suffer from the boring old issue of "inode exhaustion" it does have inode limits - based on a percentage of the filesystem size.

linux-yu4c:~ # xfs_info /mnt
meta-data=/dev/sdb1              isize=256    agcount=4, agsize=15262188 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=61048752, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=29808, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

The key is that "imaxpct" value. In this example inode's are limited to 25% of the filesystems capacity. That is a lot of inodes! But some tools and distributions may default that percentage to some much lower value - like 5% or 10% (for what reason I don't know). This value can be determined at filesystem creation time using the "-i maxpct=nn" option or adjusted later using the xfs_growfs command's "-m nn" command. So if you have an XFS filesystem with available capacity that is telling you it is full: check your "imaxpct" value

Bootstrapping "opengoupware.us"

2010-07-26T06:36:00.000-04:00

Thanks to the addition of the "pages" feature in Blogspot I've finally created a site at opengroupware.us, The OpenGroupware [Legacy] project website (I won't even bother to link to it) has been worthless for some time, and the information about the constellation of projects beyond legacy is very scattered. opengroupware.us is an attempt to at least create an index of that information as well as resurrect some of the good content from the abyss that is the docs plone. If you want to submit content to opengroupware.us or want to help edit / maintain content just let me know any I'll add your Blogspot account to the site permissions.

XSLT Transform to TXT, with LXML

2010-07-16T07:28:00.000-04:00

Maybe this should be obvious, but it wasn't to me. I've got an XML document and an XSLT stylesheet. But that stylesheet just produces text, not XML; it is essentially a template for an e-mail [with some iterations, so something more complex than OIE's very convenient rowTemplateAction can handle]. So I was extending the transformAction for performing XSLT transforms that produce other than XML... but the documentation is a bit thin and every example is XML results. The trick is pretty simple, just

unicode(result)

and make sure [of course] that you have

<xsl:output method="txt" encoding="utf-8" omit-xml-declaration="yes"/>

declared in the template. So it should be as simple as:

def do_action(self):
        source = etree.parse(self._rfile)
        xslt = etree.fromstring(self._xslt)
        transform = etree.XSLT(xslt)
        result = transform(source)
        self._wfile.write(unicode(result))

in order to transform to just about anything.

Coils: Features merged into "default"

2010-07-08T07:35:00.001-04:00

This morning the following features were merged into OpenGroupware Coils "default" branch, and will be available in the next release.

The scheduler service no longer depends on the ticktock heartbeat. It checks the run queue in its work method.
Format(s) now support logging of rejected records to a buffer. This feature is available via a new parameter on the readAction: "rejectionsLabel". This is documented in the readAction's wiki page.
The above motivated me to abstract how ActionCommand goes about creating [output] messages. It is now simple for an action to create additional messages [beyond it's 'default' output message] by just calling the store_in_message(label, wfile, mimetype='application/octet-stream') method.
New CLI tools: "coils-list-schedule" and "coils-unschedule-process" that let the administrator view and modify the workflow scheduler from the command line. Yes, we still need a "coils-schedule-process".

getopt isn't the greatest command line option parser, would be great if someone would look into swapping that out with something better.

The above motivated me to add an initialize_tool(name, argv, arguments=['',[]]) function to coils.core.utility. This provides a simple way for a tool [vs. a service] to bootstrap itself onto the OpenGroupware Coils service bus so that it can receive callbacks, etc... Return value is an AdministrativeContext (with a registered Broker for IPC) and dictionary of unconsumed command line parameters. initialize_tool automatically consumes the --store=, --add-bundle=, and --ban-bundle parameters in order to initialize the Coils environment.

In the last week or so there has been significant work on tidying things up, and the deployment instructions on the wiki have been updated as well. Getting a working OpenGroupware Coils instance should now be very straight-forward. Big work ahead is the move to WSGI which is underway in the NetComm branch (which needs to be remerged with default, btw). And with the impending release of openSUSE 11.3, and its inclusion of Evolution 1.30.x, the GroupDAV address book support can be completely tested and polished. Evolution 1.30.x contains several enhancements to make it's WebDAV address book support fully compatible with GroupDAV.

Keeping an OIE Route After Completion

2010-05-24T08:36:00.001-04:00

Once an OpenGroupware Integration Engine route has successfully completed it is automatically garbage collected. This prevents the system from filling up with messages and version information. But if an external application, like ogo-curses-putter, wants to retrieve the output of the route... it's gone! To facilitate this use-case the garbage collection can be suppressed per-route by setting the {http://www.opengroupware.us/oie}preserveAfterCompletion property of the route entity with a value of "YES". If this property value exists the processes derived from that route will not be garbage collected. The simplest way to set the property is to retrieve the route via "route::get" and use your context's property manager:

from coils.core import *
ctx = AdministrativeContext()
r = ctx.run_command('route::get', name='MTAStockOrder_TEST')
ctx.property_manager.set_property(r, 'http://www.opengroupware.us/oie', 'preserveAfterCompletion', 'YES')
ctx.commit()

The above will set the value of {http://www.opengroupware.us/oie}preserveAfterCompletion to "YES" for the route named "MTAStockOrder_TEST". The property manager will update the value of the property if such a property already exists, or it will create a new property with the specified value.

Performing PROPFIND from Python

2010-04-25T14:16:00.000-04:00

Surprising, when I went looking for examples of performing PROPFIND requests from Python I found very little information. For testing OpenGroupware Coil's WebDAV presentation I needed a simple way to perform the PROPFIND required to list the entities in a collection and then request each entity. With a bit of scratching through the thin documentation I was able to come up with the following:

import httplib, urllib2, base64, sys
from lxml import etree
PROPFIND = u'''<?xml version="1.0" encoding="utf-8"?>
<propfind xmlns="DAV:">
<prop>
<getetag/>
</prop>
</propfind>'''
SERVER_HOST = '127.0.0.1'
SERVER_PORT = 8080
SERVER_PATH = '/dav/Contacts'
CLIENT_AGENT = 'Whitemice rules/so very true'
auth_string = 'Basic {0}'.format(base64.encodestring('adam:fred123')[:-1])
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPHandler()))
connection = httplib.HTTPConnection(SERVER_HOST, SERVER_PORT)
connection.putrequest('PROPFIND', SERVER_PATH)
connection.putheader('Authorization', auth_string)
connection.putheader('User-Agent', CLIENT_AGENT)
connection.putheader('Depth', 1)
connection.putheader('Content-Length', str(len(PROPFIND)))
connection.endheaders()
connection.send(PROPFIND)
response = connection.getresponse()
if response.status == 207:
  data = response.read()
  connection.close()
  response = None
  namespace_prefix_map = { 'D' : 'DAV:' }
  document = etree.fromstring(data)
  for path in document.xpath('/D:multistatus/D:response/D:href/text()',
      namespaces=namespace_prefix_map):
    if (path != SERVER_PATH):
      connection = httplib.HTTPConnection('127.0.0.1', 8080)
      connection.putrequest('GET', path)
      connection.putheader('Authorization', auth_string)
      connection.putheader('User-Agent', CLIENT_AGENT)
      connection.endheaders()
      response = connection.getresponse()
      if response.status != 200:
        print 'Error retrieving {0}'.format(path)
        sys.exit(1)

This will make the PROPFIND for all the items in the collection, requesting the getetag property, and then from the response perform an HTTP GET for every item. Even better would be if it requested isCollection and knew better than to perform GETs on collections. I hope this simple example will be of use to someone.

OIE Progress, and lots of it.

2010-03-17T07:04:00.000-04:00

The workflow engine [OIE] in the OpenGroupware Coils project has reached a point a general usefulness. OIE is accessible via WebDAV for creating, browsing, and starting flows. The engine compiles BPML 1.0 markup to an internal format (adding additional markup-notations, such as XPDL, shouldn't be much harder than writing a SaX parser to produce the internal format). The foreach flow control structure is implemented; switch, until, and while are simply stubs at this point. switch is actually compiled but can't be executed yet. Message scope is handled correctly. Even with just linear execution and foreach a great many business processes can be modelled, especially ETL type flows.

The actions implemented so far are:

read - Translate a message using a format; fixed record length and XLS formats have been implemented.
select - SELECT from a defined RDBMS.
write - Write a StandardXML message out using a specified format.
assign - XPath select a value or assign a static value to a message.
getEntity - Retrieve an entity from the groupware database.
xpath - Create a new message from an XPath query.
ldapSearch - Search an LDAP DSA, results are in DSMLv1.0.
regularExpressionFind - Perform regexp matches.
readJSON - Translate JSON data into XML.
sendMail - Send a message via an SMTP server, including support for attachments.

All commands support label substitution in appropriate parameters.

As of yesterday there is also an insert action for inserting data into a RDMS table; this action hasn't yet been as heavily tested as the others, but it completes the tool set required for simple ETL flows.

Disabling GNOME Automount

2010-03-17T06:31:00.000-04:00

When you connect a mass storage device to a computer running the GNOME desktop environment it automatically mounts the device in /media and places a short-cut to the device on your desktop. This is an excellent default behavior; but if you are working with various devices sometimes it can get in the way. To correctly disable this feature simply execute:

gconftool-2 --type bool --set /apps/nautilus/preferences/media_automount false

The devices will still appear in computer:/// (in nautilus) where you can right click on them to select the mount action if and when you want them to be mounted. If and when you want to re-enable to auto-mounting feature execute:

gconftool-2 --type bool --set /apps/nautilus/preferences/media_automount true

Passing a column set to an SQLalchemy query

2010-02-10T17:51:00.000-05:00

In implementing the List method of the EntityAccessManager provided by the Contacts bundle in OpenGroupware Coils it seemed like it would be very efficient to allow the consumer to request what set of attributes it needed; for instance, if a WebDAV client's PROPFIND request didn't ask for a given property, why request the corresponding attribute in the query? Especially since the result set for PROPFIND queries are frequently very large [on the order of 20,000 records or so]. But how to pass a set of attributes as a parameter?

def List(ctx, attributes):
...
db = query(attributes).filter(....)
return db.all

does not work; where attributes is a list, or a set of more than one attribute. Some fiddling around and I discover that the following works:

def List(ctx, *attributes):
...
db = query(*attributes).filter(...)
return db.all()

...
ctx = AssumedContext(10100)
a = BundleManager.get_access_manager('Contact', ctx)
a.List(ctx, (Contact.object_id, Contact.version))

That single asterisk is the trick.

Configuring OpenLDAP's dynlist in cn=config

2010-02-10T07:24:00.000-05:00

Step#1: Make sure the dynlist module is loaded on the server. What modules are loaded are typically controlled by the "cn=modules{0}, cn=config" object. If your server doesn't have a modules object it probably isn't loading any dynamic modules. Create an LDIF file and import it. Our "cn=modules{0}, cn=config" looks like:

dn: cn=modules{0}, cn=config
olcModuleLoad: {0}accesslog.la
olcModuleLoad: {1}auditlog.la
olcModuleLoad: {2}constraint.la
olcModuleLoad: {3}dynlist.la
olcModuleLoad: {4}memberof.la
olcModuleLoad: {5}ppolicy.la
olcModuleLoad: {6}refint.la
olcModuleLoad: {7}seqmod.la
olcModuleLoad: {8}syncprov.la
olcModuleLoad: {9}sssvlv.la
olcModuleLoad: {10}translucent.la
olcModuleLoad: {11}unique.la
olcModuleLoad: {12}back_monitor.la
olcModulePath: /usr/lib/openldap2.4
objectClass: olcModuleList
cn: modules{0}

NOTE: Double check that the "olcModulePath" is the absolute path to the directory containing the OpenLDAP modules.

Step#2: Check that your schema has the groupOfURLs objectclass defined. If you attempt to configure the dynlist module without that schema available you still crash slapd. To define the dynamicGroup schema (if it is missing) you can import the following LDIF into your cn=config:

dn: cn=dynamicGroup, cn=schema, cn=config
olcObjectClasses: {0}( 2.16.840.1.113730.2.33         NAME 'groupOfURLs'
     SUP top STRUCTURAL         MUST cn         MAY ( memberURL $ businessCat
egory $ description $ o $ ou $                 owner $ seeAlso ) )
olcAttributeTypes: {0}( 2.16.840.1.113730.1.198         NAME 'memberURL'
     DESC 'Identifies an URL associated with each member of a group. Any type
of labeled URL can be used.'         SUP labeledURI )
objectClass: olcSchemaConfig
cn: dynamicGroup

Step#3: Create an olcOverlayConfig object in the scope of your Dit database. For example, our Dit is the 1st HDB database on the server so the appropriate object is:

dn: olcOverlay=dynlist,olcDatabase={1}hdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynamicList
olcOverlay: dynlist

Step#4: Now you are ready to actually use the dynlist module. A common use-case is to create dynamic mail alias objects; with dynlist you don't need to maintain mail aliases, they will automatically contain everyone who matches the relevent criteria. Provided you use the traditional nisMailAlias objectclass in order to define mail aliases adding the attribute -

olcDlAttrSet: {0}nisMailAlias labeledURI rfc822mailmember:mail

- to "olcOverlay=dynlist,olcDatabase={1}hdb,cn=config" will enable dynamic mail aliases. Specifically any nisMailAlias containing a labeledURI attribute will be expanded by the query specified in that attribute. The results of that query will be rewritten by the optional rfc822mailmember:mail clause which will rename the mail attributes resulting from the query into the rfc822mailmember attribute required by consumers of the nisMailAlias objects. So rather than populating the mail alias object with rfc822mailmember attributes manually, you extend the object with the auxilliary labeledURIObject objectclass and define the query in the labeledURI attribute.

dn: cn=gr_parts, ou=ListAliases, ou=Aliases, ou=Mail, ou=SubSystems, o=Morrison Industries,c=US
mail: gr_parts@morrison-ind.com
labeledURI: ldap:///ou=People,ou=Entities,ou=SAM,o=Morrison Industries,c=US?mail?one?(&(morrisonactiveuser=Y)(objectclass=morrisonuser)(departmentNumber=*P*)(morrisonbranch=GRD))
objectClass: nisMailAlias
objectClass: top
objectClass: labeledURIObject
cn: gr_parts

The object will immediately populate with rfc822mailmember attributes derived from the mail attribute of those objects matching the specified filter: "(&(morrisonactiveuser=Y)(objectclass=morrisonuser)(departmentNumber=*P*)(morrisonbranch=GRD))". The third parameter, "one", of the URI is the scope of the query so only objects immediately subordinate to "ou=People,ou=Entities,ou=SAM,o=Morrison Industries,c=US" are candidates for the filter.

Twython Presentation Uploaded.

2010-01-21T13:43:00.000-05:00

My presentation on Twython (a Python Twitter API provider) and integrating Twitter services into OpenGroupware Coils has been uploaded to the GRPUG forum.. This covers the basics of using Twython such as authentication, tweeting, retrieving time lines, searching, etc...
Because Twitter rate limits connections [as well they should] if you are going to provide Twitter access to a large numbers of users, potentially NAT'd behind a single IP, it is recommended that you create some type of middle-ware to proxy and cache your Twitter access - the Twitter support in OpenGroupware Coils is one example of how to do that.

Python Curses In Action, even on AIX.

2010-01-21T08:14:00.005-05:00

So, what if you have an old COBOL application that you want to integrate with some web services or a website? You'd need a "green-screen" application that can take that COBOL applications temporary output file, send it up to the web service, wait for a response, replace the temporary file, and then return control to the COBOL application (or whatever "green screen" application comes next). It sounds simpler than it is in practice: what if the web service takes awhile to complete [or has human involvement on the other end?!] or the web service isn't available right at the moment you try to send your request? For a real production environment your "green screen" application would need to deal with all of those things.
And you need that "green screen" application to work on both LINUX and AIX!

Fortunately pware provides Python 2.6 for AIX, including curses! The curses library and respective Python module provide a surprisingly easy way to create professional looking TUI (Text User Interface) applications like the solution required in our example. This particular application takes them temporary file and submits it via a WebDAV PUT operation to the workflow engine provided by OpenGroupware Coils. The route in the workflow engine reformats the dreadful output of the COBOL application into the required format (using the format support in route Read and Write operations). The client detects the route is complete by watching the URL specified in the header of the initial PUT, and then downloads the required data. If anything goes wrong along the way the client can retry, or provide the user the option to abort. All along the way the client provides detailed feedback to the user and a familiar dialog-box and prompt interface when feedback is required.

XAML designer for Monodevelop?

2009-10-11T16:56:00.003-04:00

If I could pick one single thing to add to my development toolchain it would be a XAML designer. These days I do 90% of my development in the excellent MonoDevelop IDE (that last 10%, maintaining the OpenGroupware.org Objective-C code base, I do in gedit) Now that MonoDevelop supports Python I can do all my work on OpenGroupware Coils there as well. But I just can't imagine developing web applications using the tricycle: HTML, CSS, Javascript. Every web development presentation I see leaves me with a single distinct impression: "What a hack!" Web development is simply too cumbersome and tedious. The future of the web is clearly web services (REST, SOAP, XMPP, and XML-RPC) combined with web backed applications using Silverlight/Moonlight and Flash. My money is on Silverlight/Moonlight being the eventual winner. And today when I brought up Monologue I see: "xaml designer for monodevelop". It isn't complete, maybe not even usable yet, but it certainly is encouraging; maybe someday I'll be able to return to web application development, and riding a 10-speed rather than a rusty old tricycle.

Wow, Slashdot covers some real FOSS!

2009-10-04T20:46:00.002-04:00

It seems like it has been ages since this last happened - but Slashdot has a blurb about a real FOSS Story: Interview With Jeremy Howard of FastMail.fm. FastMail both runs and develops services based on Cyrus IMAP. Cyrus IMAP is Open Source, has an amazing depth of features, stunning performance, and is extremely standards compliant. It is very refreshing to see some attention given to a real Open Source solution that has been serving user for a decade. This in contrast to a not-open not-standards-compliant solution whose name you can probably guess - with whom Slashdotters appear to be utterly smitten [well beyond the point of clear thinking, but this is (a) Slashdot and (b) a normal side-effect of being smitten by something.] I'm a big fan of Cyrus IMAPd which has been very feature-rich and stable since at least 1999: you get delayed expunge, full text indexing, service-side filtering (SIEVE), shared folders, partitioning, replication, modification sequences, message expiration, and more. Other IMAP/POP servers are still trying to bolt these features onto themselves. If you aren't using Cyrus IMAPd I recommend you take a look, I honestly don't know why anyone would choose anything else.

Ohio LinuxFest 2009 registration deadline is looming..

2009-09-13T09:22:00.001-04:00

Ohio Linux Fest 2009
Free and Open Source Software Conference and Expo
Columbus, Ohio : September 25-27, 2009

Registration Deadline: September 18, 2009

The Ohio LinuxFest 2009 registration deadline is September 18, 2009. If you have not registered yet, please hurry over to http://www.ohiolinux.org and register today.

Join us for the seventh annual Ohio LinuxFest conference to celebrate 40 years of Unix. We have an exciting line up of talks and this year the conference extends from Friday to Sunday.

Friday, September 25, features professional training courses (Professional package registration required) and a course for Linux beginners which includes a refurbished computer, with Linux pre- installed for you (Quick Start package required). A series of conference talks and other sessions will be held on Friday as well.

Saturday, September 26, features the expo and four conference tracks. We have a great line up of speakers; Doug McIlroy, Shawn Powers, Dr. Peter Salus, and many more.

Two certification examinations will be held on Sunday the 27th. Linux Professional Institute will host an LPI certification exam, and the BSD Certification Group will offer the BSDA certification exam for Ohio LinuxFest attendees.

Four registration packages are available for the Ohio LinuxFest this year.

The Enthusiast Package is available at no cost for students, enthusiasts, and those that want to come to the event to find out more about Free and Open Source software.

The Supporter Package is available again at a low cost of $65.00 to support the event. As a measure of appreciation, the supporter package includes lunch on Saturday and a commemorative Ohio LinuxFest tee-shirt.

The Quick Start Package is available for $250.00, which in addition to access to the events on Saturday, includes a full day of Linux Basics training on Friday and a refurbished Linux computer to take home.

The Professional Package is available for $350.00, which in addition to Saturday's activities, also includes access to the one day OLFU tutorials on Friday.

The Ohio LinuxFest welcomes people from all 50 states and international participants. We have had participants from Canada, England, Argentina, Brazil, and Australia in years past. Note that the Ohio LinuxFest is a 501c3 non-profit, volunteer organization. All proceeds are used for conference costs.

New OGo Help Desk Feature

2009-09-11T08:23:00.002-04:00

Enhancement Bug#2027 "Allow help desk users to create tasks on behalf of users" has been resolved as of r2274. This update requires a database schema change -

ALTER TABLE job ADD COLUMN owner_id INT;

UPDATE job SET owner_id = creator_id;

The database scripts pg-build-schema.psql and pg-update-1.x-to-5.5.psql have been updated. NOTE: Be careful not to run this part of pg-update-1.x-to-5.5.psql more than once or you risk modifying actual data in your database if you use the OGoHelpDeskRoleName default.

This feature allows a member of the team whose named is defined in the OGoHelpDeskRoleName default to create tasks with an owner other than themselves. Delegated and archived task lists now display tasks based on owner rather than creator. By default the owner is the creator so this has no effect on normal task behaviour. Currently the help desk feature, setting of the owner to another user, is only available via the zOGI API. The modifications to the zOGI API are documented on the Task entity.

If your help desk team name contains spaces be sure to use proper quoting - Defaults write NSGlobalDomain OGoHelpDeskRoleName '"all intranet"' - or you may not get the results you expect. Obviously if you set OGoHelpDeskRoleName to "all intranet" all users will be able to create tasks on behalf of other users.

Let The Season Of Patching Begin.

2009-09-08T09:13:00.002-04:00

It is that time of year again: later sunrises, earlier sunsets, and more hacking. I've commited my first patch for the coming season of gloom - cache removal as previously discussed back in the days-getting-longer part of the year. [Note: Pipermail archives strip attachments, but you can find the patch on the Nabble archives]. This makes trunk r2270 bumping libZSBackend.so to version 5.5.94. As far as I've been able to determine these caches are just legacy goo that servered no purpose and inflated memory use - but if you think you see any fallout from this change please report it.

As for pending ZideStore work:

Need to fix ctags for a PROPFIND with a depth greater than zero.
Test Tobias' patch concerning recurring appointments that jump daylight savings boundaries.
Change zOGI fault codes to make them more useful; the idea is to use similar HTTP error codes where appropriate.
Check out efficiency of authentication in OGoContextManager which I believe may also contain a useless cache. Also I occasionally see logging that makes is appear as if a request was authenticated twice.
Find out why large collections of vCards fail to load when using Novell's excellent Evolution PIM.

Comments, criticisms, and patches welcome!

GtkBuilder & IronPython

2009-09-01T07:50:00.003-04:00

Apparently GtkBuilder is now working with IronPython according to an article over on Mindtrove. What is GtkBuilder? It is a replacement for the aging libglade library, see the article "GtkBuilder has landed" for a good explanation. GtkBuilder support for IronPython means you can now develop .NET Gtk# applications in Python in a RAD fashion. Current versions of Glade support the new GtkBuilder library.

Consonance 0.0.9p

2009-09-01T06:57:00.004-04:00

I've uploaded Consonance 0.0.9pre to the downloads site on Google Code. And announced it on Freshmeat. This version uses the shiny new LINQ backend. Contact and Enterprise operations, including the Contact CRUD window, should be stable. That was the focus of 0.0.9. Hopefully I can get 0.0.10 out the door by this weekend: the goal for 0.0.10 is to re-enable the main task lists and stabilize the Task window. I've already got the task lists loading and as an extra re-enabled the busy-pulser on the main window. Patches welcome!

Best of...

2009-09-01T06:42:00.003-04:00

Apparently it is "Best of" season over at InfoWorld. I'd be interested in what they think is the best but their lists are basically unnavigable - I hope they didn't use any of the Best open source developer tools to design that catastrophe of a presentation. It is way too much work to see their results, but if you have patience and the minimum 25" display necessary to watch the content among the sea of adds:

At the recent BarcampGR 2009 someone asked why I would choose to develop applications on LINUX? The answer is pretty simple: GNOME is an excellent desktop. My question: Why would someone choose to use something else, especially a platform as spartan as Microsoft Windows? Out-of-the-box a GNOME desktop provides an full-suite of top-notch applications:

Manage your music and media with Banshee.
Manage your photos with F-Spot.
Keep notes, and have a Wiki for free, with Tomboy.
For e-mail, calendaring, contact management, and all that use Evolution. Except no substitutes here, certainly not that miserable runt of a mail client from the Mozilla project. No Open Source mail client, and no proprietary one, can rival Evolution. Evolution supports CalDAV, GroupDAV, Groupwise, and even Exchange.
Open Office. What more needs to be said? Open Office rises to the challenge.
Beagle will index all your data allowing you to do comprehensive searches. Once you start using this feature being without it feels like such a waste of time.
This one may seem a bit esoteric, but you know the truth: you should be more diligent about encryting important files and signing e-mails, etc... But it is such a pain. I admit I was pretty lacadazical about such things. But with Seahorse installed it becomes more natural. Now I can encrypt and decrypt files in Nautilus with a right-click as well as decrypt and encrypt on-the-fly in gedit. Seahorse also provide a very nice UI for managing all those @&^$&@@) keys. You can even remove password entries that you have stored in the GNOME keyring.

So, seriously, why use something else? All the day-to-day needs are covered with high quality applications. And we haven't even talked about the awesome development tools like MonoDevelop.

If you don't make mistakes, you're doing it wrong.

2009-08-14T08:38:00.003-04:00

This certainly applies to Computer Science and even general hackerdom. Maybe this is a good response to list trolls who attack Java, Mono / .NET, virtualization, etc... at the slightest provocation. Because presenting them with facts and performance numbers doesn't seem to matter. But they probably wouldn't get the joke anyway. Oh, well.

As for making mistakes, just look at some of my code! I must be doing it very right.

Hg ignore for MonoDevelop Gtk# Projects

2009-08-11T14:13:00.002-04:00

When using Hg (Mercurial) to do version control for Gtk# applications developed using MonoDevelop you'll want to create an .hgignore file in the root of your repository with the following contents:

syntax: glob
*/gtk-gui/*.cs
bin/Debug/*
*.userprefs
*.usertasks
*.pidb

This will make sure Hg ignores all the meta-data and temporary files created by MonoDevelop and its built in Gtk# GUI designer. Otherwise these files change constantly and either always appear as modified or, if commited, clog up the revision history.

Consonance now on Hg

2009-08-10T15:35:00.002-04:00

I have switched the Consonance project on Google Code from subversion to Mercurial. After using Sourceforge's Hg support for OpenGroupware COILS and then hacking on Consonance with little to no Internet access... subversion just had to go. To clone the Consonance repository just do a:

hg clone https://consonance.googlecode.com/hg/ consonance

Consonance Progress, at last.

2009-08-10T07:58:00.002-04:00

While on vacation I had to time to address some unresolved issues left over from the last revamp of the OpenGroupware .NET client assembly and Gtk# application [Consonance]. For the client searching now works again - that was a stupid bug where the callback was never set. Also fixed up numerous bugs in the Gtk# client so Contacts can be updated including company values, addresses, and telephone numbers. After five years of off-and-on hacking I think Whitemice.ZOGI.Backend is now stable and seems to perform very well.

Operations from the work queue now time out, so if a callback is lost the work queue doesn't hang.
Responses to background priority requests are delayed until there is no higher priority work; previously responses to RPC calls where all processed at the same priority regardless of the priority of the original request.
ContactDetailForm is now an IStorableEntityForm.

User Friendly

2009-07-29T16:28:00.004-04:00

Because Microsoft Windows is user friendly and cares about you - it is blocking itself. Has Microsoft had an epiphany?

Service BASIC HTTP Authentication with Python

2009-07-26T17:14:00.003-04:00

While building the core parts of OpenGroupware COILS I noticed there don't appear to be any example of providing HTTP's BASIC authentication scheme anywhere on the interweb. Or any other authentication schemes for that matter. To remedy that here is a rough outline, including the example code, of how it is implemented in COILS. First spin up an HTTP server:

import BaseHTTPServer

class HTTPServer(BaseHTTPServer.HTTPServer):
pass

from coils.net.handler   import HTTPRequestHandler
from coils.net.server    import HTTPServer
....
HTTP_HOST = 'localhost'
HTTP_PORT = 8080
httpd = HTTPServer((HTTP_HOST, HTTP_PORT), HTTPRequestHandler)
httpd.serve_forever()

Your app will almost certainly provide a custom HTTPRequestHandler to delegate the requests to whatever logic your application provides. In the BaseHTTPRequestHandler a GET request is handed to do_GET, a POST request to do_POST, etc... In our case the object that handles the request will decide what to do based on the request type so we channel all requests to our generic process_request method. process_request looks up the object targeted by the request via marshall_handler() [not shown] and then calls that objects do_request method. The important part for authentication is to catch the exception raised by the object handling the request if that object thinks the request is not authenticated and requires authentication. In this case that is the COILS' AuthenticationException; your application has to provide something equivalent. In order to make the client try again with authentication you need to send a "WWW-Authenticate" header telling the client to use basic authentication and what realm to use. See RFC2617 for details on Basic authentication in general

class HTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):

def process_request(self):
  """Respond to a request"""
  try:
    """ find the object mapped to the specified request
        The marshall_handler() is a COILS specific things so it isn't
        shown in this example. """
    handler = self.marshall_handler()
    handler.do_request(self)
  except AuthenticationException, err:
    """ An AuthenticationException has an error code of 401
        We need to add an authentication header so the client will know
        to respond the the failure with the appropriate credentials
        TODO: Provide a digest realm once digest authentication is supported
    """
    self.send_response(err.error_code())
    self.send_header('WWW-Authenticate', 'Basic realm="OpenGroupware COILS"')
    self.end_headers()
    self.wfile.write('Authentication failure')
  except CoilsException, err:
    # An Coils Exception has an error code of 500
    self.send_response(err.error_code(), err.error_text())
    self.end_headers()
  except Exception, err:
    # Yikes, something generic web very wrong
    self.send_response(500, err)
    self.end_headers()

def do_GET(self):
  """Respond to a GET request."""
  self.process_request()

def do_POST(self):
  """Respond to a POST request"""
  self.process_request()

In order to actually process the authentication request COILS uses an Authenticator object. Our DBAuthenticator object provides basic authentication against accounts with passwords stored directly in the database [verses accounts from LDAP or trusting an external authorization mechanism such as Kerberos]. Explanation for the steps to process a Basic authentication operation have been added to the code example:

from base64 import b64decode
from crypt  import crypt
class DBAuthenticator(Authenticator):

def _authenticate(self, context, request):
  Authenticator._authenticate(self, context, request)
  authorization = request.headers.get('authorization')
  if (authorization == None):
    raise AuthenticationException('Authentication Required')
  (kind, data) = authorization.split(' ')
  if (kind == 'Basic'):
    # Authentication method is  "Basic"
    (username, _, password) = b64decode(data).partition(':')
    """ This method provided by the parent class goes to the ORM and
        retrieves the account object for the specified username. It
        will throw an authentication exception if no such username is
        found (user entered it wrong?) or a generic Coils Exception if
        multiple objects match the username (that doesn't make sense) -
        either will stop the authentication process, but the authentication
        exception should reprompt the client to try again. """
    account = Authenticator._getLogin(self, username)
    secret = account.password
    if (secret == crypt(password, secret[:2])):
      # Password matches, user is authenticated
      self.loginId = account.objectId
      self.login   = account.login
    else:
      # Password does not match, authentication failes
      raise AuthenticationException('Incorrect username or password')
  else:
    # Authorization header indicated an authentication type other than BASIC
    CoilsException('Unsupported HTTP Authenticated Mech')

See, that easy easy.

"Oh, Boy!" & "Ditto"

2009-07-23T12:53:00.003-04:00

So it finally happened, what was in my opinion inevitable, Microsoft made a code contribution to the LINUX kernel. And, whew, when that happened you immediately knew the crazies were going to start coming out of the woodwork like orcs in the mines of Moria after you innocently dropped that stone down the well. Here they are!. What a bunch of loons. Fortunately the real Open Source community has those ready with a quick rebuff; I agree completely with David Schlesinger on both the interpretation of what is happening and what the response should be:

"I say enough is enough. We should put our collective feet down as far as strident demands from complete non-participants in the community go. We should say "no" to Faux FLOSS Fundamentalism..." [link]

Contribute or go home! My pacifism previously expressed in Tired Old Arguements was misguided.

GRLUG OpenGroupware Presentation

2009-07-08T19:46:00.003-04:00

Last night I presented on OpenGroupware to the Grand Rapids LINUX User's Group.

Presentation File

Multiple Interfaces On Windows

2009-07-08T06:37:00.003-04:00

A great post over on Ivan Zlatev's BLOG on how to get a working configuration of multiple network interfaces on a Microsoft Windows workstation. There probably isn't anyone who has been forced to use that runt of an OS and not been frustrated by how it handles network connections.

Database Changes to OpenGroupware v5.5

2009-07-08T06:29:00.004-04:00

For those building from trunk, or currently pulling packages from OBS, there is a new table required as of r2256.

CREATE TABLE ctags (
entity VARCHAR NOT NULL,
ctag INTEGER NOT NULL DEFAULT 0
);
INSERT INTO ctags (entity) VALUES ('Person');
INSERT INTO ctags (entity) VALUES ('Enterprise');
INSERT INTO ctags (entity) VALUES ('Date');
INSERT INTO ctags (entity) VALUES ('Job');
INSERT INTO ctags (entity) VALUES ('Team');

This is part of adding ctag support to OGo (specifically ZideStore). ctags allow a client to *very* quickly detect if the contents of a collection have changed; such as the /public/Contacts folder. Thus avoiding doing a PROPFIND on the entire folder to determine if a re-sync is needing. ctag support is not complete or working yet, but without this table a server post-r2256 will fails some operations with a database error.

The subversion repository contains an update script for migrating a v5.4 database to v5.5.

Announcing OpenGroupware COILS

2009-07-06T19:13:00.001-04:00

I've create a new project on the shiny new SourceForge: OpenGroupware COILS. This is "a re-implementation of the groupware functionality provided by OpenGroupware.org. COILS is parallel installable and is entirely compatible with the Objective-C version 5.5 of OpenGroupware.org". COILS is developed in Python for reasons explained on the Wiki. If you'd like to participate in the development of COILS please join the COILS mail list. I also hang out on the OGo IRC channel on FreeNode. And no, this does not mean I am going to stop working on the Objective-C OGo. But I believe there is a need for a refreshing of the OGo platform (BTW, there is also OGo/J) and there are some ideas I want to explore that really can't be done [easily] in the existing OGo/ObjC codebase.

Python XML-RPC Presentation

2009-06-29T19:26:00.002-04:00

Presented on using XML-RPC with Python to the Grand Rapids Python Users Group (GRPUG).

Presentation File

While it is admittedly a runt of an RPC solution I've found XML-RPC to be very useful and have used it extensively in both my work on OpenGroupware and other projects.

Logging USEFUL Web Access To PostgreSQL

2009-06-16T15:08:00.003-04:00

It is pretty easy to configure syslog to log system messages to a PostgreSQL database; and that sure beats flat files. With some GRANT/REVOKE goodness this setup can be made quite secure - only allowing the message import process to insert messages and denying the ability to everyone else to modify the contents of the database. But all that data is still pretty ugly to deal with as the messages are just long strings; so some method is needed to break down that "data" into "information". I had the specific need to do this with SQUID weblogs. I already had syslog logging to a PostgreSQL database so the first step was just to have SQUID use syslog to log access. That is easily accomplished like:

cache_access_log syslog:LOG_LOCAL4

After a restart SQUID messages show up in the LOCAL4 facility (table: facility_local4) with the priority of "info". Then I created a table to hold the parsed messages:

CREATE TABLE proxy_audit (
timestamp TIMESTAMP,
elapsed INT,
source VARCHAR(40),
status VARCHAR(15),
size INT,
method VARCHAR(15),
url VARCHAR(128),
domain VARCHAR(60),
identity VARCHAR(25),
mimetype VARCHAR(45));

Last a trigger on the LOCAL4 log parses incoming messages into this table:

CREATE OR REPLACE FUNCTION p_proxy_audit() RETURNS trigger AS '
BEGIN
IF (new.hostname = ''hod-a'' AND new.priority = ''info'') THEN
INSERT INTO proxy_audit (timestamp, elapsed, source, status, size,
method, url, domain, identity, mimetype)
VALUES(''epoch''::TIMESTAMP + (split_part(split_part(new.message, '' '', 1), ''.'', 1)::INT) * ''1 second''::INTERVAL,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 1)::INT,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 2),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 3)::VARCHAR(15),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 4)::INT,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 5)::VARCHAR(15),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 6)::VARCHAR(128),
split_part(split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 6), ''/'', 3)::VARCHAR(60),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 7)::VARCHAR(25),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 9));
END IF;
RETURN new;
END
' LANGUAGE plpgsql;
CREATE TRIGGER t_proxy_audit AFTER INSERT
ON facility_local4 FOR EACH ROW
EXECUTE PROCEDURE p_proxy_audit()

Once you get allot of data in your table, which happens quickly, you'll probably discover you need an index.

CREATE INDEX proxy_audit_i1 ON proxy_audit(identity, timestamp);

Now looking at a user's web usage is straight-forward. Finally.

Tired Old Arguements

2009-06-15T08:23:00.002-04:00

After a rather peaceful period it seems the anti-Mono patent-paranoia machine is ramping up again; I've seen numerous posts and references to oh-so-evil Mono in the last week. For the intelligent life forms out there I present two BLOG posts which knock these kind of trolls out of the water (assuming they can be bothered to read):
* Here we go again – why Mono doesn’t suck
* How To Defend Against Software Patent FUD
And this one is good too although more general than the other two:
* Lots of GNOME/Mono FUD Lately
But I do wonder if the old advice "Never argue with a fool, onlookers may not be able to tell the difference" is the most appropriate response to these guys. Just keep on coding and let them rant; code and working solution speak for themselves.

BTW, on a personal note, the Sylvann Inn is a very nice and affordable place.

OBS Packages Updated

2009-04-12T08:33:00.003-04:00

OpenGroupware packages on the OBS have been updated to r2207. This revision provides improved GroupDAV version 1 compatibility and adds GroupDAV version 2 support. The upside of those improvements is that the new ZideOne Outlook plugin should work well with this revision (or later) of OpenGroupware. Numerous enhancements and bug fixes in Logic and zOGI are also included.

NOTE: The GroupDAV version 2 specification has not been published yet.

Bug#394

2009-03-12T11:48:00.004-04:00

OpenGroupware revision 2181 closes Bug#394. It is now possible to set a default to determine the default storage backend for new projects. (Wow, with GNUstep based stuff one ends up using the term "default" so often in different ways....)

To select database as your default project storage backend:
Defaults write NSGlobalDomain OGoDefaultProjectStorageBackend Database

To select the filesystem as your default project storage backend:
Defaults write NSGlobalDomain OGoDefaultProjectStorageBackend FileSystem

Most people I assume will want "Database". Remember that default values are CASE SENSITIVE!

If you (at least on r2181) attempt to create a project without selecting FileSystem/Database you'll get a popup warning: "Please specify your project storage!" If the default is set then the preferred storage backend is preselected (and can be changed).

Aside: If you use FileSystem projects make sure you have the SkyFSPath default set and that the referred to directory has the correct permissions.

Packages Complete

2009-03-04T20:22:00.006-05:00

OpenGroupware packages have been completely built on the build service for CentOS5, Fedora 9, openSUSE 10.3, RHEL 5, and SLES10. The Fedora 10 and openSUSE 11.x packages needs some additional work.

Repositories at:

Initial testing on CentOS 5 seems pretty positive, after a:
$ yum install ogo-meta ogo-database-setup

Only glaring bug is that ogo-database-setup doesn't actually populate the database schema so that still needs to be done by hand:

$ cat pg-build-schema.psql | psql -h localhost -U OGo OGo

Then restarting the services gives you [as far as I can tell so far] a fully working OGo install.

P.S. Make sure your getting the most current packages from the repository. mod_ngobjweb in particular should be dated March 4th; with the previous version the web resources (icons, etc...) for the WebUI won't be found at the configured path.

Setting the TimeZone in VMware's VIMA Appliance

2009-02-24T15:47:00.004-05:00

After importing the VIMA appliance the time zone is Pacific time. When going to change the time zone you'll discover that they include neither redhat-config-date or timeconfig. So one has to go about it old-school:

[vi-admin@vima ~]$ date
Tue Feb 24 12:37:01 PST 2009
[vi-admin@vima ~]$ sudo rm /etc/localtime
[vi-admin@vima ~]$ sudo ln -s /usr/share/zoneinfo/America/Detroit /etc/localtime
[vi-admin@vima ~]$ date
Tue Feb 24 15:40:59 EST 2009

"What Carriers Aren’t Eager to Tell You About Texting"

2009-01-04T12:15:00.001-05:00

Excellent New York times article about cell carriers and the true cost of text messenging. As usual with telecom: the customer is getting robbed.

Almost there

2009-01-04T11:46:00.003-05:00

The ogo-meta package now almost installs on CentOS5 except that we don't have the ogo-environment or the ngobjweb packages. OpenSUSE 11.0 and 11.1 have some problems building anything beyond ogo-gnustep_make; OpenSUSE 10.3 builds up to the same point as CentOS5.


$ yum install ogo-meta
...
---> Package ogo-meta.i386 0:1.1-6.7 set to be updated
--> Processing Dependency: mod_ngobjweb for package: ogo-meta
--> Processing Dependency: ogo-environment for package: ogo-meta
---> Package postgresql.i386 0:8.1.11-1.el5_1.1 set to be updated
---> Package postgresql-libs.i386 0:8.1.11-1.el5_1.1 set to be updated
--> Finished Dependency Resolution
Error: Missing Dependency: ogo-environment is needed by package ogo-meta
Error: Missing Dependency: mod_ngobjweb is needed by package ogo-meta

From 72 to 56

2009-01-02T11:22:00.001-05:00

A default minimal install of CentOS5 leaves you with a fairly busy box. If you just login on the console you'll find about 72 running processes. This includes what I consider to be some pretty oddball stuff like "rpcgssd" and "rpcidmapd". How many sites actually make use of these services? A simple run through of:


chkconfig irqbalance off
chkconfig isdn off
chkconfig portmap off
chkconfig rpcgssd off
chkconfig rpcidmapd off
chkconfig sendmail off
chkconfig yum-updatesd off
chkconfig cups off
chkconfig gpm off
chkconfig mdmonitor off
chkconfig nfslock off
chkconfig smartd off
chkconfig autofs off
chkconfig bluetooth off
chkconfig acpid off
chkconfig pcscd off

- leaves one with 56 processes after boot and allot less open ports. Some of these make sense (like acpid) but I'm running in a VM so I don't want those either. Would be nice if the default state of CentOS5 was a little cleaner.

OpenGroupware Package Repositories

2009-01-01T20:50:00.002-05:00

Currently working on getting OpenGroupware (and related GNUStep-make, SOPE, & ngobjweb) packages built of the Novell Build Service. Repositories at:

Not everything working yet, but getting there.

GroupDAV TB 0.90pre16

2008-10-22T17:43:00.001-04:00

Version 0.90pre16 of the SOGo GroupDAV connector for Thunderbird is now available. This version has been tested against our OGo test instance and has successfully synchronized a folder of 19,161 vCards. Updates also seem to work although that is not yet as well tested. Please report any feedback to the GroupDAV maillist.

OpenGroupware r2150 & RSS Feeds

2008-09-21T17:56:00.001-04:00

As of r2150 OpenGroupware now supports the following RSS 2.0 feeds:

projectActions - Reports actions on tasks assigned to projects of which the user is a member either directly or via team membership.

http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/delegated-actions-rss

toDoActions - Reports actions on tasks of which the user is an executor either directly or via team membership.

http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/project-actions-rss

delegatedActions - Reports actions on tasks created by the user.

http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/todo-actions-rss

By default the 150 most recent task actions are included in the feed; this value can be adjusted by including a "limit" value in the URL.

.NET, OpenOffice, & Evolution

2008-06-13T18:30:00.001-04:00

Over on tirania.org is a article on OpenOffice-based applications with Mono and MonoDevelop:

A couple of years ago, Michael Meeks and the OpenOffice community ported the bridge to work with Mono which allows developers to create OpenOffice based solutions using any of the Mono programming languages (C#, Boo, IronPython, IronRuby, F#, VB, Nemerle and so on).

But even if the engine existed, it was not properly installed in the system and getting a C#-based OpenOffice solution required lots of Unix skills, the kind of skills that would likely be in short supply by those that interested in OpenOffice automation. We fixed this in this last development cycle, so now a Novell OpenOffice installation will have everything you need.

Michael Hutchinson, one of our MonoDevelop hackers has put together the missing pieces to simplify the process. He has created the solution templates necessary to create these solutions, and packaged them as a Mono.Addin for exiting MonoDevelop users.

Despite some issues with the exposed API (described in the article) this is pretty exciting stuff. In conjunction with the possibility of writing Evolution plugins using Mono we are closer to having a fully-managed desktop environment with an integrated [or integratable] suite of first-class tools. I'm very much looking forward to the next release of MonoDevelop and hoping some of this stuff gets included.

Widgets Widgets Widgets

2008-06-07T18:04:00.001-04:00

It has been an exciting couple of weeks for the usually quite, and overlooked, world of Gtk# developers. First there was an excellent article on CodeProject about developing custom controls. This was the first in-depth Gtk# article I've seen on CodeProject. Then there was the release of Holly widgets. Holly widgets is a set of very attractive reusable Gtk# widgets that should make developing Gtk# applications more convenient. We are especially excited about the HDateEditor widget; finally we have a good date & time picker! Following up that was a new release of Medsphere.Widgets. Medsphere.Widgets now contains a very nice 2D graph. As icing on the cake is the long overdue support for GObject property registration. Wow, when it rains it pours.

LinuxWorld Interview

2008-03-29T18:05:00.001-04:00

LinuxWorld interviewed me recently concerning OpenGroupware.org. The interview is part of their "Spotlight on Open Source in business" series.

One moment as a gas bag...

2008-03-27T18:32:00.001-04:00

Regular visitors will know that I post strictly concerning technological issues; I've got little use for, or desire to be, a pundit of any kind at all. I have a firm belief in experts and on the great debates of our time I've got little or nothing original to add - not that I'd expect anyone to listen even if I did (Why would you? Who am I?). But I just couldn't resist posting a link to OOXML, looking forward. It is concerning Open Source, etc... so it isn't far off the mark of the usual content here, but it is more "big picture" then I usually indulge. Basically the gist of it is here:

But most importantly, it is a time for all of those strong advocates of open standards to stop talking, and start walking. I look forward for all that energy that went into discussing the pros and cons of OOXML to join an open source project and start contributing code, documentation, support, create support forums, file good bug reports and help us make free and open source software better.

The world as a whole could do with less talk. I suppose this applies to almost every conceivable realm from priests and politicians all the way down to the lowly coder. But it is the one thing I can think of that simply can't be said often enough: less talk, more do.

Ok, thank you for that minute, I now yield the podium to the next gas bag.

Footnote: Specifically, an expert is someone whose spent a decade plus dedicated to the study of something, someone who has made some topic their life. They are the kind of people smug freshman like to roll their eyes about. But, in my opinion, it is those all those boring [and focused] and forgotten people who have given us the myriad blessings of technology and the sacred gift of the most unlikely thing of all - civilization. Thank you.

getAuditEntries added to zOGI r2095

2008-03-05T18:37:00.002-05:00

The getAuditEntries method was added to the zOGI API as of r920, and added to the ZideStore trunk in r2095. getAuditEntries provides the ability to retrieve the audit entries from the server's database that have occurred since a specified entry. Using this feature a service can page through server changes and synchronize some repository; this allows functionality equivalent to that provided by MOGIMon but without a back-door database connection. Since audit records are serialized with integer ids in the OpenGroupware database this acts very much like the uSNChanged attribute provided by Microsoft Active Directory.

List for Funambol GroupDAV connector

2008-03-03T18:34:00.002-05:00

A list has been created specifically for the Funambol GroupDAV connector. You can subscribe here. Be at bit patient as the site in questions is a little slow. It isn't clear if archives of this new list are available.

NetGear GS716T

2008-02-28T18:36:00.001-05:00

I've recently worked with a pair of NetGear GS716T switches and want to warn everyone off from buying these switches. While the price is pretty low and performance is very good, the feature-set is entirely bogus. NetGear advertises these as "smart" switches with VLAN, SNMP, etc... Only all these features are useless as the web interface to configure the switch simply doesn't work and no command line interface (via either telnet or SSH) is provided. If you attempt to configure the switch from FireFox you will be constantly prompted for the password, and you can't actually get past that to make any configuration changes. On the other hand, if you use IE6 you can sign in to the switch but the interface is extremely unstable. On one switch I was unable to even set an IP address, upon clicking "Apply" it would close the browser and revert to using the address it acquired via DHCP. On the other switch it took the IP address but attempting to configure port based VLANs resulted in erratic behavior ranging from hanging, to displaying raw HTML, to renaming all the other VLANs and removing all the ports assigned to other VLANs. The administrative interface of the NetGear GS716T is simply unacceptable; NetGear has clearly not performed even a minimum of quality-control or testing on the GS716T.

OpenGroupware & PostgreSQL 8.3

2008-02-19T18:39:00.001-05:00

PostgreSQL 8.3 not longer performs automatic casting of INT to TEXT when INTs are compared to character types. This change is documented in the release notes. This change causes a database exception to occur in OpenGroupare's ACL processing. The specific error is:


ERROR:  IN types character varying and integer cannot be matched

ACL queries like the following cause the exception because they contain a condition that compares string value to an array of integers: auth_id IN ( 9981, 9991,... where auth_id is a VARCHAR(255) value.

A full ACL query might look like:


SELECT DISTINCT B.company_id  FROM person AS B WHERE (( ( B.firstname ILIKE '%%yser%%' ) OR ( B.name ILIKE '%%yser%%' ) ))  AND (( B.owner_id = 10100 ) OR ( ( B.is_private = 0 OR B.is_private IS NULL ) AND (  ( 0 = ( SELECT COUNT(*) FROM object_acl WHERE object_id = B.company_id ) OR 0 < ( SELECT COUNT(*) FROM object_acl WHERE object_id = B.company_id AND action = 'allowed' AND permissions LIKE '%r%' AND ( auth_id = 10100 OR auth_id IN ( 9981, 9991, 10003, 11530, 55000, 77210, 142730, 142780, 142850, 393280, 423400, 445160, 949410, 949540, 955770, 970730, 970990, 1096320, 9153790, 9153880, 9154000, 10597021, 10735169 ) ) ) )  ) )) AND (B.is_template_user IS NULL OR B.is_template_user = 0) AND (B.db_status <> 'archived') LIMIT 50

The solution to this problem is to modify the schema of the object_acl table so that the values are integer types; this is possible since the auth_id and object_id only store object ids, which are integers, regardless of the fact that the field type was VARCHAR(255). Originally it was thought that some other values might be used in ACL processing, such as LDAP DNs, but only support for object ids exists and will exist in the forseable future.

The table can be modified using ALTER TABLE:


ALTER TABLE object_acl ALTER COLUMN object_id TYPE INT USING  object_id::int;
ALTER TABLE object_acl ALTER COLUMN auth_id   TYPE INT USING  auth_id::int;

This change will be included as the official schema in OGo 5.4. With this change OpenGroupware should work without any issues in conjunction with PostgreSQL 8.3. The thread discussing this issue is available in the archive.

Cyrus IMAP and the Ampersand

2008-02-10T18:41:00.001-05:00

An interesting thread has occurred on the Cyrus IMAPd mailing list regarding the lowly ampersand. Apparently in the encoding used by IMAP (a "modified UTF-7 encoding") the ampersand ("&") requires escaping. So it works fine to have subfolders whose name contains an ampersand as IMAP clients, of course, use the appropriate IMAP encoding. But if your username (internally the name of your INBOX) contains an ampersand you can run into trouble. this is because when the existence of the INBOX is checked the authentication request isn't necessarily encoded in the same way. So if you use an IMAP server user names should be kept in conformance with the IMAP encoding, or more simply, stick with the GOODCHARS defined in Cyrus' mboxname.c

Improvement to SOPE's PostgreSQL Adaptor

2008-02-09T18:43:00.001-05:00

Do you have lots of errors like:


Feb 17 20:39:28 ogo-zidestore-1.5 [14569]: ERROR(+[NSCalendarDate(PostgreSQL72Values) valueFromCString:length:postgreSQLType:attribute:adaptorChannel:]): unexpected string '2007-03-13 15:38:41.420456+00' for date type 'DATE', returning now (expected format: '2001-07-26 14:00:00+02')

in your ZideStore error log? This is because the time & date parser in SOPE's PostreSQL adaptor didn't understand the milliseconds portion of the value. As of SOPE r1601 this should be corrected. SOPE now just ignores the millisecond value. See the diff for details.

LINQ Provider for db4o

2008-02-06T18:44:00.001-05:00

Jb Evain has created LINQ provider for db4o. This could be potentially great for db4o developers as LINQ is [finally] an answer to the infernal "foreach...if". It is also a great example of the power of extension methods. These days it seems like every time I wish C# could do X, then an article about how X is already available magically appears on monologue.

SASL Failure with jabber-net

2008-01-14T20:27:00.002-05:00

When using the jabber-net assembly a SASL failure doesn't invoke OnAuthError as one would intuitively expect; SASL failures invoke the generic OnError method with a jabber.connection.sasl.SASLException. Just an interesting bit of trivia to avoid a potential gotcha.

Consonance Gets A Calendar UI (r470)

2007-12-31T20:28:00.001-05:00

Consonance now has a calendar UI for the scheduler tab of the main window. This is a significant milestone; up until now that feature has been very conspicuously absent. I had hoped to be be able to port some code from some other projects but ended up writing the entire widget from scratch. Consonance displayed a calendar event for the first time on 2007-12-30 at 16:33 (GMT). A screenshot is here and also attached to this post. This code is included as of r470.

OGo Getting Caffinated

2007-12-26T20:30:00.001-05:00

There has been discussion on the OpenGroupware lists about the work Helge has been doing lately in regards to Java and OpenGroupware. The relevant threads are here , here, and the message that started it all is here.

Note:Some of the messages don't thread in the archive correctly, so it might be worthwhile to just look at the archive for December, 2007 @ here

The distilled information is, roughly:

Helge does most of his programming these days in Java; what he calls the "new stuff". He doesn't see himself doing allot of work on the Objective-C code base in the future. However the SOPE code base is also used by other projects and no one expects that to become unmaintained.

A substantial portion of the Logic and Document API modules of OGo have already been re-written in Java for some of Helge's own work. He intends to release that code in Q1/Q2 of 2008. And he expects to port much more or the remaining Logic functionality in the future.

The ported code is 1:1 compatible with the Objective-C Logic and Document API, therefore the two code bases can co-exist and operate on the same database concurrently.

Currently the development is done with Eclipse and the components are hosted in Jetty although some of the pieces have been used in a Tomcat environment as well.

The Java code actually ends up being faster than the Objective-C code, mostly because it is a ground-up rewrite and one assumes sheds allot of cruft that the older code base has accumulated over the years.

This opens up lots of opportunities to support additional databases and graft in allot of functionality made available by other Java assemblies.

The key messages is here:

---snip---
I dislike talking about thing being done before they are actually
done, but at least I am shifting most OGo developments to JOPE/Java.
...
What this implies should be discussed on discuss, just wanted to raise
the point that the existing codebase might be become important for
starters in the future ;-)
---snap---
Anyways. Feel free to ask any questions ;-)
The current situation is that I have rewritten a big part of OGo Logic/
DocAPI in Java in 2007. I have a few custom play-stuff apps running on
top of that (eg a HighRise like UI) and I'm currently doing a midsize
project which uses that.
Most likely I'm going to commit the code to a public repository
beginning of next year. Its much more performant than LSLogic (not
because its Java, but because its designed from scratch ;-) and also
has some neat enhancements.
There is no real UI yet, but something which resembles the "old" OGo
should be pretty easy to do.
[Note: my work so far attempts to be fully compatible with ObjC OGo,
ie database schema and doc storage]
It could be the basis for something like OGo 6.0 ;-)
Hence, most likely I won't put a lot of effort into the Objective-C
parts besides maintenance until JOPE replicas work.
One of the biggest issues I have with ObjC is that the Apple version
and the GNU ones are bound to diverge (eg Apple with GC and ObjC 2.0
features ...), and that gnustep-base goals do not match my
requirements on API stability and OS integration (and lF is a dead end
anyways).
I'm not perfectly happy with all this but I think for 'developers'
things will get much easier. Machines are fast enough for Java now and
Java itself is now GPL, removing that dependency.

Personally I think this is a very positive development. I am not a real fan of Java, and I generally think the Java environment is simply a mess. But it works, has lots of support, and good solutions can be created in Java. And it does open up a new horizon of possibilities for OpenGroupware. The reliance on an odd environment like Objective-C has, in my opinion, been probably the single largest impediment to OpenGroupware's wild success. Hopefully, Helge will get around to releasing that code sooner rather than later.

Now I just have to get Jetty to actually work on an openSUSE box...

zOGI Now In ZideStore

2007-08-21T20:31:00.001-04:00

The main branch of zOGI is now included in Zidestore (as of r1995). So if you use nightly builds the next time you update you should automatically have the zOGI protocol bundle - no installation required. The Google Code repository will now be used as the experimental branch; those wanting a stable zOGI should use the one included in ZideStore. For now the documentation remains on the Google Code Wiki.

GroupDAV Syncronization For Mac OS/X

2007-07-11T20:33:00.001-04:00

BionicMessage has released a GroupDAV connector for Mac OS/X. Read the release announcement for more information.

Horde project leader mentions GroupDAV

2007-05-18T20:34:00.001-04:00

Jan Schneider mentions support for WebDAV, CalDAV, and GroupDAV in her blog, here. GroupDAV support in Horde adds thousands, if nor more, compatible servers to the world. Also interesting is the mention of DIMP an AJAX-ified version of IMP. Great news for the Horde and GroupDAV projects.

Shortcuts for crontab

2007-05-14T20:35:00.001-04:00

An interesting BLOG post over here. Nothing really dramatic, but some useful and apparently undocumented shortcuts supported in ISC crontab: @reboot, @yearly, @annually, @monthly, @weekly, @dialy, @midnight, and @hourly. It seems odd that I can google these shortcuts and find them in archived manual pages, but the manual pages on my local machines don't mention these shortcuts.