2010-02-10

Passing a column set to an SQLalchemy query

In implementing the List method of the EntityAccessManager provided by the Contacts bundle in OpenGroupware Coils it seemed like it would be very efficient to allow the consumer to request what set of attributes it needed;  for instance, if a WebDAV client's PROPFIND request didn't ask for a given property, why request the corresponding attribute in the query?  Especially since the result set for PROPFIND queries are frequently very large [on the order of 20,000 records or so].  But how to pass a set of attributes as a parameter?

def List(ctx, attributes):
  ...
  db = query(attributes).filter(....)
  return db.all

does not work;  where attributes is a list, or a set of more than one attribute. Some fiddling around and I discover that the following works:

def List(ctx, *attributes):
  ...
  db = query(*attributes).filter(...)
  return db.all()

...
ctx = AssumedContext(10100)
a = BundleManager.get_access_manager('Contact', ctx)
a.List(ctx, (Contact.object_id, Contact.version))

That single asterisk is the trick.

Configuring OpenLDAP's dynlist in cn=config

Step#1: Make sure the dynlist module is loaded on the server.  What modules are loaded are typically controlled by the "cn=modules{0}, cn=config" object.  If your server doesn't have a modules object it probably isn't loading any dynamic modules.  Create an LDIF file and import it.  Our "cn=modules{0}, cn=config" looks like:

dn: cn=modules{0}, cn=config
olcModuleLoad: {0}accesslog.la
olcModuleLoad: {1}auditlog.la
olcModuleLoad: {2}constraint.la
olcModuleLoad: {3}dynlist.la
olcModuleLoad: {4}memberof.la
olcModuleLoad: {5}ppolicy.la
olcModuleLoad: {6}refint.la
olcModuleLoad: {7}seqmod.la
olcModuleLoad: {8}syncprov.la
olcModuleLoad: {9}sssvlv.la
olcModuleLoad: {10}translucent.la
olcModuleLoad: {11}unique.la
olcModuleLoad: {12}back_monitor.la
olcModulePath: /usr/lib/openldap2.4
objectClass: olcModuleList
cn: modules{0}

NOTE: Double check that the "olcModulePath" is the absolute path to the directory containing the OpenLDAP modules.

Step#2: Check that your schema has the groupOfURLs objectclass defined.  If you attempt to configure the dynlist module without that schema available you still crash slapd.  To define the dynamicGroup schema (if it is missing) you can import the following LDIF into your cn=config:

dn: cn=dynamicGroup, cn=schema, cn=config
olcObjectClasses: {0}( 2.16.840.1.113730.2.33         NAME 'groupOfURLs'   
     SUP top STRUCTURAL         MUST cn         MAY ( memberURL $ businessCat
 egory $ description $ o $ ou $                 owner $ seeAlso ) )
olcAttributeTypes: {0}( 2.16.840.1.113730.1.198         NAME 'memberURL'   
     DESC 'Identifies an URL associated with each member of a group. Any type
  of labeled URL can be used.'         SUP labeledURI )
objectClass: olcSchemaConfig
cn: dynamicGroup

Step#3: Create an olcOverlayConfig object in the scope of your Dit database.  For example, our Dit is the 1st HDB database on the server so the appropriate object is:

dn: olcOverlay=dynlist,olcDatabase={1}hdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynamicList
olcOverlay: dynlist

Step#4: Now you are ready to actually use the dynlist module.  A common use-case is to create dynamic mail alias objects;  with dynlist you don't need to maintain mail aliases, they will automatically contain everyone who matches the relevent criteria.  Provided you use the traditional nisMailAlias objectclass in order to define mail aliases adding the attribute -
olcDlAttrSet: {0}nisMailAlias labeledURI rfc822mailmember:mail
 - to "olcOverlay=dynlist,olcDatabase={1}hdb,cn=config" will enable dynamic mail aliases.  Specifically any nisMailAlias containing a labeledURI attribute will be expanded by the query specified in that attribute.  The results of that query will be rewritten by the optional rfc822mailmember:mail clause which will rename the mail attributes resulting from the query into the rfc822mailmember attribute required by consumers of the nisMailAlias objects.  So rather than populating the mail alias object with rfc822mailmember attributes manually, you extend the object with the auxilliary labeledURIObject objectclass and define the query in the labeledURI attribute.

dn: cn=gr_parts, ou=ListAliases, ou=Aliases, ou=Mail, ou=SubSystems, o=Morrison Industries,c=US
mail: gr_parts@morrison-ind.com
labeledURI: ldap:///ou=People,ou=Entities,ou=SAM,o=Morrison Industries,c=US?mail?one?(&(morrisonactiveuser=Y)(objectclass=morrisonuser)(departmentNumber=*P*)(morrisonbranch=GRD))
objectClass: nisMailAlias
objectClass: top
objectClass: labeledURIObject
cn: gr_parts

The object will immediately populate with rfc822mailmember attributes derived from the mail attribute of those objects matching the specified filter: "(&(morrisonactiveuser=Y)(objectclass=morrisonuser)(departmentNumber=*P*)(morrisonbranch=GRD))".  The third parameter, "one", of the URI is the scope of the query so only objects immediately subordinate to "ou=People,ou=Entities,ou=SAM,o=Morrison Industries,c=US" are candidates for the filter.

2010-01-21

Twython Presentation Uploaded.


My presentation on Twython (a Python Twitter API provider) and integrating Twitter services into OpenGroupware Coils has been uploaded to the GRPUG forum..  This covers the basics of using Twython such as authentication, tweeting, retrieving time lines, searching, etc...  
Because Twitter rate limits connections [as well they should] if you are going to provide Twitter access to a large numbers of users, potentially NAT'd behind a single IP, it is recommended that you create some type of middle-ware to proxy and cache your Twitter access - the Twitter support in OpenGroupware Coils is one example of how to do that.

Python Curses In Action, even on AIX.


So, what if you have an old COBOL application that you want to integrate with some web services or a website? You'd need a "green-screen" application that can take that COBOL applications temporary output file, send it up to the web service, wait for a response, replace the temporary file, and then return control to the COBOL application (or whatever "green screen" application comes next). It sounds simpler than it is in practice: what if the web service takes awhile to complete [or has human involvement on the other end?!] or the web service isn't available right at the moment you try to send your request? For a real production environment your "green screen" application would need to deal with all of those things.
And you need that "green screen" application to work on both LINUX and AIX!


Fortunately pware provides Python 2.6 for AIX, including curses! The curses library and respective Python module provide a surprisingly easy way to create professional looking TUI (Text User Interface) applications like the solution required in our example. This particular application takes them temporary file and submits it via a WebDAV PUT operation to the workflow engine provided by OpenGroupware Coils. The route in the workflow engine reformats the dreadful output of the COBOL application into the required format (using the format support in route Read and Write operations). The client detects the route is complete by watching the URL specified in the header of the initial PUT, and then downloads the required data. If anything goes wrong along the way the client can retry, or provide the user the option to abort. All along the way the client provides detailed feedback to the user and a familiar dialog-box and prompt interface when feedback is required.

2009-10-11

XAML designer for Monodevelop?

If I could pick one single thing to add to my development toolchain it would be a XAML designer. These days I do 90% of my development in the excellent MonoDevelop IDE (that last 10%, maintaining the OpenGroupware.org Objective-C code base, I do in gedit) Now that MonoDevelop supports Python I can do all my work on OpenGroupware Coils there as well. But I just can't imagine developing web applications using the tricycle: HTML, CSS, Javascript. Every web development presentation I see leaves me with a single distinct impression: "What a hack!" Web development is simply too cumbersome and tedious. The future of the web is clearly web services (REST, SOAP, XMPP, and XML-RPC) combined with web backed applications using Silverlight/Moonlight and Flash. My money is on Silverlight/Moonlight being the eventual winner. And today when I brought up Monologue I see: "xaml designer for monodevelop". It isn't complete, maybe not even usable yet, but it certainly is encouraging; maybe someday I'll be able to return to web application development, and riding a 10-speed rather than a rusty old tricycle.

2009-10-04

Wow, Slashdot covers some real FOSS!

It seems like it has been ages since this last happened - but Slashdot has a blurb about a real FOSS Story: Interview With Jeremy Howard of FastMail.fm. FastMail both runs and develops services based on Cyrus IMAP. Cyrus IMAP is Open Source, has an amazing depth of features, stunning performance, and is extremely standards compliant. It is very refreshing to see some attention given to a real Open Source solution that has been serving user for a decade. This in contrast to a not-open not-standards-compliant solution whose name you can probably guess - with whom Slashdotters appear to be utterly smitten [well beyond the point of clear thinking, but this is (a) Slashdot and (b) a normal side-effect of being smitten by something.] I'm a big fan of Cyrus IMAPd which has been very feature-rich and stable since at least 1999: you get delayed expunge, full text indexing, service-side filtering (SIEVE), shared folders, partitioning, replication, modification sequences, message expiration, and more. Other IMAP/POP servers are still trying to bolt these features onto themselves. If you aren't using Cyrus IMAPd I recommend you take a look, I honestly don't know why anyone would choose anything else.

2009-09-13

Ohio LinuxFest 2009 registration deadline is looming..

Ohio Linux Fest 2009
Free and Open Source Software Conference and Expo
Columbus, Ohio : September 25-27, 2009

Registration Deadline: September 18, 2009

The Ohio LinuxFest 2009 registration deadline is September 18, 2009. If you have not registered yet, please hurry over to http://www.ohiolinux.org and register today.

Join us for the seventh annual Ohio LinuxFest conference to celebrate 40 years of Unix. We have an exciting line up of talks and this year the conference extends from Friday to Sunday.

Friday, September 25, features professional training courses (Professional package registration required) and a course for Linux beginners which includes a refurbished computer, with Linux pre- installed for you (Quick Start package required). A series of conference talks and other sessions will be held on Friday as well.

Saturday, September 26, features the expo and four conference tracks. We have a great line up of speakers; Doug McIlroy, Shawn Powers, Dr. Peter Salus, and many more.

Two certification examinations will be held on Sunday the 27th. Linux Professional Institute will host an LPI certification exam, and the BSD Certification Group will offer the BSDA certification exam for Ohio LinuxFest attendees.

Four registration packages are available for the Ohio LinuxFest this year.

The Enthusiast Package is available at no cost for students, enthusiasts, and those that want to come to the event to find out more about Free and Open Source software.

The Supporter Package is available again at a low cost of $65.00 to support the event. As a measure of appreciation, the supporter package includes lunch on Saturday and a commemorative Ohio LinuxFest tee-shirt.

The Quick Start Package is available for $250.00, which in addition to access to the events on Saturday, includes a full day of Linux Basics training on Friday and a refurbished Linux computer to take home.

The Professional Package is available for $350.00, which in addition to Saturday's activities, also includes access to the one day OLFU tutorials on Friday.

The Ohio LinuxFest welcomes people from all 50 states and international participants. We have had participants from Canada, England, Argentina, Brazil, and Australia in years past. Note that the Ohio LinuxFest is a 501c3 non-profit, volunteer organization. All proceeds are used for conference costs.

2009-09-11

New OGo Help Desk Feature

Enhancement Bug#2027 "Allow help desk users to create tasks on behalf of users" has been resolved as of r2274. This update requires a database schema change -

ALTER TABLE job ADD COLUMN owner_id INT;

UPDATE job SET owner_id = creator_id;

The database scripts pg-build-schema.psql and pg-update-1.x-to-5.5.psql have been updated. NOTE: Be careful not to run this part of pg-update-1.x-to-5.5.psql more than once or you risk modifying actual data in your database if you use the OGoHelpDeskRoleName default.

This feature allows a member of the team whose named is defined in the OGoHelpDeskRoleName default to create tasks with an owner other than themselves. Delegated and archived task lists now display tasks based on owner rather than creator. By default the owner is the creator so this has no effect on normal task behaviour. Currently the help desk feature, setting of the owner to another user, is only available via the zOGI API. The modifications to the zOGI API are documented on the Task entity.

If your help desk team name contains spaces be sure to use proper quoting - Defaults write NSGlobalDomain OGoHelpDeskRoleName '"all intranet"' - or you may not get the results you expect. Obviously if you set OGoHelpDeskRoleName to "all intranet" all users will be able to create tasks on behalf of other users.

2009-09-08

Let The Season Of Patching Begin.

It is that time of year again: later sunrises, earlier sunsets, and more hacking. I've commited my first patch for the coming season of gloom - cache removal as previously discussed back in the days-getting-longer part of the year. [Note: Pipermail archives strip attachments, but you can find the patch on the Nabble archives]. This makes trunk r2270 bumping libZSBackend.so to version 5.5.94. As far as I've been able to determine these caches are just legacy goo that servered no purpose and inflated memory use - but if you think you see any fallout from this change please report it.

As for pending ZideStore work:
Comments, criticisms, and patches welcome!