2014-11-21

Paramiko's SFTPFile.truncate()

Paramiko is the go-to module for utilizing SSH/SFTP in Python.  One one the best features of Paramiko is being able to being able to SFTPClient.open() a remote file and simply use it like you would use a local file.  SFTPClient's open() returns an SFTPFile which is a file-like object that implements theoretically the same behavior as Python's native file object.
But the catch here is file-like.  It is like a file, except when it is not like a file.
The non-likeness I encountered concern's SFTPFile's truncate method.  The native Python file object has a truncate method where the size is option, if not specified it will default to the current offset of the file.  However SFTPClient's truncate method requires the size to be specified.  Failure to specify an size value results in a TypeError exception concerning incorrect number of parameters.

2014-10-05

Creating A CUPS Backend

I gave this presentation at the Grand Rapids Barcamp (August, 2014)
http://www.wmmi.net/documents/ACUPSBackend.pdf
The presentation covers the basics for creating a print back-end on a CUPS print server.  The process is very straight forward and allows you to setup, essentially, print to anything - even your own application.  Printing to an application can have a surprising number of applications given that nearly everything no matter how old or basic can print.

2014-10-01

Testing A WINS Server

On a CIFS/SMB domain the WINS service is critical for proper function [some things use WINS, some things use DNS, it is terribly confusing, but it is what it is].  DNS is relatively easy to test and you will likely know right away if it isn't working.  But before adding those new DCs to your dhcpd.conf file -
option netbios-name-servers 192.168.1.78, 192.168.1.79, 192.168.1.65;
 - it would be nice to be equally confident WINS is operating as expected.
To that end the Samba package provides a useful tool nmblookup which support querying of a WINS server much like host/nslookup does for DNS.

$ nmblookup --unicast=192.168.1.65 --recursion arabis-red
querying arabis-red on 192.168.1.65
192.168.1.58 arabis-red<00>
$ nmblookup --unicast=192.168.1.78 --recursion arabis-red
querying arabis-red on 192.168.1.78
192.168.1.58 arabis-red<00>
$ nmblookup --unicast=192.168.1.79 --recursion arabis-red
querying arabis-red on 192.168.1.79
192.168.1.58 arabis-red<00>

The --unicast option specifies the IP address of the WINS server; but this option is only effective in the expected way if used with the [oddly named] --recursion options. Without --recursion the --unicast will not cancel use of the NetBIOS name resolution logic.
One other useful feature of nmblookup is the --translate option.  This option invokes a reverse DNS search on the IP address returned by the WINS query - allowing one step verification that WINS and DNS are on the same page.

$ nmblookup --unicast=192.168.1.79 --translate --recursion arabis-red
querying arabis-red on 192.168.1.79
ARABIS-RED.example.com, 192.168.1.58 arabis-red<00>

2014-09-28

Remotely Restarting The Management Agents On ESXi 5.x

The ESXi management agents can be restarted from the host's console -  which is not very convenient.  Fortunately they can also be restarted remotely using the SSH & ESXi Shell services - but these are not enabled by default.
In Virtual Center select the host, then the Configuration tab. In the security section select Services and Properties in order to enable "ESXi Shell" and "SSH". For both services perform a manuals start.
Once the SSH service is active you can SSH to the host as root and then restart the hostd and vpxa services like any standard LINUX System-V service.  This may cause the host to temporarily grey out in Virtual Center; it will come back automatically in a few moments.
awilliam@linux-86wr:~> ssh root@172.31.7.126
Password:
The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools. Please
see www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.
~ # /etc/init.d/hostd restart
watchdog-hostd: Terminating watchdog process with PID 9034
hostd stopped.
hostd started.
~ # /etc/init.d/vpxa restart
watchdog-vpxa: Terminating watchdog process with PID 9336
vpxa stopped.
~ # Connection to 172.31.7.126 closed.
After restarting the management agents the SSH and ESXi services should be stopped in the same manner they were started.
It would be nice if in VMware's "powerful system administration tools" they just had a button labeled "Restart Management Agents".

2014-04-25

Uncoloring ls

By default on every recent shell the output of ls is colorized. This is a great feature - but it makes using terminals that use a non-standard [not(background==black)] color-scheme awkward.  Things just disappear;  try reading directory name displayed in yellow on a yellow background.  It is difficult.
How this colorization gets setup in openSUSE is that that the ls command is aliased to "ls --color=auto".  You can see this aliasing using the alias command.
[fred@example ~]# alias
alias cp='cp -i'
alias l.='ls -d .* --color=auto'
alias ll='ls -l --color=auto'
alias ls='ls --color=auto'
alias mv='mv -i'
alias rm='rm -i'
alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'
So a simple way to turn this off is: unlias ls
There you go - no more annoying colors!  But they will be back the next time you login. Is this alias created in ~/.bashrc  - nope. ~/.bash_profile - nope. /etc/bashrc - nope. /etc/profile - nope, errr.... well, sort of.  The script /etc/profile runs every script in /etc/profile.d which ends in .sh.  Within /etc/profile.d is ...drumroll... colorls.sh which indeed does alias ls='ls --color=auto'.
To disable colorization - the aliasing of ls upon login:
mv /etc/profile.d/colorls.sh /etc/profile.d/colorls.sh.x
Now that the script does not end in .sh /etc/profile will not run it.
When you start a new interactive login[*1] shell bash runs: /etc/profile, then ~/.bash_profile, ~/.bash_login, and then ~/.profile.   Generally ~/.bash_profile will run ~/.bashrc, and generally ~/.bashrc will run /etc/bashrc.  And, of course, when /etc/profile runs it runs each of the scripts which match /etc/profile.d/*.sh.  It is almost comical; who says UNIX can't dance?

[*1] The shell distinguishes between login and secondary shells - a secondary shell is invoked by an existing shell, whereas a login shell is the first shell.  It also distinguishes between interactive and non-interactive shells - like those running a script.

2014-03-13

AIX Printer Migration

There are few things in IT more utterly and completely baffling than the AIX printer subsystem.  While powerful it accomplishes its task with more arcane syntax and scattered settings files than anything else I have encountered.
So the day inevitably comes when you face the daunting task of copying/recreating several hundred print queues from some tired old RS/6000 we'll refer to as OLDHOST to a shiny new pSeries known here as NEWHOST.  [Did you know the bar Stellas in downtown Grand Rapids has more than 200 varieties of whiskey on their menu?  If you've dealt with AIX's printing subsystem you will understand the relevance.]
To add to this Sisyphean task the configuration of those printers have been tweaked, twiddled and massaged individually for years - so that rules out the wonderful possibility of saying to some IT minion "make all these printers, set all the settings exactly the same" [thus convincing the poor sod to seek alternate employment, possibly as a bar-tender at the aforementioned Stellas].
Does IBM really truly not provide a migration technique?  No.  But I now present to you the following incantation [to use at your own risk]:
scp root@OLDHOST:/etc/qconfig /etc/qconfig
stopsrc -cg spooler
startsrc -g spooler
rsync --recursive --owner --group --perms \
  root@OLDHOST:/var/spool/lpd/pio/@local/custom/ \
  /var/spool/lpd/pio/@local/custom/
rsync --recursive --owner --group --perms  \
  root@OLDHOST:/var/spool/lpd/pio/@local/dev/ \
  /var/spool/lpd/pio/@local/dev/
rsync --recursive --owner --group --perms  \
  root@OLDHOST:/var/spool/lpd/pio/@local/ddi/ \
  /var/spool/lpd/pio/@local/ddi/
chmod 664 /var/spool/lpd/pio/@local/ddi/*
chmod 664 /var/spool/lpd/pio/@local/custom/*
enq -d
cd  /var/spool/lpd/pio/@local/custom
for FILE in `ls`
 do
   /usr/lib/lpd/pio/etc/piodigest $FILE
 done
chown root:printq /var/spool/lpd/pio/@local/custom/*
chown root:printq /var/spool/lpd/pio/@local/ddi/*
chmod 664 /var/spool/lpd/pio/@local/ddi/*
chmod 664 /var/spool/lpd/pio/@local/custom/*
Execute this sequence on NEWHOST and the print queues and their configurations will be "migrated". 
NOTE#1: This depends on all those print queues being network attached printers.  If the system has direct attached printers that correspond to devices such as concentrators, lion boxes, serial ports, SCSI buses,.... then please do not do this, you are on your own.   Don't call me, we never talked about this.
NOTE#2: This will work once.  If you've then made changes to printer configuration or added/removed printers do not do it again.  If you want to do it again first delete ALL the printers on NEWHOST.  Then reboot, just to be safe.  At least stop and start the spooler service after deleting ALL the printer queues.
NOTE#3: I do not endorse, warranty, or stand behind this method of printer queue migration.  It is probably a bad idea.  But the entire printing subsystem in AIX is a bad idea, sooo.... If this does not work do not call me; we never talked about this.

2014-03-11

gedit's amazing External Tools

In a few recent conversations I have become aware of an unawareness - an unawareness of the awesome that is gedit's best feature: External Tools.  External Tools allow you to effortlessly link the power of the shell, Python, or whatever into an otherwise already excellent text editor yielding maximum awesome.  External Tools, unlike some similar features in many IDEs is drop-dead simple to use - you do not need to go somewhere and edit files, etc... you can create and use them without ever leaving the gedit UI.
Plugins tab of the Preferences dialog.

To enable External Tools [which is a plugin - as is nearly every feature in gedit] go to the Plugin tab of Preferences dialog and check the box for "External Tools".  External Tools is now active.  Close the dialog and proceed in defining the tools useful to you.
With External Tools enabled there will be a "Manage External Tools" option in the Tools menu.  When in the tools menu not there is also an "External Tools" submenu - every external tool you define will be available in the menu, automatically.  The list of defined tools in that submenu will also include whatever hot-key you may have bound to the tool - as you likely will not remember at first.
Manage External Tools Dialog

Within the Manage External Tools dialog you can start defining what tools are useful to you.  For myself the most useful feature is the ability to perform in-place transformations of the current document; to accomplish this set input to "Current Document" and Output to "Replace Current Document".  With that Input & Output the current document is streamed to your defined tool as standard input and the standard output from the tool replaces the document.  Don't worry - Undo [Ctrl-Z] still works if your tool did not do what you desired.
What are some useful External Tools?  That depends on what type of files and data you deal with on a regular basis.  I have previously written a post about turning a list of value into an set format - that is useful for cut-n-paste into either an SQL tool [for use as an IN clause] or into a Python editor [for x=set(....)].   That provides a simple way to take perhaps hundreds of rows and get them into data very simply. 
Otherwise some tools I find useful are:

Format JSON to be nicely indented

#!/bin/sh
python -m json.tool

Use input/output settings to replace current document.

Open a terminal in the directory of the document


#!/bin/sh
gnome-terminal --working-directory=$GEDIT_CURRENT_DOCUMENT_DIR &


Set the input/ouput for this action to "Nothing"

Remove leading spaces from lines


#!/bin/sh
sed 's/^[[:blank:]]*//'


Use input/output settings to replace current document. 

Remove trailing spaces from lines


#!/bin/sh
sed 's/[[:blank:]]*$//'


Use input/output settings to replace current document.

Keep only unique lines of the file


#!/bin/sh
sort | uniq


Use input/output settings to replace current document. 

Format an XML file with nice indentation


#!/bin/sh
xmllint --format - -


Use input/output settings to replace current document.