Advocating Tcl/Tk for GUI development

I am really surprised of very few uses of Tcl/Tk for GUI building. It’s a super-simple programming language, you can learn the syntax in only one day if you are accustomed to programming in other languages. Portable across Unix/Win/Mac OS. The Tk toolkit allows you to build GUIs directly by defining the hierarchical relations between GUI elements (widgets) and attaching event-response functions with parameters. It is an object-based rather than object-oriented system and this preserves simplicity, easy of learning and maintaining. There is no need for a graphical GUI builder too, everything can be scripted with much less code to write compared to other solutions. I’d like to see a desktop environment fully based on Tcl/Tk, at least on Linux. I really do not like the complexity of Qt and Gtk. GUI development should not be done in system languages like C and alike, but in high-level scripting languages. There is no need to scale up, every GUI is used only by one user. Sun decided to market Java, a system language, but without any operating system based on it, so any advantage of being low level is just a disadvantage. Using Java for some high-level task like GUI building is certainly possible but very far from optimal. But without starting a language war, we should just recognize no language is best at everything. This is why is often worth using more than one language in the same project. Tcl/Tk can be embedded in various system languages, so the GUI can be implemented in Tcl/Tk and the rest in the hosting language, if more appropriate. I wish there were more Tcl/Tk users and more support, but I must acknowledge the reality: there are many languages in computer science, some generic, some specific, and they are often misused. Programmers are responsible to make the right technical choices. If managers do that without considering alternatives, then you have a disaster: a lot of money lost and development headaches, which could be avoided.

Building a cheap software sound mixer with Linux

You will need at least two USB mics. Use the alsamixer command. Press F6 to select the sound card and F4 to adjust capture volumes. E.g. condenser microphones should be operated at lower volume than dynamic ones, because they are more sensitive to noise. To start and stop recording from both sources at practically the same time by using the following shell script:

#!/bin/sh
arecord -q -D hw:2,0 -t wav -f s16_le -c1 -r44100 track1.wav &
arecord -q -D hw:3,0 -t wav -f s16_le -c1 -r44100 track2.wav &
read -n1 -r -p "Press any key to stop recording..."
kill $(jobs -p)

replace hw:3,0 and hw:2,0 with your actual audio sources or add more. You can list all audio sources with:

arecord -l

Device names may change order depending on the USB connection order, so for example if you are recording vocals and guitar, you will have to check which track will contain which by playing each track separately. This is something you will want to do anyway to check how the recording from each mic went.

To mix, use Audacity: for each track, select a silence part and use Effect/Noise Reduction/Get Noise Profile. Then type J K to select the whole track and again Effect/Noise Reduction but this time hit OK to reduce noise for that track only based on the profile. You have to do this for each track separately. Then use the gain knob to reduce or increase volume of each track. Select all tracks by clicking on each track panel while you keep shift pressed then go to Tracks/Mix and Render. In the same menu, you can also mix and render to another track, if you do not want to loose the original ones.

The eggs queue

Rationale: I was a computer expert for over 10 years, but I was never been able to find and intelligent job, whereby I could apply what I know about computer science. I have always been ordered about by people much more stupid than me, who wanted me to use the wrong tool for the job, just like knocking a nail with a chainsaw or worse with your bare hands. Lately I become an independent organic farmer and, surprisingly, I can finally apply all I know about computers, although there aren't any in my farm!

One application is a circular queue to differentiate between fresh and old eggs. My chickens now lay about 5-6 eggs per day. I only eat two eggs a day and put the rest aside to make sweets or for possible guests.

I always want to eat eggs on a FIFO basis: First egg In, First egg Out, meaning I always use the oldest one, so to avoid some eggs are eaten too late and go bad. I reused an egg cardboard of an egg carton I took from the supermarket and numbered each of its eggcups, up-bottom and left-to-right.

Starting from 1, I keep adding eggs to the end of the queue and take ones to eat from the start. Initially, the queue head is the lowest number filled by an egg, while its tail is the highest number, but when you reach the end of the queue - 30 in my case - this is not true anymore. E.g. in this picture

the oldest egg is number 20 and egg number 1 comes just after egg number 30! This is why it is called a "circular" queue: although the cardboard shape is squared, you have to imagine its egg cups as arranged in a circle, so that 1 really comes after 30 physically. Queued eggs all make one single circular piece of "snake", which changes its length and goes around the circle clockwise or counterclockwise as eggs are added and taken, depending on your convention.

Now look at this picture:

Here I have 29 eggs and only one eggcup free, which happens to be number 26. Oldest egg is thus number 27 - this is the one I would eat first, while latest laid is 25. You can't add another egg in eggcup 26, since if you do it, you would not be able to tell fresh from old eggs, that is the beginning and end of the queue. If you put some markers, you could fill the queue completely, but for the sake of simplicity, I prefer to lower the queue capacity by 1 rather than having to use head and tail markers. This way queue head and tail is located visually, with no need for markers. Of course, your mind implements an algorithm to do it. If you are a programmer, can you formalize it in a programming language?

BTW If you want to store more than 30 eggs, of course you need another cardboard. You can just have two separate pieces or cardboard and remember which is the first one, or you can glue them together so that you have a 30*2=60-egg circular queue or even more if you have so many eggs to store - and chickens to feed!

The bottom line: chickens are more intelligent than IT managers. Oh well, do not take it personally if you are one of them.

Alert me by email when some Unix log files became non empty, without having sendmail enabled!

Here's a bash script that does that. Let the server alert you when something's wrong. Probably the tiniest STMP client implementation possible :)

In bold what you need to change to make it work for you.

#!/usr/local/bin/bash
# A script to alert the programmer by email when
# one or more log files in a dir become not empty.
# To be used in a crontab, e.g.:
#
## use /usr/loca/bin/bash to run commands, overriding the default set by cron 
#SHELL=/usr/local/bin/bash
## run once a the end of the day
#0 20 * * *     $HOME/bin/alert_not_empty.sh

dir=/path/to/logs/
files=(file1_log file2_log)

notempty=()
for file in "${files[@]}"; do
  if [ -s "$dir/$file" ]; then
    notempty=( "${notempty[@]}" "$file" )
  fi
done

if [ ${#notempty[@]} -ne 0 ]; then
  notempty=$(printf "%s\n" "${notempty[@]}")

  # You can use mail or mailx, but what if sendmail is disabled for remove
  # delivery? (e.g. for security reasons). Let's help ourselves a bit LOL
  # Use "host -t mx yourispdomain.name" to find out yourispmailserver
  exec 1<>/dev/tcp/gmail-smtp-in.l.google.com/25
  a=$(cat <<mailend yourserver.domain.name
MAIL FROM: <yourusername@yourserver.domain.name>
RCPT TO: <youremail@yourispdomain.name>
DATA
From: yourusername@yourserver.domain.name
To: youremail@yourispdomain.name
Subject: message subject
You have to check the following log files:
 
$notempty
.
QUIT
MAILEND
  )
  
  # Needed to write whole commands in one go.
  IFS='
'

  # This converts a to an indexed array b.
  declare -a b=($a)
  
  for x in "${b[@]}"; do
    # We always sleep after each line, even in data mode.
    # This is slower, but makes the implementation simpler.
    # For slow connections, 1 second may not be enough.
    sleep 5
 
    # Send command. Be sure to use DOS end of lines
    # or some SMTP servers may choke on the commands.
    echo -ne "$x\r\n"
  done
fi

Here's a more advanced versions: it has debugging output, actually reads replies and is much faster when sending the body of the email, since it doesn't sleep after each line has been sent:

function read_reply {
  # DEBUG.
  echo -n "sleeping..."

  # Sleep to make sure a reply has arrived.
  sleep 10

  # DEBUG.
  echo -en "\nread reply..."

  # Read reply.
  read -u 3

  # DEBUG.
  echo

  # DEBUG: print replies.
  echo "$REPLY"
}

# Needed to write whole commands in one go
IFS='
'

# This converts a to an indexed array b!
declare -a b=($a)

# At the beginning, we are in command mode.
# The only other mode is data mode.
cmd=true

# Read server greeting to begin.
read_reply

for x in "${b[@]}"
 do
   # Send command. Be sure to use DOS end of lines.
   echo -ne "$x\r\n" >&3

   # DEBUG: print commands.
   echo "$x"

   [ "$x" = "." ] && cmd=true

   $cmd && read_reply

   [ "$x" = "DATA" ] && cmd=false
 done

HTML generation: templates, DSL and embedded scripts compared

Most web sites aren't 100% dynamic like Gmail or Facebook, even in this Web 2.0 era. There is usually one person who writes the HTML+CSS code and, if needed, a bit of client-side code (JavaScript), but he often uses WYSIWYG tools and is not a good programmer. Then another dude, a specialized programmer and data base expert, adds the server side code where it is needed.

PHP fits this division of labour nicely by providing a tag to embed PHP code directly into HTML.

It works, but it's an unreadable mess, quite hard to maintain, especially because of the verboseness of HTML. To address this problem, a lot of template systems have been developed for PHP, in an attempt to separate the HTML from the PHP code. Instead of inventing a new template language and writing a parser and an interpreter/compiler for it, the most flexible template kits just use PHP itself as a template language!

The PHP template file is just require-d or include-d by the PHP script that implements the page business logic. Such PHP templates are editable by webmasters using WYSIYWG HTML editors in a way that allows them to change most presentational details without screwing up any code.

E.g. in a typical PHP-based template you will find HTML code interrupted by <?php ?> sections that either print out values of variables, define a loop or choose HTML code to print conditionally, usually no more than that. Most WYSIYWG HTML editors display them using small icons, as they were external objects embedded into HTML. All that a webmaster has to take care of is, for instance, make sure that all PHP code sections that print values in a loop are put inside the two outer sections that define the loop itself. Any surrounding HTML code can be changed at will.

There are some disadvantages of using templates, though: they are slower and use more central memory when processed on each request. Caching can help ease this drawback, but for highly dynamic resources is not always easy and effective.

E.g. if you are formatting a big table with some frequently-changing data pulled out from a database, you will need to store them all in memory, then pass this structure to a template engine, which will send the generated HTML page to the web server, either at one bound or, if you're lucky, piecewise, but, you still need to keep all your data in memory, if not the HTML code too.

You can't pass a row at a time as soon as it is returned by the database from the business to the presentational logic. In order to do that you would need to run these two layers as separate processes or threads, but then you have to face the complicated issue of process/thread synchronization as well as buffer the data exchanged. It is so overkill that I do not know of any "scalable" template engine that uses multi-processing or multi-threading. In simple cases, which are the majority, it would be slower than single-process template systems.

This is a scenario where templates don't scale. If you were using PHP embedded into HTML, the output could be flushed more often, virtually after each row is printed, and, in the case of a slow database or overloaded server, the user does not have to wait for the whole page to be generated server-side to actually see or begin to see something - assuming there is some browser support for partial table rendering and such. And the user is not always a rendering browser, it could be web crawler like that of a search engine.

Other than embedding PHP code directly into HTML, whether you use templates or just embed PHP code into a tag, you can also generate the HTML code by using another language, that is a DSL which generates HTML. E.g. in Lisp you can use s-exprs so you don't have to close tags - you close parentheses, but at least any editor with Lisp support matches them for you automatically. The result is less messy than HTML code. HTML has a very bad syntax, indeed! By using Lisp you get some syntax checking too, e.g. it is impossible to overlap tags this way - Lisp closing parentheses are all the same, unlike HTML closing tags. By intermixing Lisp forms (that is commands) you can add any dynamic behaviour you want. Lisp provides you with full support for functional programming, that is you can code functions devoid of side effects, which are easy to test and help to implement a modular approach to HTML generation.

If you define your Lisp macro to generate HTML code which returns strings that are concatenated it will use more memory than PHP templates with <?php ?> tags. If you turn them into (write-string) forms, like CL-WHO does, you save memory but it is slower than both some hand-made (format) code to print HTML tags directly from Lisp and, of course, static HTML. Whatever the method you are using, you pay a performance price for printing HTML from your code using a nicer syntax, the cost of converting from the DSL to HTML. Moreover you have to convince the non-programmer webmasters to learn the Lisp syntax and abandon all their WYSIWYG HTML editors. Quite tough, if not impossible, to do on a large scale!

What is more, this DSL-based model works only if the front-end and back-end developer are the same person, with the same skill sets, but, even in this case, it isn't always feasible! E.g. you may need to add a bit of dynamic behaviour to some existent static HTML code. Suppose you don't have time to convert the whole document using your DSL of choice, they do not pay you for that and it would not be worthwhile anyway, since it is a mostly static page. This is why a <?php ?> is not always a bad idea. Even if you have an automatic HTML to DSL convertor, it is still slower to generate all the bulk of the HTML, which is static, using the DSL at each request.

Ideally you should have both ways to generate dynamic HTML code at hand in a server-side language and choose one case by case. If the page is highly dynamic, go for the DSL, otherwise use templates if they give you roughly the same performance as directly embedded PHP.

The problem with PHP is that it does not make easy and efficient to develop flexible DSLs for HTML generation. This is where Lisp, with its powerful macros, can provide an added value. But AFAIK Lisp-based server solutions do not provide anything like the PHP tag <?php ?>, for quick & dirty dynamic additions or PHP-based template implementations, apart from a small project mod_ecl which has been abandoned.

Lista di parole italiane

In Linux, una volta installato il pacchetto aspell e il dizionario italiano aspell-it, ecco come ottenere una wordlist con tutte le parole, le forme verbali, i superlativi, etc (tecnicamente, nel gergo di aspell si dice "espansione dei flag degli affissi"). Si tratta di una lunga pipe, di cui spiego a lato la funzione dei vari pezzi

$ aspell -l it dump master |	dump di tutto il dizionario di base
aspell -l it expand |	espansione dei flag degli affissi
sed "s/[^ ]*'[^ ]* *//gi" |	eliminazione delle forme elise (es. d'amore, bell'uomo, etc)
tr ' ' '\n' |	per avere una parola per linea
sed '/^$/d' |	eliminazione delle righe vuote
sort -fu >italian.wordlist	ordinamento case insensitive, eliminazione duplicati e salvataggio in un file

Il risultato è un file di testo di circa 25 Mbytes, contenente poco meno di un milione e novecentomila parole!

Fate attenzione che gli autori hanno inserito una parola di 50 caratteri, la più lunga: CopyrightGianlucaTurconiDavidePrinaLicenzeLGPLoGPL. Se volete eliminarla, usate nella pipe un grep -v, ma ricordate che la licenza è la GPL!

Notate che vi sono anche molti numeri in lettere. Tolta quella di prima la parola più lunga è di 39 lettere e corrisponde ad un numero: quattromilaquattrocentocinquantaquattro.

Multi-DVD or (multi-CD) spanning backups with tar in Linux

Download

You can download all three source files below with one click if you want to save some cut&paste.

Just extract this file into your home dir, edit .multibak.conf as outlined below and launch multibak:

$ cd
$ tar zxvf multibak-2009-09-10.tar.gz
$ vi .multibak.conf
$ multibak

Setting up the config file

Create a hidden file named .multibak.conf in your home directory and edit the list of directories to back up as it suits you (default is your entire home):

# .multibak.conf

# in kbytes (1 kbyte = 1024 bytes)
# 4508876 is a safe size for ~4.7 GB DVD (single-sided, single-layer)
DISK_SIZE=4508876

# list of files/directories to backup
BAK_DIRS=($HOME)

# tar options:
# -P does not strip leading '/'s from file names
# -h dereferences symlinks - dumps the files they point to
TAR_OPT="-hP"

# cdrecord options:
# -v increment verbosity level
# -dao disk at once mode
# -overburn test it with your drive before enabling (adjust DISK_SIZE also)
# speed=X (X is in 1385 kB/s units): 0 lowest, omit for max
DVD_OPT="-v -dao"

# drive device name
# /dev/dvdwriter is a good general choise; e.g. if you attach an external
# USB drive, udev changes this link to that of the newly connected drive
DEV="/dev/dvdwriter"

# location of tar new volume script
TAR_SCRIPT="~/bin/multibak-tar-script"

# method to gain root privileges needed to record media;
# choose one of: none, su, sudo
SU=sudo

You should also review the other options too. If your system does not come with sudo installed and properly configured make sure to change SU to su.

Install the script

Put this script in your private command directory ~/bin/ and ensure this is on your $PATH environment variable and its executable permission bit is set. I called it multibak:

#!/bin/bash
# multibak: Multi-DVD or (multi-CD) spanning backups with tar

# source the config file
readonly CONFIG=~/.multibak.conf
readonly PROG=$(basename $0)
. "$CONFIG" || { echo "$PROG: config file $CONFIG missing" >&2; exit 1; }

# make temporary archive file name (unique)
readonly export TMP_TAR_FILE=$(mktemp -t $PROG.XXXXXXXXXX) || exit 1
trap "rm -f $TMP_TAR_FILE" HUP INT QUIT TERM EXIT

# make config variables read only and export
# those we need to transmit to the tar script
readonly DISK_SIZE BAK_DIRS TAR_OPT DVD_OPT DEV TAR_SCRIPT
export DEV DVD_OPT TMP_TAR_FILE SU

# archive and burn
tar $TAR_OPT -L $DISK_SIZE -F "$TAR_SCRIPT" -cf $TMP_TAR_FILE "${BAK_DIRS[@]}"
"$TAR_SCRIPT" -last

Another piece is needed, that's the "new volume script", run by tar at end of each media. Name it ~/bin/multibak-tar-script:

#!/bin/bash
# multibak-tar-script: new volume script used by multibak
# See: http://www.gnu.org/software/automake/manual/tar/Multi_002dVolume-Archives.html

[ "$DEV" ] || { echo "This script should not be run directly"; exit 1; }

# close the tray, if left open
eject -t $DEV

# format media, if it's a DVD+-RW
echo dvd+rw-format $DEV

# burn this piece
CMD="cdrecord $DVD_OPT dev=$DEV $TMP_TAR_FILE"
case $SU in
  none) echo $CMD ;;
  su) echo su -c "$CMD" ;;
  sudo) echo sudo $CMD ;;
esac

eject $DEV

[ "$1" != "-last" ] && read -p "Prepare volume #$TAR_VOLUME for backup and hit return: "

Be sure this is executable too otherwise tar will refuse to run it.

Dependencies are obvious. Here they are for CentOS or Arch Linux: cdrecord, dvd+rw-tools.

Start a backup

To make a consistent backup your applications must have been shut down first. Note tar(1) is smart enough and will warn you if a file gets modified meanwhile it gets archived. If you see such warnings it means there is some process mangling with the file.

$ multibak

If you want to do this on a Mac read here.

Pretty print a bunch of code files and save paper

I used to do:

$ find path... -type f | xargs pr -Fe2 | lpr

This works quite well. But when most of the files are short and only a few take more than one page I end up with a lot of blank lines. A waste of paper.

What I want is to omit pagination but retain the headers which allow me to tell where one file ends and another begins. pr(1) options -t and -T do not help, so I came up with my own solution:

#!/bin/sh
# printcode: prints out a bunch of text code files
# expanding each tab to two spaces and marking the
# beginning of each file, without using form feeds.
#
# Public domain
# Author: Antonio Bonifati http://ninuzzo.freehostia.com

# By default pretty print all files in the current directory.
[ "$1" ] || set .

for i in $(find "$@" -type f); do
  echo -e "\n==========> $i\n"
  expand -t2 $i
done

VRML plugins and viewers

VRML97 (Virtual Reality Modelling Language, aka VRML 2.0) is the ISO standard for displaying 3D data over the web. A browser plugin is required. Here are my experiences with some free plugins and players. See also a NIST page about web 3D plugins.

view3dscene (great)

Displays many formats: VRML, X3D, 3DS, MD3, Wavefront OBJ, Collada. Open source, stable, small, quick, fast even on cheap hardware. Easy to install under recent Linux distros. I'm using SVN version and works under 64 bit linux too. I maintain the Arch Linux package. Only one con: it's not a browser plugin.

FreeWRL and OSX poor support (not recommended)

I downloaded FreeWRL for OS X and installed it for using with Safari. The first simple demo I ran was this grey cone.

At the time I was writing there are some annoying bugs: when I open a WRL file in a tab all other tabs in the same windows became black and are thus unusable. This also happens when a VRML scene is embedded in page. Scrolling doesn't work the way it should as well. Another limitation of FreeWRL is that can only have 1 active window in Safari.

First I lived with it by always opening WRL files in a new window. Then I decided to use the FreeWRL application for all testing and development and return to Safari only for a final test. Fortunately in FreeWRL menu there is a File/Reload option. Also make sure to enable the Console Register under Window. Hope Safari support will be better in the future.

But even the FreeWRL application is unstable and slow. This may be because FreeWRL is Java-based :-P This is also why I don't use it anymore.

PS: When you make a mistake in a WRL file you will hear a female voice from FreeWRL actually speaking (!) the error message. Cool at first, but after a a few repetitions of the same errors I've decided to mute the speakers and read errors using the Utility -> Console OSX application.

OpenVRML under Linux (not recommended)

I maintain the ArchLinux package. A pain to compile from sources. Doesn't load complicated worlds and it's slow.

Octaga Player (not recommended)

Not open source. Unstable plugin. Doesn't work with 64-bit Linux. A pain to install for 32-bit Linux. Easy to install in Mac OS X but not working very well.

MyName - A simple IP updater for dynDNS.it

MyName should work with many Unix versions, including Linux, BSD's, Mac OS X. If not let me know.

Download and installation

Latest version is Oct 26, 2008.

Download the myname script, place it in a directory included in your PATH. A suitable location is /usr/local/bin/ for Unix and /usr/bin for Mac OS X. Please ensure the executable bit is set.

Also download the configuration file myname.conf and place it under /etc/. Then edit this file to insert your username and password and the hostname(s) to update.

$ sudo mv myname /usr/local/bin/ $ sudo chmod a+x /usr/local/bin/myname

Go to http://dyndns.it/ if you need to register a new account. Edit myname.conf with a plain text editor. Save and complete install with:

$ sudo cp myname.conf /etc
$ sudo chown root:wheel \
/usr/local/bin/myname /etc/myname.conf

Note: root:wheel is the user and group suitable for Mac OS X and BSD's. Linux usually uses root:root.

Manual usage and testing

At the shell prompt just type:

$ myname

You should see a response similar to this:

good YOUR_IP

You can then try to resolve your host with:

$ host YOUR_HOSTNAME

If all goes well you should have receiveid a message like:

YOUR_HOSTNAME has address YOUR_IP

Automatic usage with modems

BSD, Linux, Mac OS X

As root, edit /etc/ppp/ip-up and add the myname command at the end on a line by itself. This way your IP will be registered every time your ADSL or PTSC connection is brought up by your modem. If this file does not exist please create it as root and ensure it is executable. This is the case with Mac OS X.

You can use these commands:

$ sudo echo myname >>/etc/ppp/ip-up
$ sudo chmod a+x /etc/ppp/ip-up

In Unix installations where sudo is not available, you can use

$ su

to become root. Then issue the same commands as above without the sudo prefix.

Automatic usage with routers

TODO

Changelog

• Oct 26, 2008: first version.