Fail2ban: an enemy of script-kiddies
on 29.04.2007, 05:00
in packages-news
DebConf7 logo
on 26.04.2007, 22:47
in debconf-news
Amarok: listening to music will never be the same
on 25.04.2007, 05:00
in packages-news
Debian Weekly News 2007/05
on 24.04.2007, 00:00
in weekly-news
sshfs: Easy (and secure) access to a remote file system
on 22.04.2007, 05:00
in packages-news
DebConf6 Final report, DebConf7 list of events and reconfirmation
on 20.04.2007, 23:45
in event-announce, news
most: an alternative and powerful pager program
on 20.04.2007, 05:00
in packages-news
enigma: addictive puzzle game with a high dose of dexterity
on 18.04.2007, 05:00
in packages-news
moreutils: A collection of useful command-line tools
on 15.04.2007, 05:00
in packages-news
Debian GNU/Linux 4.0 released
on 08.04.2007, 12:42
in news, release
TinyCA: simple user interface to manage a small CA
on 08.04.2007, 05:00
in packages-news
Debian GNU/Linux 3.1 updated
on 07.04.2007, 22:17
in news, release
Gobby: A Collaborative Text Editor
on 04.04.2007, 05:00
in packages-news
fortunes: Fortune cookies for all
on 01.04.2007, 05:00
in packages-news

Fail2ban: an enemy of script-kiddies

published on Sun Apr 29 05:00:00 2007 in packages-news

I bet there is only a little part of auth.log-aware GNU/Linux users, who has not experienced a pleasure of browsing thousand of lines of the failed authentication attempts. If you do not yet know what to look for in your auth.log, just run:

> zgrep 'Failed password for illegal user' /var/log/auth.log* | wc -l

On the system which I just tried, the result is 125835! since July of 2006. Yeah yeah — 99.999% of those failed logins are due to silly dictionary attacks, which (unfortunately) work in some % of the cases. Are you sure that your password and passwords of all the users on your system are strong enough to survive such an attack?

Also, I guess, there is a (hopefully small) group of system administrators, who experienced a pleasure of DoS attack on their services. Or web-server admins, who have a pleasure to stare at the attempts to access non-existing (most of the time) on the webserver /php/bla-admin.X.Y.bleh.

For both those groups (as well as for other problems too), there is a straightforward solution — just reject (or in other terms - ban) abuser’s IP as soon as you detect an attempt to get an unauthorized access to your box. Unfortunately, we do not stare at the log files 24×7, so we can not react in time. To substitute such a weak part of the chain in this process, i.e. a human operator, Fail2ban tool was created by Cyril Jaquier.

The idea behind Fail2ban is very simple: temporarily or permanently ban an IP which performed multiple undesired actions, such as unsuccessful authentication, access to restricted area, etc.
Originally it was developed to catch illegal SSH login attempts, but later on it grew up into an easily customizable toolkit for speedy reaction on some events (such as detected failed login attempts) recorded in the log files.

In the following sections I will describe a bit more of internals of Fail2ban configuration, but that knowledge is not really required to get the tool working for you. For that, it is sufficient to run “apt-get install fail2ban”. You might like to read the section on jails below if you simply want to enable some additional jails shipped with the Fail2ban package.

Debian/Ubuntu Presence of Fail2ban

Fail2ban is present in sarge from backports.org, and it is native to Etch and Sid. Sarge version in backports is from a 0.6 branch of the Fail2ban, and it has different configuration scheme than current 0.7 (soon 0.8) branch. 0.7 uses split configuration files and orthogonally separates notions of a filter (pretty much a python regular expression with associated set of files) and an action to be taken (banning via iptables/hosts.deny, or sending an email).

Fail2ban is also present in Ubuntu releases since Dapper release.

Configuration

Default configuration in both branches (0.6 and 0.7) enables ssh logins monitoring right out of the box, so no changes are necessary to get Fail2ban running.

If necessary, all changes in the configuration of Fail2ban 0.6.x have to be made in the original configuration file, and sections can be also enabled via command line switch (-e iirc) (N.B. this cmdline option is specific to Debian release of Fail2ban and is not present in upstream version). 0.7 branch uses completely different configuration scheme, and it is very convenient: any change or addition which has to be done in file /etc/fail2ban/X.conf can simply be made in file /etc/fail2ban/X.local — parameters in .local override ones in .conf. This way .conf file stays intact, and during your upgrade there is no necessity to mess with patching config files if they get changed upstream. Since I prefer 0.7 branch, I will describe details of its configuration.

As I mentioned above, 0.7 branch comes with an orthogonal configuration between filters and actions. A filter specifies what to
look for (like a ‘failed login attempt from …’ in auth.log, or a message ‘please brew some coffee, Mike’ in your .xchat/history/private.log), and an action describes possible scenario to play (to ban an IP, or to send an a single packet authenticator to a coffee maker to start brewing a fresh cup of coffee).

Filter

So here is an example of a filter:

> grep -v  '^#' /etc/fail2ban/filter.d/sshd.conf

[Definition]

failregex = Authentication failure for .* from 
            Failed [-/w]+ for .* from 
            ROOT LOGIN REFUSED .* FROM 
            [iI](?:llegal|nvalid) user .* from 

ignoreregex =

“failregex” is a list of python regular expressions (with “” simply be a shortcut for “(?:::f{4,6}:)?(?P\S+)” to match an IP or a host name. “ignoreregex” allows to infiltrate some false positives.

Standard sid Debian installation of Fail2ban comes with filters for various services (ssh, ftp, http), various implementations (exim, postfix; proftpd, pure-ftpd, wuftpd, etc), and for some additional events (normal illegal login in ssh vs DDOS attack on sshd).

If you want to write your own filter to store under /etc/fail2ban/filter.d/blah.conf, there is a very handy helper tool: fail2ban-regex, which can test your regular expression on the existing logfile and tell if you it works fine.

> fail2ban-regex /var/log/auth.log 'Failed [-/w]+ for .* from ‘

Running tests
=============

Use regex line : Failed [-/w]+ for .* from 
Use log file   : /var/log/auth.log

Results
=======

Failregex:
[1] Failed [-/w]+ for .* from 

Number of matches:
[1] 2 match(es)

Addresses found:
[1]
    69.115.175.240 (Sun Apr 01 23:58:20 2007)
    69.115.175.240 (Sun Apr 01 23:58:27 2007)

Date template hits:
2 hit: Month Day Hour:Minute:Second
0 hit: Weekday Month Day Hour:Minute:Second Year
0 hit: Year/Month/Day Hour:Minute:Second
0 hit: Day/Month/Year:Hour:Minute:Second
0 hit: Year-Month-Day Hour:Minute:Second
0 hit: TAI64N
0 hit: Epoch

Success, the total number of match is 2

However, look at the above section ‘Running tests’ which could contain important
information.

Here instead of regular expression to test, you could simply provide the file of you tentative filter.

Action

A typical action for most of the cases would be to ban detected IP of an abuser using iptables, and that action is described in the following Fail2ban action definition:

> sudo grep -v  '^#' /etc/fail2ban/action.d/iptables.conf

[Definition]

actionstart = iptables -N fail2ban-
              iptables -A fail2ban- -j RETURN
              iptables -I INPUT -p
 –dport
 -j fail2ban-

actionstop = iptables -D INPUT -p
 –dport
 -j fail2ban-
             iptables -F fail2ban-
             iptables -X fail2ban-

actioncheck = iptables -n -L INPUT | grep -q fail2ban-
actionban = iptables -I fail2ban- 1 -s  -j DROP
actionunban = iptables -D fail2ban- -s  -j DROP

[Init]

name = default
port = ssh
protocol = tcp

Default action in 0.7 branch of Debian package though is iptables-multiport, which can be used to ban multiple ports at once. Besides it, there are other actions available such as

  • hostsdeny — ban using hosts.deny mechanism
  • shorewall,ipfw — use firewall cmdline interfact to ban/allow an IP
  • mail-* — email about the performed action to a sysadmin

Jail

And now we came to a point where both notions (filter + action) should be used together. “Jail” is the specification containing a filter and desired set of actions to be performed. Here is an example from original upstream version of /etc/fail2ban/jail.conf.

[ssh-iptables]

enabled  = false
filter   = sshd
action   = iptables[name=SSH, port=ssh, protocol=tcp]
           mail-whois[name=SSH, dest=yourmail@mail.com]
logpath  = /var/log/sshd.log
maxretry = 5

In this example, the jail ssh-iptables defines the name of the filter to be used (so the full file name is implied to be /etc/fail2ban/filters.d/sshd.conf). Also it defines the list of actions to be performed: TCP port 22 has to be banned after 5 unsuccessful attempts, and an email has to be sent to yourmail@mail.com informing about such action.

While preparing Debian package of Fail2ban, I tuned up Debian-shipped version of jail.conf so that jail specifications becomes minimalistic, since most often all the jails should perform the same chosen action. If there is a need in a jail-specific action, it can always be specified in “action” parameter of the jail. The same jail in Debian-shipped jail.conf looks like

[ssh]

enabled = true
port    = ssh,sftp
filter  = sshd
logpath  = /var/log/auth.log
maxretry = 6

Since the rest of the jails present in jail.conf are not active by default, desired jails can easily be enabled in /etc/fail2ban/jail.local. Here you can see a part of my locally customized jail.local:

[DEFAULT]

bantime  = 3600
destemail = root@localhost

banaction = shorewall
action = %(action_mwl)s

[apache]
enabled = true
maxretry = 4

[sasl]
enabled  = true

[courierauth]
enabled  = true

# custom jail which used to be not present in shipped jail.conf
[apache-noscript]
enabled = true
port    = http
filter  = apache-noscript
logpath = /var/log/apache*/*error.log
maxretry = 6

Screenshots




Weblinks

DebConf7 logo

published on Thu Apr 26 22:47:00 2007 in debconf-news

DebConf7 has a new logo:

Thanks to Valessio Brito for coming up with it!

Amarok: listening to music will never be the same

published on Wed Apr 25 05:00:57 2007 in packages-news

Entry submitted by Grant Thomas. DPOTD needs your help, please contribute !

Amarok is a fully featured music player well integrated into the KDE environment. Amarok uses a database (SQLite, MySQL, PostgreSQL) delivering fast collection access, and a wide array of searching/sorting methods.

Current Feature list: (credited to http://amarok.kde.org/wiki/What_is_Amarok?)

  • Quick and simple drag and drop play list creation
  • Super eye-candy interface
  • Multiple back ends supported (xine, NMM and Helix)
  • 10 band equaliser
  • Automatic cover art download using Amazon services
  • The unique and powerful Context Browser
  • Automatic play-statistics generation (iRate style)
  • Full lyrics download
  • Funky visualisations from libvisual and XMMS
  • Streaming from any KIO source
  • Cross-fading
  • Fully configurable translucent OSD for track changes
  • K3B (CD-burning) integration
  • KDE integration
  • Style your Context Browser with custom CSS styles.
  • Save space in your Context Browser with collapsible boxes
  • Show the Context Browser without Amarok open through the Konqueror sidebar!
  • Full support for last.fm! Share your music taste with friends on the net
  • Generate dynamic play lists based on last.fm suggestions
  • Support for SQLite, MySQL and PostgreSQL databases ensuring fast collection access
  • Support for iPod®, iRiver® and generic UMS mp3 players with the all new media-browser
  • Powerful scripting interface, allowing for easy extension of Amarok
  • Complete DCOP access
  • Translated into more than 35 languages, thanks to the KDE internationalisation team

Amarok can play many audio formats through one of the back end engines. Formats include mp3, ogg, flac, wma, wav, and others. Basically any file format that the selected engine can play, Amarok can use. For more information, see http://amarok.kde.org/wiki/Audio_Engine_Comparison

Amarok includes full support of last.fm, which allows users to record each track played in an online community. From last.fm, Amarok can bring back similar artists and tracks, as well as other recommended artists / tracks. Amarok also supports last.fm play lists, which are play lists of music stored on last.fm’s website, allowing a user to share their music taste with the world.

Suggested songs

Amarok also supports the Magnatune album label and online purchase of Magnatune’s albums. Magnatune albums aren’t inflicted with DRM, which allows complete freedom of how a user can listen to their personally purchased music. Follow this link for a little more information about Magnatune.

Amarok is extensible, and already has a growing library of scripts and plug-ins. For your convenience: Scripts available for Download

Amarok also has integrated support with Musicbrainz, allowing a track, for example, with no id3 or other tags, and named generically to be analysed, and identified. The section immediately below will illustrate some support of Musicbrainz

Retrieving the information:
Musicbrainz 1

Applying the information:
Musicbrainz 2

The following screen shots show a small example of how a search may be done.

The following image shows a search done with the simple string ‘america’. Notice how Amarok searches through all fields to search for ‘america’.

search 1

This image shows a search with the string ‘artist:america’. This tells Amarok to search only the artist column for ‘america’.

search 2

This image shows a search further filtering the results with the search string ‘artist:america title:horse’. Note that searches can be filtered by fields not showing at the time.

search 3

Note that when you filter the play list and Amarok changes songs, it will pick from the filtered list. This can be a boon or a burden depending on what you are doing, just keep this in mind.

Amarok also has quite a few User Interface goodies…

  • Use the mouse wheel over the volume bar in the Amarok window, or the task bar icon to adjust the volume up and down.
  • Use the mouse wheel over the time bar to seek forward / backward
  • The Amarok task bar icon shows the percent complete on the icon. The icon is bright at the beginning, and dark at the end of a track. In between it’s as if it is emptying of a liquid:

    Taskbar
  • Right click on the task bar icon to use previous track, play / pause, stop, and next track functions
  • Most functions are able to be mapped through DCOP, allowing for quick, easy keyboard shortcuts
  • Using DCOP, it is possible to use a remote control to manipulate Amarok
  • Notice the context menu on the image below:

    Context clues
    • The “Write ‘Iron Maiden’ for Selected Tracks” button will write the selected field to all tracks currently selected. Below, it will write ‘Iron Maiden’ for the Artist track below where it is blank. This also works on multiple records, so it is easy to make a change to selected songs, or all using (shift+right click) or (ctrl+right click)
    • Also notice the “Edit ‘Artist’ Tag” entry; This will allow you to edit the selected track’s Tag field, and when you are finished editing, it populates the new information in any selected tracks.
  • Amarok will also organise and rename your files based upon tags and a little user input:

    Organise 1Organise 2

Amarok is available in all recent releases of Debian and Ubuntu

Debian Weekly News 2007/05

published on Tue Apr 24 00:00:00 2007 in weekly-news

Welcome to this year's 5th issue of DWN, the newsletter for the Debian community. Roland Mas announced that Alioth users can use Mercurial for version control. Robert Millan announced version 0.4.0 of the Debian loader for Windows operating systems including Vista. Joey Schulze reported that security updates are available via IPv6 from official servers as well. The new release of Debian GNU/Linux 4.0 is celebrated all over the world.

Saving Money with Debian GNU/Linux

The leader of the IT department of Germany's Federal Foreign Office, Rolf Schuster, reported that they have seriously cut their IT costs by consequently using Free Software. Driven by the urge to save money on license fees and to escape from update cycles the office started the move in 2002 and has since then connected 230 embassies with the secure intranet gateways. More than 300 laptops of diplomats also run a specialised distribution based on Debian GNU/Linux.

Debian participates in Google's Summer of Code

Steve McIntyre announced that the Debian project has been accepted for this year's Google Summer of Code. During this Google sponsors the creation and future development of Free Software. A Wiki page has been set up to coordinate the participation and to collect ideas and proposals for possible projects. The list of accepted student applications was published on April 9th.

New GNU/kFreeBSD CD Image

Aurelien Jarno announced that a new installation CD image is available for Debian GNU/kFreeBSD. This port is based upon the GNU C library and the FreeBSD kernel. The CD image uses kernel version 6.2 which supports more recent hardware than previous versions and is available for the i386 and amd64 platforms.

Sam Hocevar elected as Debian Project Leader

Manoj Srivastava announced Sam Hocevar as the winner of this years' project leader election whose term starts on April 17th. A total of 482 developers casted their vote. The Debian project would like to wish Sam all the best for his upcoming tasks and decisions. We also want to thank Anthony Towns for his past term as Debian project leader.

Debian GNU/Linux 3.1 updated

Alexander Schmehl announced that sarge, the old stable distribution, has been updated for the sixth time to incorporate security updates and minor corrections. With the release of etch as the new stable distribution sarge has been moved to oldstable. Users who want to stick with sarge have to take special care before upgrading to 3.1r6.

Debian GNU/Linux 4.0 released

The Debian project announced the release of Debian GNU/Linux 4.0, codenamed etch. It includes a new textual and graphical installer that supports 58 languages and encrypted partitions, packages and repository verification, and a lot of new and updated software packages. It is strongly recommended to read the release notes before upgrading. After nearly two years of development, this release marks another milestone in the history of Debian.

Interview with Ian Murdock

Debian founder Ian Murdock was interviewed by LinuxFormat. He said that he is happy how Debian has developed, but that it is a pity that the project failed to release etch on time and how this happened. Ian opposes the democratic structures in Debian because no leader feels empowered to make decisions unless everyone agrees with him and mentioned Ubuntu as a better example. In respect to the latter he expressed his concern about compatibility with Debian.

Debian Conference Reconfirmation

Jörg Jaspert asked all potential attendants of this year's Debian conference to reconfirm their participation until May 3rd. Only confirmed attendants are considered for sponsored accommodation and food if asked for. Guests paying for everything on their own are welcome as well, of course.

From teTeX to TeX Live

Frank Küster announced that teTeX will be replaced with TeX Live. This is the successor of teTeX and uses most of the scripts developed for teTeX. The old teTeX packages will vanish and only continue to exist as transitional packages to give users a sensible choice of TeX stuff.

Debian 4.0 CD Usage

Joey Hess explained that etch consists of 331 CD and DVD images in total but 324 of them are only rarely needed. Of the remaining 7 images the most important one is the multi-architecture DVD that boots on 32 and 64 bit x86 systems as well as on powerpcs. It will detect the architecture and automatically boot the right one. Enough software is included to install a nice desktop even without network access.

sshfs: Easy (and secure) access to a remote file system

published on Sun Apr 22 05:00:56 2007 in packages-news

Entry submitted by Diego Essaya. DPOTD needs your help, please contribute !

I’m sure you are already familiar with the ssh command. (If that’s not the case, maybe this article is not for you). Most likely you have also discovered scp ages ago. But it is probable that you have never heard of sshfs before.

SSHFS is a file system client based on the SSH File Transfer Protocol. It allows to mount a remote file system in your box, and use it as if it was a local directory. Besides the fact that it is a secure protocol, the main advantage of SSHFS is that it is very easy to setup and use. It has only two easy to meet requisites:

  1. The local system needs to have the FUSE kernel module loaded.
  2. The remote machine needs to be running a SSH server that understands the SSHFS protocol.

Preparation

First of all we must install the SSHFS package in the local system:

# apt-get install sshfs

The package is available in both Debian and Ubuntu repositories.

Next, let’s make sure that condition #1 is met. In the local system, type (as root):

# modprobe fuse

This will load the FUSE kernel module. Besides SSHFS, the FUSE module allows to do lots of other nifty tricks with file systems, such as the BitTorrent file system, the Bluetooth file system, the User-level versioning file system, the CryptoFS, the Compressed read-only file system and many others.

As for condition #2, chances are it is already met: the OpenSSH server is
already installed and running in most Debian and Ubuntu systems. If this is not your case, just run the following command on the remote system:

# apt-get install ssh

Usage

Luckily, SSHFS is very simple to use. The following command:

$ sshfs user@host: mountpoint

will mount the home directory of the user@host account into the local directory named mountpoint. That’s as easy as it gets. (Of course, the mountpoint directory must already exist and have the appropriate permissions).

If you want to mount a directory other than the home directory, you can specify it after the colon. Actually, a generic sshfs command looks like this:

$ sshfs [user@]host:[dir] mountpoint [options]

Alternatives

The classic alternatives to access remote file systems are NFS and SMBFS. The main advantages of SSHFS are:

  • Easy to setup and run
  • Secure link

If you are sharing files between Windows machines, perhaps SMBFS is the best option. If you are not concerned about security and you need a faster alternative to SSHFS, go for NFS.

Links:

DebConf6 Final report, DebConf7 list of events and reconfirmation

contributed by Joerg Jaspert, published on Fri Apr 20 23:45:00 2007 in event-announce, news

The DebConf team has released the final report for the seventh annual Debian Conference, which took place in Oaxtepec, Mexico in May, 2006. The report is intended for a large audience, and includes impressions and facts from the conference.

Meanwhile, preparations for DebConf7, to be held in Edinburgh from Sunday 17 June until Saturday 23 June, continue. Recently the team has released a list of events that have been planned to occur during the conference. Whilst the actual scheduling has not yet occured, the draft schedule gives a good impression that the upcoming conference will again cover a wide set of topics, providing a good forum for not only developers, but interested users and wannabe maintainers.

If you are interested to meet the people that produce your favorite Linux Distribution you are encouraged to register in the conference management system, to let the organisers know you want to attend. If you have already registered for an account, make sure you reconfirm your attendance before May 3rd.

About DebConf

DebConf is the Debian Project's developer conference. In addition to a full schedule of technical, social and policy talks, DebConf provides an opportunity for developers, contributors and other interested people to meet in person and work together more closely. It has taken place annually since 2000 in locations as varied as Canada, Finland and Mexico.

DebConf is preceded by DebCamp, which is a smaller, less formal event that gives an opportunity for group work on Debian projects. Between both events, DebianDay takes place. DebianDay is a short conference aimed at Debian users, and others interested in learning more about free software.

most: an alternative and powerful pager program

published on Fri Apr 20 05:00:21 2007 in packages-news

Entry submitted by Emmanuel Bouthenot. DPOTD needs your help, please contribute !

Most is a powerful “pager”, similar to more and less. It is written in C using the slang library. It can display:

  • compressed (bzip, gzip) files on the fly
  • manpages with fancy output
  • one or multiples files in windowed mode
  • arbitrary binary files

Usage:

To use most as the default pager you can add this into your start-up user script (~/.bashrc, ~/.zshrc, etc.)

[ -x /usr/bin/most ] && export PAGER=most

You could also set it up to replace more and less:

[ -x /usr/bin/most ] && alias more='most' && alias less='most'

Screenshots

Manpage view :

Windowed view of compressed files :

Binary file view:

Most has been available in Debian and Ubuntu for a long time now, and is well maintained.

enigma: addictive puzzle game with a high dose of dexterity

published on Wed Apr 18 05:00:24 2007 in packages-news

This week, an extra DPOTD article will be published on Friday, remember to check it out!

Shameless self-promotion submitted by Erich Schubert. DPOTD needs your help, please contribute !

Enigma is an addictive puzzle game

A re-invention of the discontinued game “Oxyd” available for Atari, Mac and (some versions) DOS, with hundreds of levels and improved graphics.

The game principle of Enigma is simple: uncover pairs of stones as in the “Concentration” (also known as “Memory” or “Pairs”) board game.

Simple? Yes. Easy? Not by far!

You’ll first have to reach these stones. Your actor is a black marble controlled via the mouse - and influenced heavily by physics. Different floors show different friction properties, blocks might need a good bump to move in the intended direction, bouncers, slopes and rubber bands might be pushing or pulling your marble in a different direction than you’d like it to go. Sometimes you have to hit exactly the right angle to make the marble bounce of a block in space right towards your goal.

And then the blocks are hidden in labyrinths, protected by lasers and traps, and all kinds of puzzles you’ll have to solve first before being able to reach them. There are dozens of items you’ll discover and need to find out how to use right to reach the goal.

Enigma levels are very different in nature. Some levels are well-known Sokoban levels (except you’ll have to be careful to not move boxes you didn’t intend to) and similar well known puzzles ported to be controlled with a marble, some are vast labyrinths where you have to carefully balance your marble on a small ledge. Some levels require speed and mouse dexterity, others can only be solved by bright minds. Having to control 10 marbles connected with rubber bands and charged with different magnetic charges at the same time is just one of the challenges you’ll be facing in enigma. Such levels, that blend all these features into a unique mix can best be described as “Enigma”.

Here are some screenshots, you can find more in the homepage.

EnigmaEnigmaEnigma

Enigma is available in Debian since Sarge and in Ubuntu since Warty. Unfortunately, Enigma 1.00 was not released on time to be included with Debian Etch. Enigma 0.92 was in Etch when the freeze was called.

moreutils: A collection of useful command-line tools

published on Sun Apr 15 05:00:04 2007 in packages-news

Entry submitted by John Beisley. DPOTD needs your help, please contribute !

Moreutils contains a suite of utilities for command-line users. Written by multiple authors, they are individually quite modest, and perhaps too limited in scope to exist in a package in their own right, but together they form a useful accompaniment to existing command-line tools.

Following are a few examples of the utilities in moreutils, and how they can be used.

Sponge - Soaking file redirects

Many users of the command line have made a mistake such as the following:

$ grep -v someuser /etc/passwd > /etc/passwd

As the shell runs this command it will immediately truncate the destination file, and then run the grep command on it. The end result is an empty file - certainly not what was intended! After learning from their mistake, the user might instead redirect the output to a temporary file, then move the file on top of the original:

$ grep -v someuser /etc/passwd > /etc/passwd.tmp
$ mv /etc/passwd.tmp /etc/passwd

Sponge is a simple, but convenient, command that exists to roll this process up into a single step:

$ grep -v someuser /etc/passwd | sponge /etc/passwd

Sponge ’soaks up’ its standard input, waiting for it to complete before writing to the specified output file. In this way the result is what was intended, and without the need to fiddle with temporary files.

Vipe - Interactively editing pipes

Sometimes it is desirable to put yourself in the middle of a pipe processing chain, where it will be quicker to just edit the pipe content, rather than create complex filters with sed and friends. Maybe the pipe content will be unknown, perhaps a list of files that the user would like to manually filter.

Say that you would like to touch all files in the current directory, but would like to do so selectively, you could do the following:

ls | vipe | xargs -d ‘\n’ touch

In this example a list of files is opened up in your editor (as set by either the EDITOR or VISUAL environment variables), in which you may edit the list of files to be touched at your leisure. When you are satisfied, you can save the file and quit your editor, and vipe will regurgitate the content you edited to its standard-out - touching only the files retained in the editor.

Combine - Boolean/Set operations with text files

Performing set operations with lines in two text files can also be useful. This can be useful in knowing what file names are common to two directories, for example:

Directory foo contains the files:

tom and harry
Directory bar contains the files:

tom and dick

The file lists can be written to two files ready for use with combine with the following bash command:

ls /path/to/foo > foo_list && ls /path/to/bar > bar_list

It can be complicated to find which files are present in both directories, which is where combine really comes into its own:

$ combine foo_list and bar_list
tom

How about files that are in foo, but not in bar?

$ combine foo_list not bar_list
harry

A note of caution, however: while using combine, ‘or’ can be used to find the complete list of files, it has the slightly odd effect of listing files common to both files twice:

$ combine foo_list or bar_list
harry
tom
dick
tom

Obtaining the union of the two lists of file names is therefore best performed with the standard sort command:

$ cat foo_list bar_list | sort -u
dick
harry
tom

Summary

This introduction to moreutils has only touched upon some of the commands that it provides. It is worth experimenting with the others, very briefly described here:

isutf8
Checks if a file consists of valid UTF-8
ts
Reproduces standard input on standard output, with a timestamp prefix - good for annotating non-timestamped log files.
vidir
Enables a user to “edit” a directory with their text editor. That is, to conveniently delete and rename files.
ifdata
Outputs requested information about a specific Linux network interface, without the need to parse ifconfig output.
pee
Acts like tee, but pipes to given commands, instead of standard out and files.
zrun
Automatically recognises compressed files passed to a given command, uncompresses them to a temporary file and passes the uncompressed files in place of the original compressed file arguments.
mispipe
Connects two commands with a pipe, as in command1 | command2, but returns the exit value of the first command, rather than the second.

moreutils has been available in Debian in testing and unstable distributions, and in Ubuntu from Edgy onwards.

glabels: Label, business card and media cover creation program

published on Fri Apr 13 05:00:34 2007 in packages-news

Entry submitted by Dave Seff. DPOTD needs your help, please contribute !

gLabels is a program for creating labels and business cards. It is designed to work with various laser/ink-jet peel-off label and business card sheets that you’ll find at most office supply stores.

This application is a gem of a program for anyone who want to create their own business cards, address labels, CD Labels and covers, and much more. It has an extensive database of label and sticker formats for most major brands of labels such as Avery, Neato, and Memorex. It also has a template creation feature which allows for custom labels and designs. For this example I will explain the basics on how to create a set of business cards.

When you choose new from the file menu, you are presented with a list of various standard label sets. I chose Avery 5871 Business cards as that was what I had purchased from the store.

From here you can start designing. You can place and format text, images, and basic shapes to your liking. Don’t worry about needing to duplicate these to cover the entire sheet of labels. gLabels knows to print the entire sheet as per the template used. Once you are satisfied with your design, just send it to the printer once you have loaded it with the store bought labels.

Screenshot

As you can see the interface is quite simple to use. Editing labels is simple and straight forward. News, updates and support can be found at the gLabels homepage, http://glabels.sourceforge.net/.

gLabels is written for the Gnome desktop environment but just like most Linux applications, it will run under any window manager. KDE has a similar application called kbarcode. The name is misleading however it does provide the same functionality. It does not seem to be as straightforward as gLabels, but it does support batch printing if you are mass producing labels for commercial use.

gLabels has been available in Debian since Sarge, and in Ubuntu since Warty.

Reportbug-NG: An easy to use alternative to Debian’s classic reportbug

published on Wed Apr 11 05:00:13 2007 in packages-news

This week, an extra DPOTD article will be published on Friday, remember to check it out!

Entry submitted by Bastian Venthur. DPOTD needs your help, please contribute !

Reportbug-NG is an alternative to Debian’s classic reportbug written with the end-user in mind. It has a nice and clean QT interface allowing you to search for existing bug reports and filter them easy and fast. You can give additional information to existing bug reports or create new ones. In both cases a dialog will ask you for all relevant information (summary, severity, tags, …) and will finally start your favourite mail client, where you only have to fill the gap with your full report and click the “send” button.

A screencast showing Reportbug-NG in action (click on the image for full size):
Reportbug-NG screencast

The major advantages over reportbug are:

  • Graphical interface (true desktop application)
  • Severity and Status of bugs are associated with colors, so it’s easy to distinguish the different classes.
  • Easy and fast filtering of bug reports
  • Ability to write you bug reports with your mail client (spellchecker, copy-paste, attachments). The following Clients are currently supported:
    • Evolution
    • GoogleMail
    • Iceape/Mozilla
    • Icedove/Thunderbird
    • KMail
    • Mutt
    • Pine
    • Sylpheed
    • Sylpheed-Claws
    • Sylpheed-Claws-Gtk

Note to Ubuntu users: since reportbug-ng submits reports to the Debian BTS, it isn’t useful for reporting bugs in Ubuntu packages.

Reportbug-NG is actively developed and maintained by Bastian Venthur. It is fully functional and currently no major flaws are known. As it’s fairly new, it’s only available in Debian unstable.

Some features I planned for the next versions:

  • Porting R-NG from qt3 to qt4.
  • Include some advanced BTS-features for developers like tagging, reassigning, merging, etc.

Project homepage: http://reportbug-ng.alioth.debian.org/

Debian GNU/Linux 4.0 released

published on Sun Apr 8 12:42:18 2007 in news, release

The Debian Project is pleased to announce the official release of Debian GNU/Linux version 4.0, codenamed etch, after 21 months of constant development. Debian GNU/Linux is a free operating system which supports a total of eleven processor architectures and includes the KDE, GNOME and Xfce desktop environments. It also features cryptographic software and compatibility with the FHS v2.3 and software developed for version 3.1 of the LSB.

Using a now fully integrated installation process, Debian GNU/Linux 4.0 comes with out-of-the-box support for encrypted partitions. This release introduces a newly developed graphical frontend to the installation system supporting scripts using composed characters and complex languages; the installation system for Debian GNU/Linux has now been translated to 58 languages.

Also beginning with Debian GNU/Linux 4.0, the package management system has been improved regarding security and efficiency. Secure APT allows the verification of the integrity of packages downloaded from a mirror. Updated package indices won't be downloaded in their entirety, but instead patched with smaller files containing only differences from earlier versions.

Debian GNU/Linux runs on computers ranging from palmtops and handheld systems to supercomputers, and on nearly everything in between. A total of eleven architectures are supported including: Sun SPARC (sparc), HP Alpha (alpha), Motorola/IBM PowerPC (powerpc), Intel IA-32 (i386) and IA-64 (ia64), HP PA-RISC (hppa), MIPS (mips, mipsel), ARM (arm), IBM S/390 (s390) and – newly introduced with Debian GNU/Linux 4.0 – AMD64 and Intel EM64T (amd64).

Debian GNU/Linux can be installed from various installation media such as DVDs, CDs, USB sticks and floppies, or from the network. GNOME is the default desktop environment and is contained on the first CD. The K Desktop Environment (KDE) and the Xfce desktop can be installed through two new alternative CD images. Also newly available with Debian GNU/Linux 4.0 are multi-arch CDs and DVDs supporting installation of multiple architectures from a single disc.

Debian GNU/Linux can be downloaded right now via bittorent (the recommended way), jigdo or HTTP; see Debian GNU/Linux on CDs for further information. It will soon be available on DVD and CD-ROM from numerous vendors, too.

This release includes a number of updated software packages, such as the K Desktop Environment 3.5 (KDE), an updated version of the GNOME desktop environment 2.14, the Xfce 4.4 desktop environment, the GNUstep desktop 5.2, X.Org 7.1, OpenOffice.org 2.0.4a, GIMP 2.2.13, Iceweasel (an unbranded version of Mozilla Firefox 2.0.3), Icedove (an unbranded version of Mozilla Thunderbird 1.5), Iceape (an unbranded version of Mozilla Seamonkey 1.0.8), PostgreSQL 8.1.8, MySQL 5.0.32, GNU Compiler Collection 4.1.1, Linux kernel version 2.6.18, Apache 2.2.3, Samba 3.0.24, Python 2.4.4 and 2.5, Perl 5.8.8, PHP 4.4.4 and 5.2.0, Asterisk 1.2.13, and more than 18,000 other ready to use software packages.

Upgrades to Debian GNU/Linux 4.0 from the previous release, Debian GNU/Linux 3.1 codenamed sarge, are automatically handled by the aptitude package management tool for most configurations, and to a certain degree also by the apt-get package management tool. As always, Debian GNU/Linux systems can be upgraded quite painlessly, in place, without any forced downtime, but it is strongly recommended to read the release notes for possible issues. For detailed instructions about installing and upgrading Debian GNU/Linux, please see the release notes. Please note that the release notes will be further improved and translated to additional languages in the coming weeks.

TinyCA: simple user interface to manage a small CA

published on Sun Apr 8 05:00:19 2007 in packages-news

Entry submitted by Julien Valroff. DPOTD needs your help, please contribute !

TinyCA aims at helping you in the certification authority (CA) management task. Despite its easy-to-use and somewhat intuitive interface, it provides extended functions for advanced users who want to simplify their life.

Screenshot: CA & certificate list tabs

TinyCA - main window

TinyCA - certificate list

It works like a front-end for OpenSSL and offers (almost) the same features, amongst which:

  • Unlimited CAs: you can work with several CAs, choice is made when opening the main window or from an easily accessible button
  • Creation and revocation of x509 - S/MIME certificates
  • Import already existing CAs
  • Export client and server certificates in PEM, DER, TXT and PKCS#12 formats

Screenshot: create & import CA dialogs

TinyCA - creating a new CA

TinyCA - importing an already existing CA

Users already familiar with OpenSSL will be able to work immediately with TinyCA, while the more inexperienced will first have to look for documentation on certificate management elsewhere, as suggested by TinyCA documentation page:

TinyCA - help

TinyCA reveals particularly useful when you have to manage a bunch of certificates, eg. in the case of an OpenVPN tunnel. In this case, I do only regret TinyCA is not able to generate Diffie-Hellman keys, for which you still need to use OpenSSL from the command line.

TinyCA is available in Debian Sarge in the GTK version, whereas the latest version in Debian testing and unstable is based on the Perl-Gtk2 bindings (this lead upstream developer to call the new generation TinyCA2, whereas the Debian package kept the name tinyca). The package is also available for Ubuntu users in the universe component.

Once the package is installed, do not look for a TinyCA entry in your GNOME or KDE menu, none is provided. Instead, just look in the Debian menu, or simply launch the command in a terminal emulator: tinyca2.

If you end up using TinyCA, remember to keep your ~/.TinyCA directory private, as it contains the private keys for your CA and your certificates! Also do use a secure method for transferring your keys to the machine, eg. ssh.

Debian GNU/Linux 3.1 updated

contributed by aba, published on Sat Apr 7 22:17:00 2007 in news, release

The Debian project has updated the stable distribution Debian GNU/Linux 3.1 (codename Sarge). This update mainly adds security updates to the stable release, along with a few corrections to serious problems. Those who frequently update from security.debian.org won't have to update many packages and most updates from security.debian.org are included in this update.

In preparation for the upcoming release of Debian GNU/Linux 4.0 (codename Etch), Debian GNU/Linux 3.1 will be moved to the 'oldstable' part of the archive. Users who would like to continue using Debian GNU/Linux 3.1 are advised to update their /etc/apt/sources.list network source to refer to 'sarge' instead of 'stable'.

Please note that this update does not constitute a new version of Debian GNU/Linux 3.1 but only updates some of the packages included. There is no need to throw away 3.1 CDs. Instead you only need to update against ftp.debian.org or a mirror after an installation, in order to incorporate those changes. New CD and DVD images will be delayed until after the release of Etch and will be available at the regular locations.

Upgrading to this revision online is usually done by pointing the 'apt' package tool (see the sources.list(5) manual page) to one of Debian's many FTP or HTTP mirrors. A comprehensive list of mirrors is available at:

http://www.debian.org/distrib/ftplist

Miscellaneous Bugfixes

This stable update adds a few important corrections to the following packages.

Package                Reason
base-installer         Fix for kernel ABI bump (fix regression from 3.1r5)
glibc                  Get architectures back in sync

Security Updates

This revision adds the following security updates to the stable release. The Security Team has already released an advisory for each of these updates.

Advisory ID   Package(s)               Correction(s)
DSA 1240      links2                   Arbitrary shell command execution
DSA 1262      gnomemeeting             Arbitrary code execution
DSA 1263      clamav                   Denial of service
DSA 1264      php4                     Several vulnerabilities
DSA 1265      mozilla                  Several vulnerabilities
DSA 1266      gnupg                    Signature forgery
DSA 1267      webcalendar              Remote file inclusion
DSA 1268      libwpd                   Arbitrary code execution
DSA 1269      lookup-el                Insecure temporary file
DSA 1270      openoffice.org           Several vulnerabilities
DSA 1271      openafs                  Remote privilege escalation
DSA 1272      tcpdump                  Denial of service
DSA 1273      nas                      Multiple remote vulnerabilities
DSA 1274      file                     Arbitrary code execution
DSA 1275      zope2.7                  Cross-site scripting flaw
DSA 1276      krb5                     Several vulnerabilities
DSA 1277      xmms                     Arbitrary code execution
DSA 1278      man-db                   Arbitrary code execution

The complete list of all accepted and rejected packages together with rationale is on the preparation page for this revision:

http://release.debian.org/stable/3.1/3.1r6/

URLs

The complete lists of packages that have changed with this revision:

http://ftp.debian.org/debian/dists/sarge/ChangeLog

The current stable distribution:

http://ftp.debian.org/debian/dists/stable

Proposed updates to the stable distribution:

http://ftp.debian.org/debian/dists/proposed-updates

Stable distribution information (release notes, errata etc.):

http://www.debian.org/releases/stable/

Security announcements and information:

http://www.debian.org/security/

Gobby: A Collaborative Text Editor

published on Wed Apr 4 05:00:16 2007 in packages-news

Entry submitted by Adam Victor Nazareth Brandizzi, Pedro Vaz de Mello de Medeiros and Pedro Araújo Chaves Júnior. DPOTD needs your help, please contribute !

Gobby is a Free Software collaborative text editor. That means that various users can edit the same text file simultaneously.

Using Gobby is easy: just create a session, then other writers may connect to your host and subscribe to any publicly available documents in the Document List dialog, and/or create a new shared document. Files are opened in tabs in the main window, so that the user can edit them simultaneously. Any user can contribute with his own set of files to the pool of shared files, just like the session owner, and anyone can save a local copy of them.

screenshot

The user interface is very clear and simple: the main window is divided in two parts: the text being edited on the top and a chat window below with an IRC look and feel (but only one IRC command is supported, though: the good old /me ;)). In the main window, each fragment of text is colour-coded to indicate which user wrote it. The list of currently connected users, along with their corresponding colours, can be checked in the Online node in the User List dialog.

Gobby has support for some common text-editing features such as auto indenting, tab-to-space replacement, smart Home key, font selecting, syntax highlighting by file type, etc. Gobby will try to guess the right file type, but if it doesn’t, syntax highlighting can be changed on the fly from a long drop-down menu list. It is also possible to define a password-protected session; this is useful if you want to control which users are allowed to connect.

The chat functionality provides a separate channel that allows authors to communicate and coordinate their efforts —even if they are not physically close to each other—, which can be particularly useful if they want to discuss, for instance, guidelines or other things that don’t exactly translate to text editing. Or you can just fire up Gobby to have a simple and lightweight chat session with some friends ;).

screenshot

The Gobby developers also provide the specifications of the Obby protocol used for implementing the collaborative editor, sobby, the Obby dedicated server, and libobby, the library for developers interested in creating their own clients and servers.

Known issues:

There are some reports about crashes on the Gobby site. We have not observed any of them ourselves, though. On functionality, Gobby does not provide any Undo/Redo capabilities —and we have really missed them. Also, there is no way to keep track of deleted text. Another missing feature is a graphic representation of the cursors of all users and means to distinguish them from each other, but this functionality is already expected for Gobby v0.5.0.

License:

Gobby is licensed under the GNU General Public License (v2).

Availability:

Gobby is available in Debian Etch and Sid, as well as in the Ubuntu Universe since Breezy Badger. You can get more information at the Gobby home page.

fortunes: Fortune cookies for all

published on Sun Apr 1 05:00:46 2007 in packages-news

Entry submitted by Gaurav Vaidya. DPOTD needs your help, please contribute !

Fortune cookie programs store long lists of quotes, sayings, aphorisms, adages, IRC transcripts, and any other text which might be interesting. Since time immemorial (okay, okay, since 1979) Unix users have added fortune cookies to their .bashrc (or equivalent) files, displaying a random quote every time a terminal is created. Quotes are drawn from subjects and sources of interest to geeks, from William Shakespeare to Douglas Adams, from Ambrose Bierce to Ziggy, from Linus Torvalds to the Fortune editors themselves.

On Debian/Ubuntu, the fortune cookie program of choice is available through the package fortunes.

Usage

The program is ridiculously simple to run: executing /usr/games/fortune (or /usr/bin/fortune) displays a pithy quote:

       "I assure you the thought never even crossed my mind, lord."
       "Indeed?  Then if I were you I'd sue my face for slander."
               -- Terry Pratchett, "The Colour of Magic"

Or perhaps something insightful:

It's hard to tune heavily tuned code.  :-)
            -- Larry Wall in <199801141725.JAA07555@wall.org>

Or even self-referential:

This fortune would be seven words long if it were six words shorter.

Although the program works fine without any arguments, the following
may be helpful:

-o
Display only offensive fortunes (you will need to install the offensive fortunes first; they are available in the fortunes-off package).
-a
Display both offensive and non-offensive fortunes.
-w
Wait for a while after displaying the comment, but before exiting. The amount of time waited depends on the number of characters in the fortune displayed; longer quotes will be ‘waited for’ longer. Very nice for scripting; for instance, running while true; do fortune -w; done; in a Bourne shell will display fortune after fortune, with a convenient pause after each one. My preferred script, while true; do echo === `date` ===; fortune -w; echo; done; will do the same, except with a time-stamp at the top of each fortune.

Installing

To install, you will need to install the fortunes package. This package contains 15,000 quotes itself, and depends upon fortune-mod, which contains the executables, and fortunes-min, which contain other fortunes. These packages have been available in Debian and Ubuntu since long ago.

Other packages (available for both Debian and Ubuntu) allow you to install quotes in Chinese, Bulgarian, Portuguese, Czech, Slovak, German, Esperanto, Spanish, French, Gaelige (Irish), Italian, and Polish, as well as Debian hints and BOFH excuses.

The quotes used in this submission are from fortune-mod version 9708.