rdiff-backup: Easy incremental backups from the command line
on 26.10.2008, 05:00
in packages-news
Debian at Systems Expo in Munich, Germany
on 20.10.2008, 20:22
in event-announce
Debian Project News 2008/13
on 20.10.2008, 00:00
in weekly-news
memstat: Identify what is using up virtual memory
on 19.10.2008, 05:00
in packages-news
smartmontools: control the health of your hard disk
on 12.10.2008, 05:00
in packages-news
Debian Project News 2008/12
on 08.10.2008, 00:00
in weekly-news
logstalgia: pong-like apache log viewer
on 05.10.2008, 05:00
in packages-news

rdiff-backup: Easy incremental backups from the command line

published on Sun Oct 26 05:00:06 2008 in packages-news

Storage is becoming cheaper and cheaper: you can find hard drives that cost less than a dollar per GiB. Buying an external hard drive to make backups (or even having a backup server) is a must if you value your work and what you have stored in your computer. However, doing backups should be easy enough to be done on a regular basis. The more automated, the better.

So, I find no excuse not to do regular backups and looked for a tool easy-to-use but powerful. rdiff-backup is a python script that helps doing local and remote incremental backups. To backup your $HOME to an external hard drive mounted in /media/backup simply do:

$ rdiff-backup $HOME /media/backup/home_backup

If after some days you want to backup your new files, run the same command to update the backup.

Now, in /media/backup/home_backup you have an exact copy of your home as it was when you did the last backup. If you want to restore a directory, you can just copy it:

$ cp -a /media/backup/home_backup/src/myprogram ~/src/

Which is equivalent to:

$ rdiff-backup --restore-as-of now /media/backup/home_backup/src/myprogram ~/src/

Of course, you can restore previous versions of the file. For example, to restore the source of myprogram as it was a mounth ago:

$ rdiff-backup --restore-as-of 1M /media/backup/home_backup/src/myprogram ~/src/

You can see all the incremental backups you have done executing:

$ rdiff-backup --list-increments /media/backup/home_backup

If you run out of space in your backup device and you’re sure you don’t need the backups you made three years ago, you can remove them with:

$ rdiff-backup --remove-older-than 3Y /media/backup/home_backup

rdiff-backup works exactly the same with remote directories. You need to have ssh access and rdiff-backup must be installed in the remote(s) machine(s). Note that in any example above, you can change the local directories to remote ones, so you can backup a remote machine locally, or do a backup of this machine to a remote backup-server. For example, say backup.mysite.org is your backup server. You can backup regularly using:

$ rdiff-backup local-dir/ user@backup.mysite.org::/remote-dir

If you use RSA or DSA authentication, you can even put that in a cron job.

See rdiff-backup documentation and other examples to discover all the functionality of this package.

Similar packages

Frontends for rdiff-backup:

  • keep is a GUI (KDE) frontend for rdiff-backup.
  • archfs is a fuse (filesystem in userspace) virtual filesystem that lets you browse each version of a rdiff-backup repository as if they were any other directory. Adam Sloboda has stated his intention to package archfs for Debian.
  • rdiff-backup-web (not in Debian, no WNPP yet) is a web frontend for rdiff-backup.

There are a ton of other programs to make backups. I will list here some of them (but this list is no where near complete) that are similar to rdiff-backup:

  • backup2l also makes local backups, but seems to miss the remote-backup feature.
  • backuppc is a perl script that also makes incremental backups, and has an http user interface to help manage and restore backups.
  • duplicity also makes remote incremental backups, but encrypts the data using gnupg. I haven’t test it myself, but it can be useful if you don’t trust the remote file server.
  • storebackup also makes local incremental backups. It makes a new tree in every snapshot, but disk space is preserved by hard-linking unchanged files.

Conclusions

Pros:

  • Easy to use. Now there’s no excuse not to do backups!
  • Works from the command line, so you can easily put it in a script or cron job.
  • Simple recovery from last snapshot, you can use standard tools like cp or find.

Cons:

  • Not having a GUI may scare some users.
  • It stores the last snapshot uncompressed, so depending on what you are backing up, it can be very space consuming. Older snapshots are compressed, which makes this con a not-so-con ;-).

rdiff-backup has been available in Debian since Sarge (perhaps even longer), and in Ubuntu since Dapper.

Debian at Systems Expo in Munich, Germany

published on Mon Oct 20 20:22:19 2008 in event-announce

The Debian project will be present again at this year's Systems expo which takes place from October 21-24 in Munich, Germany. The Debian booth will be located in the "Prospects for Open Source" area together with other Debian-related projects like Skolelinux/Debian-Edu as well as other Free Software projects in Area B2, Booth 315.

At the booth, a preview of the next version of Debian, lenny, will be presented. Further, t-shirts, keychains, stickers and other Debian merchandise can be obtained at the booth. All interested Debian users are invited to pass by the booth for questions or a chat.

Besides the official Debian booth, there will be some other interesting booths for visitors concerning Debian:

  • The City of Munich's LiMux project, which migrates all city council's desktops to GNU/Linux will be present Area B2, Booth 420.
  • GeNUA GmbH (Area B3, Booth 321) provides system administration services for Debian GNU/Linux
  • Gonicus GmbH (Area B2, Booth 317) provides consulting, services and support for the deployment of and migration to Debian-based infrastructure and presents their GOsa/FAI based management/deployment system
  • Univention GmbH (Area B2, Booth 426A) presents their Debian-based Univention Corporate Server (UCS) which integrates cross-site and cross-platform identity and infrastructure management

Debian Project News 2008/13

published on Mon Oct 20 00:00:00 2008 in weekly-news

Welcome to this year's 13th issue of DPN, the newsletter for the Debian community.
Some of the topics covered in this issue include:

Bits from the Debian CD team

Steve McIntyre posted some bits from the Debian CD team. With the upcoming release of Debian GNU/Linux 5.0 Lenny, Blu-ray Disc images will be available as well as CD and DVD images. For the first time in several years, users should be able to fit all of the packages for one architecture on a single disc. But due to the increased size of the Debian archive and considerations of storage and network bandwidth (especially on our mirrors) only a limited number of versions of each image will be available for direct download on our mirrors.

Debian, software patents and archive layout

A recently announced Intent To Package moonlight, an open source implementation of a patented proprietary product, caused some discussion regarding software patents. Packaging such software and distributing it via our mirror network could be a legal risk for Debian and its mirrors.

Statistics about packages.debian.org usage

Since packages.debian.org has been moved to a new host sponsored by 1&1, Jörg Jaspert took the opportunity to publish some data about the usage of this service. Since the start of this year, packages.debian.org has already seen 5,777,672 unique visitors downloading the page 122,314,763 times (both numbers excluding bots, such as crawlers for search engines). Nearly half of the visitors went directly to packages.debian.org, while less than a quarter came by way of a search engine.

Unofficial bug tracker and delayed queue

The unofficial tracker for Release Critical bugs now incorporates bugs fixed by uploads to the so-called delayed queue. This makes it easier to track bugs needing to be worked on. The delayed queue is used for package uploads not carried out by the regular maintainer of the package (so-called NMUs).

Bits from the Eee PC team

Ben Armstrong delivered a report on the status of the port of Debian for the Eee PC netbooks. A recent upload of the Linux kernel fixed some thermal problems and solved issues with ACPI. Also, the ath5k driver included with the Linux kernel 2.6.27 supports the wireless adaptors of the 701, 900, 900A and 1000HD models, which previously required a non-free driver.

Policy for backporting KDE features

Sune Vuorela wondered how to deal with features introduced in recent versions of the desktop environment KDE. Other distributions often seem to backport new features to older versions they ship in their releases. He lists some pros and cons, and wonders whether Debian should do likewise.

Listing of debian.net domains

Nico Golde reported that he has removed his list of debian.net subdomains (which can be registered by any Debian Developer) due to privacy issues. He asks developers to add their debian.net subdomains to a wiki page.

Report from Skolelinux/Debian-edu gathering

Sune Vuorela reported on the recent gathering of Skolelinux/Debian-Edu developers. He spent some time working on educational videos and KDE related education packages as well as GNASH, a free software flash player.

Slogan for Lenny

The Debian Art team is seeking a slogan for the next stable release, Debian GNU/Linux 5.0 Lenny. The discussion, as well as a poll, can be found in the archives of the debian publicity mailing list.

Systems Expo 2008

From Tuesday 21 October to Friday 24 October, the Debian Project will participate at the Systems expo 2008 in Munich, Germany. The Debian booth will be located in Area B2, Booth 315 along with other FLOSS projects. Please see the events page for further details.

Technical Dutch Open Source Event 2008

From Saturday 25 October to Sunday 26 October, the Debian Project will participate with a booth at the Technical Dutch Open Source Event (T-DOSE) in Eindhoven, Netherlands. Please see the events page for further details.

New Developers

Since the previous issue of the Debian Project News, 7 applicants have been accepted as Debian Developers, and 3 as Debian Maintainers. Please welcome Adam Barratt, Julian Andres Klode, Jonny Lamb, Joachim Reichel, Bradley Smith, Behan Webster and Y Giridhar Appaji Nag as well as Emmanuel Bouthenot, Samuel Thibaul and David Paleino to our project!

Important Debian Security Advisories

Debian's Security Team recently released advisories for these packages (among others): iceweasel, openldap, ruby1.8, ruby1.9, linux-2.6.18 and libxml2. Please read them carefully and take the proper measures.

New and noteworthy packages

The following packages were added to the unstable Debian archive recently (among others):

Work-needing packages

Currently 441 packages are orphaned and 124 packages are up for adoption. Please take a look at the recent reports to see if there are packages you are interested in or view the complete list of packages which need your help.

Want to continue reading DPN?

Please help us create this newsletter. We still need more volunteer writers to watch the Debian community and report about what is going on. Please see the contributing page to find out how to help. We're looking forward to receiving your mail at debian-publicity@lists.debian.org.

memstat: Identify what is using up virtual memory

published on Sun Oct 19 05:00:07 2008 in packages-news

Article submitted by Todd Troxell. Guess what? We still need you to submit good articles about software you like!

This tool lets you discover what libraries and programs are using up memory. It is very simple to use. Here is an example of its output (the -w makes memstat not truncate lines at 80 columns):

gaius% memstat -w
    256k: PID  5465 (/lib/ld-2.3.6.so)
    368k: PID 13019 (/var/db/nscd/passwd)
   3352k: PID 13914 (/usr/lib/gconv/gconv-modules.cache)
      8k: /usr/bin/memstat 5465
     12k: /lib/libcap.so.1.10 13019
    256k: /lib/libncurses.so.5.5 13914
     88k: /lib/ld-2.3.6.so 5465 13019
    256k: /lib/libncurses.so.5.5 13019
   1212k: /lib/tls/libc-2.3.6.so 13914
     32k: /lib/tls/libnss_compat-2.3.6.so 13914
     24k: /lib/tls/libcrypt-2.3.6.so 13914
     12k: /lib/tls/libdl-2.3.6.so 13914
    144k: /lib/tls/libm-2.3.6.so 13914
     76k: /lib/tls/libnsl-2.3.6.so 13914
     40k: /lib/tls/libnss_files-2.3.6.so 13914
     36k: /lib/tls/libnss_nis-2.3.6.so 13914
     60k: /lib/tls/libpthread-2.3.6.so 13914
     28k: /lib/tls/librt-2.3.6.so 13914
     88k: /lib/ld-2.3.6.so 13914
   1212k: /lib/tls/libc-2.3.6.so 5465 13019
     12k: /lib/tls/libdl-2.3.6.so 13019
    144k: /lib/tls/libm-2.3.6.so 13019
     76k: /lib/tls/libnsl-2.3.6.so 13019
    480k: /bin/zsh-beta 13019
    212k: /var/db/nscd/passwd 13019
    788k: /usr/bin/irssi 13914
    148k: /usr/lib/libpcre.so.3.12.0 13019
    176k: /usr/lib/perl5/auto/Irssi/Irssi.so 13914
     80k: /usr/lib/perl5/auto/Irssi/Irc/Irc.so 13914
     80k: /usr/lib/perl5/auto/Irssi/UI/UI.so 13914
     12k: /usr/lib/gconv/CP1252.so 13914
     24k: /usr/lib/gconv/gconv-modules.cache 13914
     76k: /usr/lib/libz.so.1.2.3 13914
    584k: /usr/lib/libglib-2.0.so.0.1200.4 13914
   1128k: /usr/lib/libperl.so.5.8.8 13914
     12k: /usr/lib/libgmodule-2.0.so.0.1200.4 13914
   1240k: /usr/lib/i686/cmov/libcrypto.so.0.9.8 13914
    248k: /usr/lib/i686/cmov/libssl.so.0.9.8 13914
      8k: /usr/lib/zsh-beta/4.3.2-dev-1/zsh/terminfo.so 13019
     24k: /usr/lib/zsh-beta/4.3.2-dev-1/zsh/zutil.so 13019
     56k: /usr/lib/zsh-beta/4.3.2-dev-1/zsh/compctl.so 13019
    116k: /usr/lib/zsh-beta/4.3.2-dev-1/zsh/complete.so 13019
    196k: /usr/lib/zsh-beta/4.3.2-dev-1/zsh/zle.so 13019
--------
  13480k

This output lists many libraries and processes loaded into memory and their sizes. First of all, processes and the size of their private memory are listed. This does not include their shared memory. Afterwards, shared objects are listed, and finally the total is listed.

In case you are wondering, shared object are libraries like /lib/tls/libc-2.3.6.so that are shared across all processes that need them to save memory and make things run faster. Instead of loading this library into memory for every process, Linux loads one copy and uses this for any process that wants to use the library. Therefore, you may notice that these values sometimes do not add up to the amount of memory you see used on your system. If you look at top(1) you will see two columns related to memory: RSS and VSZ. For each process, RSS is the amount of memory used by the process, and VSZ is the amount used counting shared objects. To add up memory correctly, you must count each shared object only once for the system.

It is possible to output per user statistics by running memstat as an unprivileged user. When run as root, memstat will list everything on the system.

Memstat works by scanning files in /proc and then searching for binaries in the paths listed at /etc/memstat.conf. The default file should be sufficient for most cases. If you have libraries or binaries in a non-standard place, you may need to modify this file to get accurate results.

Memstat was authored by Joshua Yelon. It is available in all current Debian and Ubuntu releases.

smartmontools: control the health of your hard disk

published on Sun Oct 12 05:00:05 2008 in packages-news

Article submitted by Noel David Torres TaƱo. Guess what? We still need you to submit good articles about software you like!

One of the packages I manually install in every new installation is smartmontools. I’ve some expertise in managing computers and networks, and it is a fact that pirate hackers and software bugs are not the main cause of problems in small and medium installations. Hardware is.

Thus, you have hardware that can fails, and Murphy says that if it can fail, it will. The point is not to avoid hardware failures, which would be impossible, but to detect them early or even prevent them.

Particularly for hard disks, the tool in charge is smartctl from the package smartmontools. IDE disks (if they’re not of the age of dinosaurs) have an integrated self-testing tool called SMART which means “Self-Monitoring, Analysis and Reporting Technology”. Modern SCSI disks have it too if they’re SCSI 3 or newer. It happens that inside the disk chipset there are routines to check parameters of disk health: spin-up time, number of read failures, temperature, life elapsed… And all of those parameters are not only registered by the disk chipset, but they have designated security limits and both parameters and limits can be checked by software who access the disk using the appropriate I/O instructions.

And that software is smartctl, a piece of the smartmontools deb package. Of course, since they access the disk in a raw way, you need to be root to use these commands.

smartctl can ask the disk for its smart identification:

# smartctl -i /dev/sda
smartctl version 5.38 [i686-pc-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Model Family:     Fujitsu MHV series
Device Model:     FUJITSU MHV2060BH
Serial Number:    NW10T652991F
Firmware Version: 00850028
User Capacity:    60,011,642,880 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   7
ATA Standard is:  ATA/ATAPI-7 T13 1532D revision 4a
Local Time is:    Mon May 12 02:39:31 2008 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

More interesting, smartctl can ask the disk for its parameter values:

# smartctl -A /dev/sda
smartctl version 5.38 [i686-pc-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   100   100   046    Pre-fail  Always       -       124253
  2 Throughput_Performance  0x0004   100   100   000    Old_age   Offline      -       18284544
  3 Spin_Up_Time            0x0003   100   100   025    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   099   099   000    Old_age   Always       -       1199
  5 Reallocated_Sector_Ct   0x0033   100   100   024    Pre-fail  Always       -       8589934592000
  7 Seek_Error_Rate         0x000e   100   087   000    Old_age   Always       -       1761
  8 Seek_Time_Performance   0x0004   100   100   000    Old_age   Offline      -       0
  9 Power_On_Seconds        0x0032   079   079   000    Old_age   Always       -       10866h+57m+47s
 10 Spin_Retry_Count        0x0012   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       1199
192 Power-Off_Retract_Count 0x0032   099   099   000    Old_age   Always       -       283
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       6953
194 Temperature_Celsius     0x0022   100   100   000    Old_age   Always       -       45 (Lifetime Min/Max 14/58)
195 Hardware_ECC_Recovered  0x001a   100   100   000    Old_age   Always       -       62
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       459276288
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x000e   100   082   000    Old_age   Always       -       22371
203 Run_Out_Cancel          0x0002   100   100   000    Old_age   Always       -       1533257648465
240 Head_Flying_Hours       0x003e   200   200   000    Old_age   Always       -       0

As you can see, there are some attributes marked as “Pre-fail”. If any of these attributes goes beyond its threshold, the disk is about to fail in hours, maybe minutes.

Even if there are more options to smartctl , the last ones I will comment here are -a and -t.

smartctl -t launches a disk test. It needs a parameter indicating the type of the test, and in the longest case it can last for tens of minutes and will check the electrical and mechanical performance as well as the read performance of the disk, going through all its surface. smartctl -a, in its turn, shows all available information about the disk, including self testing results. Since tests will span minutes or tens of minutes, we can not see them happening. All what we will get when launching tests is like:

# smartctl -t long /dev/sda
smartctl version 5.38 [i686-pc-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: "Execute SMART Extended self-test routine immediately in
off-line mode".
Drive command "Execute SMART Extended self-test routine immediately in
off-line mode" successful.
Testing has begun.
Please wait 41 minutes for test to complete.
Test will complete after Mon May 12 05:44:03 2008

Use smartctl -X to abort test.

Here, we’re being informed that (maybe) we will get a slightly lower performance on the disk for the next 41 minutes, since the test has started. It is completely background, or better ‘underground’, since it does not happen under the kernel control at all: everything is happening internally to the disk, and all what we can get is the result.

smartctl -a, in turn, show a very large amount of SMART information about the disk: almost all stored SMART information parsed for us. It is usually better to use a more specific switch, see the man page for details.

Finally, I want to comment that there is a daemon in the smartmontools package, smartd, who can take care of doing tests for you. It works by running smartctl in a periodic way (typically every 30 minutes) and logging all errors and parameter value changes to the syslog. The default configuration in Debian will also mail root if there’s any problem detected. I will not explain here about it, because I want you to read its (short and easy) documentation, but remember that in order to use it you must enable it in /etc/default/smartmontools.

The smartmontools package has been available both in Debian and Ubuntu since a long time ago.

Debian Project News 2008/12

published on Wed Oct 8 00:00:00 2008 in weekly-news

Welcome to this year's 12th issue of DPN, the newsletter for the Debian community.
Some of the topics covered in this issue include:

Bits from the DPL

Steve McIntyre sent out another Bits from the DPL mail. His first topic was the recently finished eighth Debian Conference in Argentina. Even though many developers and contributors could not travel there he considered it to be a successful conference. He especially thanked the video team, who did an amazing job this year making most of the sessions available via stream as well as forwarding questions via Internet Relay Chat. Steve is already looking forward to next year's Debian Conference, which will take place in the region of Extremadura in Spain.

What you can do for Lenny

Unfortunatly, Debian GNU/Linux 5.0 Lenny hasn't been released yet. Alexander Reichle-Schmehl briefly explained the problems and listed some open issues which need to resolved before Lenny can be released. He points out that even a simple user (meaning everyone) can help.

500,000th bug reported

Christian Perrier noted that the 500,000th bug has been reported to Debian's bug tracking sytem. In it, Nobuhiro Iwamatsu (岩松 信洋) requested a feature for the common debian build system, a tool used to create Debian packages, and even provided a patch.

Valid-Until field in Release files

While the current archive structure prevents injection of malicious packages through a digital trust path (e.g. at a bad mirror), it still has a small flaw. A potential attacker could use outdated release information to force people to use an outdated mirror, leaving out the latest security updates. To address this problem, Jörg Jaspert has added a valid until field to the release information. APT (or another package manager) can then check if the data available on the mirror is up to date. Work has already begun to integrate this feature into the apt package manager and tools based upon it; however, some questions remain unresolved.

Choosing a language during NAS installations

Martin Michlmayr reported that due to changes of the internal structure of the debian-installer, it is now it is now possible to choose the language (and the resulting system) for installations on NAS machines. Installations on headless NAS devices are typically done remotely via SSH and up until now, the network had been started after the language had already been chosen, thus the ability to choose a language interactively was completely disabled for such devices. Due to changes in the component responsible for choosing the locale, this feature can now be enabled for these kinds of devices.

m68k moved to debian-ports

After missing release criteria for both Etch and Lenny, the m68k port made the switch from using the wanna-build instance on Debian infrastructure to the one on Debian-Ports. This is a necessary step before m68k can be removed from the Debian archive. Buildd.Net still supports the m68k architecture and has already adopted the change. The m68k port was one of two official ports in the first Debian release, Debian 2.0 (the other being i386).

Other news

Christian Perrier released the final number of languages which will be supported in the debian-installer of the upcoming release. All-in-all 63 languages will be supported, which is 5 more than in the current release.

Linux Kongress 2008

From Thursday the 9th of October to Friday the 10th of October, the Debian Project will participate with a booth at the Linux-Kongress 2008 in Hamburg, Germany. Please see the respective events page for further details.

Technical Dutch Open Source Event 2008

From Saturday the 25th of October to Sunday the 26th of October, the Debian Project will participate with a booth at the Technical Dutch Open Source Event (T-DOSE) in Eindhoven, Netherlands. Please see the respective events page for further details.

New Developers

6 applicants have been accepted as Debian Developers since the prior issue of the Debian Project News. Please welcome Tobias Grimm, Chris Lamb, Manuel Prinz, Patrick Schoenfeld, Sandro Tosi, Jan Wagner and Barry deFreese in our project!

Important Debian Security Advisories

Debian's Security Team recently released advisories for these packages (among others): openssh, twiki, phpmyadmin, horde3, mplayer, lighttpd, squid and php5. Please read them carefully and take the proper measures.

New and noteworthy packages

ddclient (an utility to get access to home servers despite having a dynamic IP), FlameRobin (a GUI to Administer Firebird/Interbase SQL servers) and logstalgia (a pong-like apache log viewer) where presented by Debian Package of the Day.

Work-needing packages

Currently 444 packages are orphaned and 125 packages are up for adoption. Please take a look at the recent reports to see if there are packages you are interested in or view the complete list of packages which need your help.

Want to continue reading DPN?

Please help us create this newsletter. We still need more volunteer writers to watch the Debian community and report about what is going on. Please see the contributing page to find out how to help. We're looking forward to receiving your mail at debian-publicity@lists.debian.org.

logstalgia: pong-like apache log viewer

published on Sun Oct 5 05:00:24 2008 in packages-news

Article submitted by Andrew Caudwell

Logstalgia (inspired by glTail) is a website traffic visualization tool that replays or streams Apache access logs as a pong-like battle between the web server and an unrelenting army of requesting hosts. It is rendered using OpenGL, so you’ll need a 3D accelerated video card to run logstalgia.

Logstalgia

Requests appear as colored balls (the same color as the host) which travel across the screen to arrive at the requested location. Successful requests are hit by the pong paddle while unsuccessful ones (such as 404s) are missed and pass through.

The paths of requests are summarized within the available space by identifying common path prefixes.

Related paths are grouped together under headings. For instance, by default paths ending in png, gif or jpg are grouped under the heading Images. Paths that don’t match any of the specified groups are lumped together under a Miscellaneous section. Groups can be customized to the page layout of your website fom the command line by specifying a heading, an associated regular expression and a screen percentage.

The simulation can be paused at any time by pressing space. While paused, individual requests can be inspected by passing over them with the mouse.

Logstalgia can read from either a file or standard input. To replay an apache log just run:

logstalgia /var/log/apache/access.log

You can combine Logstalgia with other tools like tail and ssh to watch the access.log of your web server in real time. eg:

ssh user@yourserver.com tail -f /var/log/apache/access.log | logstalgia -

Check out a video of Logstalgia in action:

Logstalgia is available in Debian since Lenny and in Ubuntu intrepid. A version of the package for Debian Etch is available on the homepage.