DebCamp9 video streams
on 24.07.2009, 16:19
in debconf-news
logcheck: brilliantly simple log monitoring
on 19.07.2009, 05:00
in packages-news
logcheck: brilliantly simple log monitoring
on 19.07.2009, 05:00
in packages-news
DebCamp9 haz internets
on 16.07.2009, 17:59
in debconf-news
DebConf9 schedule
on 06.07.2009, 20:09
in debconf-news

DebCamp9 video streams

published on Fri Jul 24 16:19:00 2009 in debconf-news

DebConf9 has finally begun with OpenDay today, Information about video streams is available on our wiki and now I need to stop blogging for our daily organisational meeting, which we conviniently timed at lunch time, so we have a good reason to keep it short :-)

We are still looking for volunteers for various jobs: food ticket checking, frontdesk work, videoteam work, announcing the speakers and making sure everything is fine with the talks - so if you are here and can spare a few hours once or on several days, please come to frontdesk and speak with us!

Enjoy!

logcheck: brilliantly simple log monitoring

published on Sun Jul 19 05:00:33 2009 in packages-news

Logcheck is a simple yet great idea, an almost set-it-and-forget-it way to monitor your server logs for problems of all kinds. You create three pattern (grep regex) lists:

  • Known bad stuff
  • Looks bad but isn’t
  • Known good stuff

Logcheck periodically checks various syslog (or other) log files and picks up where it left off the last time. During each run it takes the new messages and looks for “known bad” things but first removes stuff that “looks bad but isn’t” and saves the messages as “this is known to be bad.” Then it rewinds, removes the known bad it just collected, removes the “known good” and stuff that “looks bad but isn’t” and saves whatever is left as “unknown.” Then it emails you the results.

Over time, as you tune your files, you end up only being alerted to known bad or new (not yet classified) stuff. Brilliant. I even did a (cheesy) Windows port of it.

Originally written by Marcus J. Ranum and Fred Avolio as frequentcheck.sh for the TIS Gauntlet firewall toolkit, it was adapted by Craig Rowland and applied to system logs. It spent some time as logsentry as part of Psionic’s Abacus/Sentry tools until they were bought by Cisco and the tools moved to SourceForge. The version in Debian is a re-write which was then inherited by Ubuntu.

But the best about the Debian/Ubuntu implementation is that almost all of the patterns you need are already Just There™. I usually only have to add a handful to work around odd things I’m doing or minor bugs. See the example at the bottom.

If you run a server you should be using fcheck and logcheck. And probably tmpreaper, etckeeper and maybe monit too. Articles about all these tools will be published soon, stay tuned!

As drawbacks, it should be noted that it may require some tuning, especially on a workstation or newer distro versions, and that may not be scalable for a lot of servers.

Is also worth mentioning that there are a variety of commercial and Managed Security Monitoring solutions that will scale and provide more information about events, but none are this easy.

Availability

The logcheck package is available in Debian since at least etch, and in Ubuntu since at least Dapper. See also the logcheck-database package.

Sample configuration

/etc/logcheck/ignore.d.server/LOCAL.ignore (lines wrapped for readability)

# /usr/sbin/logcheck automatically removes blank lines and comments.
# See 'man run-ports' for file name restrictions.

# For testing, create a sample log file and:
# su -s /bin/bash -c "/usr/sbin/logcheck -tsol sample" logcheck
# e.g.: su -s /bin/bash -c "/usr/sbin/logcheck -tsol /tmp/mylog" logcheck

# # DHCP Client lease renewals
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ dhclient: New
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ dhclient: DHCP(REQUEST|ACK)
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ NetworkManager:   DHCP daemon state
is now 3 \(renew\) for  interface

# # NTP, usually: 4001/0001
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ntpd\[[0-9]+\]: kernel time sync status
change [0-9]+

# # Syslog restarts (morning or all)
# ^\w{3} [ 0-9]{2} 07:[45][:0-9]{4} [._[:alnum:]-]+ syslogd
1\.5\.0#[0-9]ubuntu[0-9]: restart\.
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ syslogd 1.5.0#[0-9]ubuntu[0-9]: restart\.

# # fcheck
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ fcheck: “INFO: Rebuild of the fcheck
database  /var/lib/fcheck/fcheck\.dbf begun for [._[:alnum:]-]+ using config
file /etc/fcheck/fcheck\.cfg”

# # lm-sensors (normal)
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ kernel: \[[0-9. ]+\] CPU[01]:
Temperature/speed normal
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ kernel: \[[0-9. ]+\] Machine check events
logged

# # Wireless
# ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ NetworkManager:   \(eth1\):
supplicant connection state:

logcheck: brilliantly simple log monitoring

published on Sun Jul 19 05:00:33 2009 in packages-news

Article submitted by JP Vossen. DebADay needs you more than ever! Please submit good articles about software you like!

Logcheck is a simple yet great idea, an almost set-it-and-forget-it way to monitor your server logs for problems of all kinds. You create three pattern (grep regex) lists:

  • Known bad stuff
  • Looks bad but isn’t
  • Known good stuff

Logcheck periodically checks various syslog (or other) log files and picks up where it left off the last time. During each run it takes the new messages and looks for “known bad” things but first removes stuff that “looks bad but isn’t” and saves the messages as “this is known to be bad.” Then it rewinds, removes the known bad it just collected, removes the “known good” and stuff that “looks bad but isn’t” and saves whatever is left as “unknown.” Then it emails you the results.

Over time, as you tune your files, you end up only being alerted to known bad or new (not yet classified) stuff. Brilliant. I even did a (cheesy) Windows port of it.

Originally written by Marcus J. Ranum and Fred Avolio as frequentcheck.sh for the TIS Gauntlet firewall toolkit, it was adapted by Craig Rowland and applied to system logs. It spent some time as logsentry as part of Psionic’s Abacus/Sentry tools until they were bought by Cisco and the tools moved to SourceForge. The version in Debian is a re-write which was then inherited by Ubuntu.

But the best about the Debian/Ubuntu implementation is that almost all of the patterns you need are already Just There™. I usually only have to add a handful to work around odd things I’m doing or minor bugs. See the example at the bottom.

If you run a server you should be using fcheck and logcheck. And probably tmpreaper, etckeeper and maybe monit too. Articles about all these tools will be published soon, stay tuned!

As drawbacks, it should be noted that it may require some tuning, especially on a workstation or newer distro versions, and that may not be scalable for a lot of servers.

Is also worth mentioning that there are a variety of commercial and Managed Security Monitoring solutions that will scale and provide more information about events, but none are this easy.

Availability

The logcheck package is available in Debian since at least etch, and in Ubuntu since at least Dapper. See also the logcheck-database package.

Sample configuration

/etc/logcheck/ignore.d.server/LOCAL.ignore (lines wrapped for readability)

# /usr/sbin/logcheck automatically removes blank lines and comments.
# See 'man run-ports' for file name restrictions.

# For testing, create a sample log file and:
# su -s /bin/bash -c "/usr/sbin/logcheck -tsol sample" logcheck
# e.g.: su -s /bin/bash -c "/usr/sbin/logcheck -tsol /tmp/mylog" logcheck

# # DHCP Client lease renewals
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ dhclient: New
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ dhclient: DHCP(REQUEST|ACK)
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ NetworkManager:   DHCP daemon state
is now 3 (renew) for  interface

# # NTP, usually: 4001/0001
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ ntpd[[0-9]+]: kernel time sync status
change [0-9]+

# # Syslog restarts (morning or all)
# ^w{3} [ 0-9]{2} 07:[45][:0-9]{4} [._[:alnum:]-]+ syslogd
1.5.0#[0-9]ubuntu[0-9]: restart.
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ syslogd 1.5.0#[0-9]ubuntu[0-9]: restart.

# # fcheck
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ fcheck: “INFO: Rebuild of the fcheck
database  /var/lib/fcheck/fcheck.dbf begun for [._[:alnum:]-]+ using config
file /etc/fcheck/fcheck.cfg”

# # lm-sensors (normal)
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ kernel: [[0-9. ]+] CPU[01]:
Temperature/speed normal
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ kernel: [[0-9. ]+] Machine check events
logged

# # Wireless
# ^w{3} [ :0-9]{11} [._[:alnum:]-]+ NetworkManager:   (eth1):
supplicant connection state:

DebCamp9 haz internets

published on Thu Jul 16 17:59:00 2009 in debconf-news

The basic infrastructure has already been set up: we have internet at the main venue \o/ via a 20mbit symetric connection. Currently cat5 cables are being placed in the venue, the work to setup the wireless APs has begun, we have a basic frontdesk and the catering is also already providing us with food. Even the temperature is still bearable - inside and outside in the shadow ;-)

What we are lacking mostly atm are attendees and volunteers - if you can dedicate some time during DebConf or Camp, please talk to the frontdesk and look at our ToDo list.

DebConf9 schedule

published on Mon Jul 6 20:09:00 2009 in debconf-news

The schedule for the upcoming DebConf9 is available. Most of it should be set already, but of course there still can be small changes until the conference starts, and honestly, until it ends :-) But we will try our best to avoid changes less then 48h in advance - and we count on you all here :-)

As usual we offer various ways to access the schedule:

  • Plain nice html export, available for everyone without needing a login.
  • The same as above but needs a login to access. This one has one extra feature above the non-login version, you can rate an event and its speaker(s) after you attended it.
  • An iCal file
  • A xCal file
  • A XML file

This should leave enough options for you to deal with it, have fun. :)