My two ¢'s

puppetmasterd and passenger

daniel — Thu, 18 Mar 2010 17:30:16 +0000

It seems like some people have trouble configuring/undestanding how to combine puppetmasterd and Passenger (aka mod_rails). Let’s get it on:

Install depencies

You need the following components on your puppetmaster server:

Phusion Passenger (http://modrails.com/)
Rack (http://rubyforge.org/projects/rack)

Configure puppetmaster

Your puppet package should contain a config.ru. I found mine in /usr/share/doc/puppet-0.25.4-r1/ext/rack/files/config.ru.bz2 (Gentoo).

mkdir /etc/puppet/rack
mkdir /etc/puppet/rack/public
cp [YOUR_CONFIG.RU] /etc/puppet/rack
chown puppet:root /etc/puppet/rack/config.ru

The final chown line is important! This way rack determines under which user to run the puppetmaster processes.

Add the following lines to your puppet.conf:

[puppetmasterd]
...
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY

Apache config

You can keep your passenger config as is and modify it when required. Here’s a example vhost config:

Listen 8140
 
<VirtualHost *:8140>
    ServerName puppet
    DocumentRoot /etc/puppet/rack/public/
 
    CustomLog "|/usr/sbin/rotatelogs /var/www/puppet/logs/access_log.%Y%m%d-%H%M 86400" common
    ErrorLog  "|/usr/sbin/rotatelogs /var/www/puppet/logs/error_log.%Y%m%d-%H%M 86400"
 
    PassengerHighPerformance on
    PassengerMaxPoolSize 15
    PassengerPoolIdleTime 300
    PassengerUseGlobalQueue on
    PassengerStatThrottleRate 120
    RackAutoDetect Off
    RailsAutoDetect Off
 
    RackBaseURI /
 
    SSLEngine on
    SSLProtocol -ALL +SSLv3 +TLSv1
    SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP
    SSLCertificateFile /var/lib/puppet/ssl/certs/XXXXXXXXXXXX.pem
    SSLCertificateKeyFile /var/lib/puppet/ssl/private_keys/XXXXXXXXXXXX.pem
    SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCACertificateFile /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCARevocationFile /var/lib/puppet/ssl/ca/ca_crl.pem
    SSLVerifyClient optional
    SSLVerifyDepth 1
    SSLOptions +StdEnvVars
 
    <Directory "/etc/puppet/rack/public/">
        Options None
        AllowOverride None
 
        Order allow,deny
        Allow from all
    Directory>
Virtualhost>

Restart apache and when clients connect are are triggered via puppetrun you may see something like this with passenger-status:

passenger-status
----------- General information -----------
max      = 20
count    = 9
active   = 0
inactive = 9
Waiting on global queue: 0
 
----------- Domains -----------
/etc/puppet/rack:
  PID: 19160   Sessions: 0    Processed: 39      Uptime: 20s
  PID: 19202   Sessions: 0    Processed: 70      Uptime: 17s
  PID: 18934   Sessions: 0    Processed: 95      Uptime: 45s
  PID: 18977   Sessions: 0    Processed: 66      Uptime: 42s
  PID: 19008   Sessions: 0    Processed: 63      Uptime: 40s
  PID: 19184   Sessions: 0    Processed: 2       Uptime: 19s
  PID: 19103   Sessions: 0    Processed: 8       Uptime: 28s
 
/var/www/puppet-dashboard/htdocs:
  PID: 19158   Sessions: 0    Processed: 6       Uptime: 22s
  PID: 19236   Sessions: 0    Processed: 3       Uptime: 10s

Also refer to the offical puppet documentation on passenger here.

That’s it.

OpenVPN active/active cluster

daniel — Mon, 01 Feb 2010 18:49:10 +0000

Code snippets are for Gentoo systems but it should be easy to adapt this for other systems

This is a small howto explaining how to run a active/active Cluster (keeplaived) setup with OpenVPN. The active/active reflects that both cluster nodes run the same OpenVPN instance. In server mode this setup leads to routing problems as both nodes have the tunnel route added during startup (not after connect). This results in routing trouble as i needed the passive node to access the VPN tunnel via the active node. This is how i solved it:

Routing setup

Both firewall nodes have a static route which forwards tunnel traffic to one of the internal cluster IP’s. The metric for this route is 2 so a active tunnel is preferred over that static route.

routes_eth2=( "192.168.44.0/24 via 192.168.20.254 metric 2" )

It doesn’t matter which node is active and gets the VPN connects as the other node has the right routing entries.

OpenVPN configuration

This setup requires the use of sudo or to run the OpenVPN daemon as root (sudo, sudo, sudo!!!). First we disable to automatically adding of routes and specify scripts for client-connect and client-disconnect. Dont forget to set script security.

route-noexec
client-connect /etc/openvpn/cluster_routing.sh
client-disconnect /etc/openvpn/cluster_routing.sh
script-security 3

cluster_routing.sh

This is a simple version of the script but it should be sufficient to work in most scenarios.

#!/bin/bash
 
## This is useful for debugging and to get the available env vars
##exec > /tmp/ovpn.debug.$$ 2>&1; set -x
 
if [ "${script_type}" == "client-connect" ];
then
    /usr/bin/sudo /sbin/route add -net ${route_network_1}/24 gw ${route_vpn_gateway}
else
    /usr/bin/sudo /sbin/route del -net ${route_network_1}/24 gw ${route_vpn_gateway}
fi
 
exit 0

This script adds and removes the needed route to get a operational tunnel.

Short review: Puppet for Gentoo servers

admin — Fri, 18 Dec 2009 10:51:58 +0000

I recently started to integrate Puppet with my company’s OS installer to build custom Gentoo servers in almost no time. The install/build system reached a stable state and i want to share some information’s on what i did to get it working. The Gentoo support of puppet is not perfect but sufficient for my use case. There are still some issues to solve:

Slots don’t work (I’m working on a patch for the portage provider to address this issue)
No nice way to manage /etc/conf.d/net
Only the runlevel “default” can be managed (This is sufficient for most cases)
No built in USE flag support (i use a binhost so this doesn’t really affect my setup). Check this site for a possible solution

The missing slots integration is especially important when it comes to Tomcat. Tomcat requires sun-jdk-1.5 and sun-jdk-1.6. I solved this by adding sun-jdk-1.5 to our install image. Apart from this problems it works very well.

Binhost setup

The easiest way to ensure that all systems run the same software (version, use flags) is to setup a portage binhost and force all clients to use this server as only package source. There are a lot of howto’s out there on creating a binhost so i won’t explain it detail. To force all clients to use only binary packages set the following statements in make.conf (Puppet distribution!):

PORTAGE_BINHOST="http://192.168.1.1/x86_64"
EMERGE_DEFAULT_OPTS="--getbinpkgonly --usepkgonly"

The parameter EMERGE_DEFAULT_OPTS is important because Puppet will run the command “emerge xxx/yyy” so you can’t specify extra parameters. This parameter setup ensures that the package cannot be installed when the binary package is missing on the binhost.

New helper script: SVN updater using Growl notifications

daniel — Wed, 27 May 2009 10:16:27 +0000

This small helper script should be added to cron and checks your local working copies against the online repository. It can automatically update your working copies if a newer revision is available. The user will be informed using Growl popups. As Growl is only available for Mac users it’s a Mac only tool by now.

The script currently registers with Growl using the classes from Python bindings. This is not necessary and would also be done by growlnotify. I was just interested in how this whole Growl stuff works. Feel free to remove this parts.

When i started i was willing to do the whole Growl stuff manually which has worked very well from the command line but not from cron. Seems like the context or something else get’s lost and no notifications pop up anymore. To be able to run via cron i switched to growlnotify.

Depencies

Screenshots

Download

Download:

svn_growl_updater / version 0.3 / size 60 kB / dlcount 58

New Project: Django-powered Nagios configuration

daniel — Thu, 05 Mar 2009 14:11:32 +0000

It’s time for a new private project: The Django-powered Nagios configuration.

The idea resulted in reading the book “Python Web Development with Django”. One of the authors describes one of his projects for internal use: a Nagios config manager and i think it’s a great idea. I just played around with the idea but it got pretty usable so far. It’s not in a production ready state but as soon as it may be usable to you i will release it. Here’s a development screenshot:

Modify monitored host

More informations to come… Stay tuned

Using Django with Komodo IDE

daniel — Tue, 10 Feb 2009 20:44:58 +0000

I recently started to work on a new Django project and tried use my default IDE: Komodo IDE. Unfortunately it didn’t work out the way i wanted. Code cpmpletion didn’t work just as the import of my app. Here’s how i solved it:

Add a new Python import directory: Preferences -> Languages -> Python

My Django resides in /Library/Python/2.5/site-packages. Your installation may be in a different location.
Create a Komodo project for your new Django project and edit it’s properties: Properties -> Languages -> Python My project resides in /Users/dkerwin/development/django/stag_party with several application inside.

Installing iPython under MacOS X

daniel — Mon, 02 Feb 2009 12:54:14 +0000

If you don’t know iPython by now: check it out

It’s a pretty nice tool if you work with python from the terminal. Especially the easy way to get information about modules and functions. The easiest way to install (especially in a Mac environment) is to use the “alldeps” tarball.

deathstar-mac ~ $ wget http://ipython.scipy.org/dist/alldeps/ipython-alldeps-0.9.1.tar
deathstar-mac ~ $ tar xvf ipython-alldeps-0.9.1.tar
deathstar-mac ~ $ cd ipython-alldeps-0.9.1.tar
deathstar-mac ~ $ make

The default install location is ~/usr/local. You may change this by modifying the install scripts but for now use the default. Now it’s time to make some modifications to your profile. Add the following lines:

deathstar-mac ~ $ export PATH="$PATH:/Users/YOUR_USERNAME/usr/local/bin"
deathstar-mac ~ $ export PYTHONPATH=$PYTHONPATH:/Users/YOUR_USERNAME/usr/local/lib/python2.5/site-packages

Let’s complete the installation process:

deathstar-mac ~ $ iptest
deathstar-mac ~ $ cd ipython-alldeps-0.9.1/ipython-0.9.1
deathstar-mac ~ $ python setup.py install --prefix=~/usr/local
 
deathstar-mac ~ $ ipython
Leopard libedit detected.
Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26) 
Type "copyright", "credits" or "license" for more information.
 
IPython 0.9.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.
 
In [1]:

Enjoy!

Inbound Traffic Shaping made (pretty) easy

daniel — Wed, 28 Jan 2009 17:50:40 +0000

It’s not that easy to get trafic shaping done with Linux especially when it comes to inbound traffic. After some serious research i found this howto which is a excellent starting point. The supplied script can be easily customized and works pretty good. If there’s interest i can post my version for incoming traffic only.

You have to apply the IMQ patch (can be found here) to your kernel to get this working. IMQ is necessary because Linux can only limit outgoing traffic with builtin kernel settings.

Zattoo: Watch (free) TV on your PC

admin — Wed, 10 Dec 2008 23:59:15 +0000

Always been looking for a good, reliable, fast and free solution to watch TV on your PC? Take a look at Zattoo. There are binaries for Mac, M$ and Linux. Give it a try!

Perl: Use of “flock”

daniel — Wed, 10 Dec 2008 21:31:39 +0000

This is a small example on flock. It may help you to prevent multiple running instances of the same script. Assume you run the script via cron and it may not be finished when cron attempts to start it again. This few lines of code solve this issue.

if ( ! open( LOCK, ">/var/run/my_app.lock" ) ) {
    print "Failed to open lock file: $!\n";
    exit(1);
}
 
## Create exclusive, non blocking lock: LOCK_EX(2) + LOCK_NB(4)
if ( ! flock( LOCK, 6 ) ) {
    close( LOCK );
 
    if ( ! open( PID, " ) ) {
 
        print "Failed to read PID file: $!\n";
        exit(1);
    }
 
    my $pid = ;
    close( PID );
 
    print "Failed to accquire lock. Another instance (PID $pid) running!\n";
    exit(1);
}
else {
    if ( ! open( PID, ">/var/run/my_app.pid" ) ) {
 
        print "Failed to open pid file: $!\n";
        exit(1);
    }
 
    print PID $$;
 
    if ( ! close( PID ) ) {
        print "Failed to write PID file: $!\n";
        exit(1);
    }
}

That’s it!