Tuesday, September 24, 2013

Puppet Time through abuse of inline_template

Stardate 91334.3

I was asked a pretty reasonable question about puppet:

Can I get access to the time in Puppet without resorting to writing a fact?

This, seemingly reasonable task, is not so easy to do. Puppet does not define this for you. However, we can use the inline_template() function for great good.

For the uninitiated, inline_template() calls out to the ERB templating processor without having to have a template file. This is very useful for simple file resources or conjunction with the file_line resource from the Puppet Labs stdlib module.

file { '/etc/motd':
   ensure  => file,
   content => inline_template("Welcome to <%= @hostname %>"),
}

However we can abuse this by doing anything in ERB that we can do in Ruby. A friend of mine, famililar with the jinja templating system from Python, remarked: 'so its like PHP, and I'm not even being derogatory.' This means we can acces time using Ruby's built in time modules.

$time = inline_template('<%= Time.now %>')
However this is being evaluated on the Puppet Master, not the node. So if the two are in different timezones, what then? The first way to improve this is to set it to utc.
$time = inline_template('<%= Time.now.utc %>')
But we can actually go further and define two variables, one for time in UTC of catalog compilation and one for local time for the checking in node. While we don't have a fact for the time on the node, we do have a fact for its timezone.
$time_utc = inline_template('<%= Time.now.utc %>')
$time_local = inline_template("<%= (@time + Time.zone_offset(@timezone)).strftime('%c') %>")
We use the strftime('%c') to strip the UTC timezone label off of the time.

Going further:

We can take this a step further by using the inline_template() function to do time comparisons:


$go_time = "2013-09-24 08:00:00 UTC" # We could seed this in hiera!

$time = inline_template("<%= Time.now.utc %>")

if str2bool(inline_template("<%= Time.now.utc > Time.parse(@go_time) %>")){

    notify { "GO GO GO: Make the changes!": }

}
What the above code does is gate changes based on time. This allows us to 'light the fuses' and only make changes to production after a certain time, after a downtime window begins for instance. Note that the Puppet clients are still going to check in on their own schedule, but since we know what time our downtime is starting, we can use Puppet to make sure they kick off a Puppet run at the right time.

$go_time   = "2013-09-24 08:00:00 UTC" # We could seed this in hiera!
$done_time = "2013-09-24 12:00:00 UTC" # We could seed this in hiera, too!

$time = inline_template("<%= Time.now.utc %>")

if str2bool(inline_template("<%= Time.now.utc > Time.parse(@go_time) %>")){

  notify { "GO GO GO: Make the changes!": }

  cron { 'fire off the puppet run':
     command => 'puppet agent --no-daemonize',
     day     => '24', # we can seed the date here in hiera, albiet more verbosely 
     hour    => '8',
     minute  => '1',
     user    => 'root',
     ensure  => 'absent',
  }

} else { 

  cron { 'fire off the puppet run':
     command => 'puppet agent --no-daemonize',
     day     => '24', # we can seed the date here in hiera, albiet more verbosely 
     hour    => '8',
     minute  => '1',
     user    => 'root',
     ensure  => 'present',
  }
} 
What is this doing? We put this code out at 4pm. Get some dinner. Log in at 7:30 pm and wait for our 8:00 pm downtime. In Puppet runs before 8:00 pm a cronjob will be installed that will effecat a Puppet run precisely one minute after the downtime begins. In all Puppet runs before 8:00 pm, the resources that are potentially hazardous are passed over. But in all Puppet runs after 8:00 pm, the new state is ensured and the cronjob is removed. Then this code, which should define the new state of the system, can be hoisted into regular classes and defined types.

Monday, September 16, 2013

Custom categories with Puppet data in modules

In the old way of doing things, we would have a hierarchy in our hiera.yaml that looked something like this:

:hierarchy:
  - defaults
  - %{clientcert}
  - %{environment}
  - global

In the new way the hierarchy has been renamed categories, and each level of it is a category.

We can define category precedence in the system wide hiera.yaml, the module specific hiera.yaml, and the binder_config.yaml

The following binder_config.yaml will effectively insert the species category into the category listing:


---
version: 1
layers:
  [{name: site, include: 'confdir-hiera:/'},
   {name: modules, include: ['module-hiera:/*/', 'module:/*::default'] }
  ]
categories:
  [['node', '${fqdn}'],
   ['environment', '${environment}'],
   ['species', '${species}'],
   ['osfamily', '${osfamily}'],
   ['common', 'true']
  ]

This means we can use the species category if one is defined in a module. An example hiera.yaml from such a module is:
---
version: 2
hierarchy:
  [['osfamily', '$osfamily', 'data/osfamily/$osfamily'],
   ['species', '$species', 'data/species/$species'],
   ['environment', '$environment', 'data/env/$environment'],
   ['common', 'true', 'data/common']
  ]
Which means when we run Puppet...

root@hiera-2:/etc/puppet# FACTER_species='human' puppet apply modules/startrek/tests/init.pp 
Notice: Compiled catalog for hiera-2.green.gah in environment production in 1.07 seconds
Notice: janeway commands the voyager
Notice: /Stage[main]/Startrek/Notify[janeway commands the voyager]/message: defined 'message' as 'janeway commands the voyager'
Notice: janeway is always wary of the section 31
Notice: /Stage[main]/Startrek/Notify[janeway is always wary of the section 31]/message: defined 'message' as 'janeway is always wary of the section 31'
Notice: Finished catalog run in 0.11 seconds

You can see full example code in the startrek module.

You can pre-order my book, Pro Puppet 2nd Ed, here.

Puppet 3.3 and Data in Modules

Puppet 3.3 was released last week. As part of that release, hiera2 and puppet data in modules is available in the testing mode. I have been working with William van Hevelingen to build an example module/tutorial on Puppet data in Modules. Our example module is available at: https://github.com/pro-puppet/puppet-module-startrek. We hope to expand this further as the community learns more about how to use this new feature.