Hi all,
This is one of those “complicated” problems I mentioned a bit ago.
I’m looking for help simplifying some of Puppet’s internals,
specifically the parts related to transactions and idempotency.
Transactional support is important for the standard reasons plus good
reporting, and idempotency allows me to apply a configuration and
only change the bits that are out of compliance, so the same
configuration can be applied, say, every half hour and it will just
fix anything that’s somehow gotten broken, which is both safer and
faster.
Puppet’s idempotency and transactions are currently closely related.
Puppet is organized around a lot of high-level types like ‘user’,
‘group’, ‘package’, and ‘service’. Each of these types has
parameters that affect how they function (like ‘loglevel’ and
‘recurse’) and parameters that actually modify the system (like ‘uid’
and ‘gid’ on files). The complicated ones are those that modify the
system.
Currently, each of these parameters is defined in a separate class,
and that class has to define a ‘retrieve’ and a ‘sync’ method.
‘retrieve’ sets ‘@is’ to be the current value (e.g., for UID on a
file, it would do a stat and get the UID from the stat), and ‘sync’
basically takes the desired configuration (in ‘@should’) and modifies
the system so it matches. So, the types have hooks for idempotency.
It’s the transactions that actually provide idempotency. I create a
transaction and pass it a list of type instances (e.g., a bunch of
files, services, whatever). It steps through each instance, checks
to see if the instance is out of sync, and syncs any parameters that
are out of sync. This gives me great reporting, because the
transaction can always log exactly what it’s doing, and rollback is
pretty easy because I can just switch ‘@is’ and ‘@should’ and sync
again for most cases.
There are two significant problems with this scenario: First, it’s
pretty annoying to have to maintain @is and @should separately in the
parameters. It would be much, much better if ‘retrieve’ and ‘sync’
could just work like getter and setter methods, returning or
accepting a value. Second, this system makes it pretty annoying for
someone else to use the library. I want to get to the point where
anyone can use the Puppet library to make changes to the system, but
for that to happen, the library interface needs to be simple. I want
something like this:
sudoers = Puppet::Type.create(:type => :file, :path => “/etc/
sudoers”)
sudoers.uid = 0 unless sudoers.uid == 0
Instead, you pretty much have to use a transaction to do any work
right now:
sudoers = Puppet::Type.create(:type => :file, :path => “/etc/
sudoers”)
trans = Puppet::Transaction.new(sudoers)
trans.evaluate
It’s totally unclear what’s going on there, and it’s not exactly easy
to use. I’d also like to make it simple for people to use
transactions if they want, but I want it to be a good bit simpler:
report = Puppet.transaction do
sudoers = Puppet::Type.create(:type => :file, :path => “/etc/
sudoers”)
sudoers.uid = 0 unless sudoers.uid == 0
end
That way people could still get the logging and rollback that always
come with transactions, but only if they wanted them and in a way
that they can see what’s happening. By the way, the objects often
live much longer than the transactions – I have a long-running
daemon that instantiates the objects once and applies them all in a
new transaction every half hour.
I think all of these problems (getting rid of ‘@is’ and ‘@should’,
simplifying transactional use, and simplifying use of the objects)
can have a single solution, but I don’t know what it is. It could be
something like objects somehow knowing whether they’re running under
a transaction, but I don’t know how I’d do that without making
transactions either a singleton (which I can’t afford, because I know
sometimes I’ll need subtransactions) or very complex (e.g., creating
a ‘transaction’ instance variable for every object, and then nil’ing
that variable at the end of the transaction).
Anyone have any ideas? Any recommendations for what you’d want this
library interface to look like, either using transactions or not?
If you want to look at the code more closely, you can get it from svn
at http://reductivelabs.com/svn/puppet/trunk, or in Trac at https://
reductivelabs.com/cgi-bin/puppet.cgi/browser/trunk . The transaction
class is relatively straightforward, and most of the types are simple
enough to understand, although the Type baseclass is a bit long and
messy for my tastes.