# Refactoring transactional support within Puppet (long) **URL:** \ **Category:** Ruby\ **Created:** [August 18, 2006, 8:06pm UTC](https://www.ruby-forum.com/t/refactoring-transactional-support-within-puppet-long/65502 "2006-08-18T20:06:07Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![lynxinator](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/848f3c/32.png) [@lynxinator](https://www.ruby-forum.com/u/lynxinator)\ **Post date:** [August 18, 2006, 8:06pm UTC](https://www.ruby-forum.com/t/refactoring-transactional-support-within-puppet-long/65502/1 "2006-08-18T20:06:07Z")
Hi all, This is one of those “complicated” problems I mentioned a bit ago. I’m looking for help simplifying some of Puppet’s internals, specifically the parts related to transactions and idempotency. Transactional support is important for the standard reasons plus good reporting, and idempotency allows me to apply a configuration and only change the bits that are out of compliance, so the same configuration can be applied, say, every half hour and it will just fix anything that’s somehow gotten broken, which is both safer and faster. Puppet’s idempotency and transactions are currently closely related. Puppet is organized around a lot of high-level types like ‘user’, ‘group’, ‘package’, and ‘service’. Each of these types has parameters that affect how they function (like ‘loglevel’ and ‘recurse’) and parameters that actually modify the system (like ‘uid’ and ‘gid’ on files). The complicated ones are those that modify the system. Currently, each of these parameters is defined in a separate class, and that class has to define a ‘retrieve’ and a ‘sync’ method. ‘retrieve’ sets ‘@is’ to be the current value (e.g., for UID on a file, it would do a stat and get the UID from the stat), and ‘sync’ basically takes the desired configuration (in ‘@should’) and modifies the system so it matches. So, the types have hooks for idempotency. It’s the transactions that actually provide idempotency. I create a transaction and pass it a list of type instances (e.g., a bunch of files, services, whatever). It steps through each instance, checks to see if the instance is out of sync, and syncs any parameters that are out of sync. This gives me great reporting, because the transaction can always log exactly what it’s doing, and rollback is pretty easy because I can just switch ‘@is’ and ‘@should’ and sync again for most cases. There are two significant problems with this scenario: First, it’s pretty annoying to have to maintain @is and @should separately in the parameters. It would be much, much better if ‘retrieve’ and ‘sync’ could just work like getter and setter methods, returning or accepting a value. Second, this system makes it pretty annoying for someone else to use the library. I want to get to the point where anyone can use the Puppet library to make changes to the system, but for that to happen, the library interface needs to be simple. I want something like this: sudoers = Puppet::Type.create(:type =\> :file, :path =\> “/etc/ sudoers”) sudoers.uid = 0 unless sudoers.uid == 0 Instead, you pretty much have to use a transaction to do any work right now: sudoers = Puppet::Type.create(:type =\> :file, :path =\> “/etc/ sudoers”) trans = Puppet::Transaction.new(sudoers) trans.evaluate It’s totally unclear what’s going on there, and it’s not exactly easy to use. I’d also like to make it simple for people to use transactions if they want, but I want it to be a good bit simpler: report = Puppet.transaction do sudoers = Puppet::Type.create(:type =\> :file, :path =\> “/etc/ sudoers”) sudoers.uid = 0 unless sudoers.uid == 0 end That way people could still get the logging and rollback that always come with transactions, but only if they wanted them and in a way that they can see what’s happening. By the way, the objects often live much longer than the transactions – I have a long-running daemon that instantiates the objects once and applies them all in a new transaction every half hour. I think all of these problems (getting rid of ‘@is’ and ‘@should’, simplifying transactional use, and simplifying use of the objects) can have a single solution, but I don’t know what it is. It could be something like objects somehow knowing whether they’re running under a transaction, but I don’t know how I’d do that without making transactions either a singleton (which I can’t afford, because I know sometimes I’ll need subtransactions) or very complex (e.g., creating a ‘transaction’ instance variable for every object, and then nil’ing that variable at the end of the transaction). Anyone have any ideas? Any recommendations for what you’d want this library interface to look like, either using transactions or not? If you want to look at the code more closely, you can get it from svn at [http://reductivelabs.com/svn/puppet/trunk](http://reductivelabs.com/svn/puppet/trunk), or in Trac at https:// [reductivelabs.com/cgi-bin/puppet.cgi/browser/trunk](http://reductivelabs.com/cgi-bin/puppet.cgi/browser/trunk) . The transaction class is relatively straightforward, and most of the types are simple enough to understand, although the Type baseclass is a bit long and messy for my tastes. ---
**Author:** ![lynxinator](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/848f3c/32.png) [@lynxinator](https://www.ruby-forum.com/u/lynxinator)\ **Post date:** [August 19, 2006, 10:51pm UTC](https://www.ruby-forum.com/t/refactoring-transactional-support-within-puppet-long/65502/2 "2006-08-19T22:51:38Z")
On Aug 18, 2006, at 2:11 PM, removed\_email\_address@domain.invalid wrote: > ``` > @in_transaction = false > @in_transaction = true > > ``` > > … > > def foo() t{ @foo = 42 } end > def bar() t{ @bar = 42 } end > def foobar() t{ foo and bar } end Hmm. If I do this, then at the very least I want to organize things so that the developer doesn’t need to know about transactions; either I use intermediate methods that handle the fact that it’s in a transaction, or I do some method-renaming so that the direct methods get replaced with methods that go through a transaction. That’s somewhat immaterial, though, I guess. You’re basic point is that there should only ever be one transaction at a time, right? There’d be no concept of sub-transactions, but anything that got done in the middle of a transaction would automatically be included in the transaction. I’m not sure about always working within a transaction; I’m not sure it’s reasonable to assume that every user of Puppet’s library will want to use transactions, but I’m not sure it harms anything to do so. I’d probably want things set up so that if there is a transaction, all work is done within that transaction, and if there is not one, then no transaction is used. So that gives me an idea of how to handle transactions essentially transparently (with some modification necessary to make it transparent to the developer, also), but I still need to figure out how to transparently handle the three-phase collect, compare, commit operations. It seems that some controlling process would need to do that, and currently my transaction is the controlling process, but with your recommendation the transaction moves completely into the background (which is probably where it belongs, largely). There’s also a lot of logging, error handling, and event handling that currently take place in the transaction, so I would need to translate that into these behind-the-scenes transactions, but I wouldn’t guess that would be too difficult. Thanks. ---
**Author:** ![lynxinator](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/848f3c/32.png) [@lynxinator](https://www.ruby-forum.com/u/lynxinator)\ **Post date:** [August 18, 2006, 9:15pm UTC](https://www.ruby-forum.com/t/refactoring-transactional-support-within-puppet-long/65502/3 "2006-08-18T21:15:17Z")
On Sat, 19 Aug 2006, Luke K. wrote: > with transactions, but only if they wanted them and in a way that they can > or very complex (e.g., creating a ‘transaction’ instance variable for every > object, and then nil’ing that variable at the end of the transaction). > > Anyone have any ideas? Any recommendations for what you’d want this library > interface to look like, either using transactions or not? 1. make transactions re-entrant AND singleton 2. make **all** operations take place in a transaction eg def initialize … @transaction\_mutex = Mutex.new @in\_transaction = false … end def transaction @transaction\_mutex.sychronize do if @in\_transaction yield else it = @in\_transaction begin @in\_transaction = true yield ensure @in\_transaction = it end end end end alias\_method “t”, “transaction” … def foo() t{ @foo = 42 } end def bar() t{ @bar = 42 } end def foobar() t{ foo and bar } end the state may have to global/module-level for this to work, but you get the idea. probably the easiest way is module Transaction # all transaction (global) state and methods end class C include Transaction end etc. 2 cts. -a