that’s whacked

thinking out loud

Monday, Dec 12. 2005  –  Category: Musings, OpenSolaris

i’m off to sunny st. paul, MN in a few hours for the TOSTA conference. looking forward to it…

got into work early this morning to crank out some code before i go. i’m working on an interesting problem, mirrouring ON externally via a read-only Subversion gateway.

internally, we use TeamWare/SCCS for SCM (source code management). the long-term plan is to adopt a new SCM solution that will enable us to move the gate outside of Sun so that everyone can have equal access to it. this requires a long, complicated evaluation process to whittle down the candidates that probably involves pagan rituals, games of Twister, and undoubtedly much drinking.

in the meantime, i’m trying to setup a read-only Subversion gateway that will at least enable people to checkout code without having to download an 80 megabyte tarball each time.

enter sccs2svn, a handy open-source script written by Robert Zeh to faciliate SCCS->SVN conversion. very cool. very time-saving. it borked on ON due to the sheer size of some of our changesets, but a quick hack to break up large changesets into multiple smaller ones seems to have fixed that.

the wrinkle to our specific problem is “squelch”-ing. this is my script which goes through the history of ON and removes any comments/code-deltas pre-2000/06/14, and any code-deltas in between 2000/06/14->2005/06/14 (but leaving comments for that time-frame). this keeps the lawyers happy.

so there are two problems i’m facing:

1. doing incremental squelching
2. incremental SCCS->SVN conversion

while a full squelch-ing run is not long (takes about 40-45 minutes for ON), it’s still tedious and a waste of time/resources. unfortunately, once a file has been squelched – it is no longer “bringover-able”, meaning you can’t update it simply within the realm of SCCS updates. so i’ve been working on a Perl script triggered by a procmail recipe which monitors the onnv-gate-notification mails, and will parse the list of files updated/renamed/removed/created by a putback, copy over the new files, and re-squelch them incrementally. but then there is the matter of serialising these actions. i.e., let’s say the order happens in:

1. person FOO puts back files A,B,C
2. person BAR puts back files D,E,F
3. person QWERTY does a putback removing file B

actions 1 & 2 can happen in parallel since they are independent of each other. but action 3 shouldn’t happen until action 1 occurs. but if all 3 happen quickly (within a minute of each other, let’s say), then procmail spins off 3 instances of Perl, one handling each event. while event 1 is happening, let’s say it’s processing/re-squelching file A, then event 3 also occurs removing file B from ON. event 1 then goes on to update and re-squelch file B, so file B is now back in ON.

bad. very bad. we need to serialise this shit.

so now i’ve got something mocked up which uses a client/server model & sockets. a Perl daemon stays running the whole time, and everytime procmail spawns the handler for the recipe, it calls a client which connects to the server and sends the request. the server can then evaluate whether the action can be allowed to happen in parallel, or if it must wait for a previous action to complete first.

does this seem reasonable? or is this overkill? i can’t decide. i’m sure something cool can be done using simpler file locking primitives… but i was in the mood to cook me up some sockets. plus there’s something nice about the alliteration of using ‘sockets to solve the problem of serialising this shit’.

the incremental SCCS->SVN update step is the next one to tackle once i iron out the details of this serialisation problem. i haven’t thought much about it, but on the surface it seems easy right?