One of my proudest achievements in my IT career is when I managed to crash a mainframe through user input. Admittedly, I was trying to do it – I was employed as a tester. But there's a certain evil glee in realizing that it's down to you that everyone working on the system has just been torpedoed by your finding a bug (and it shows that you're doing something as well).
Of course, if I'd really taken down the voicemail server that served the 3 million customers of Bouygues Telecom, it might not have been so much fun. I guess that's why I was working off a test-bed.
Standard operating procedure; just as developers test their code before merging into the main software build, just as hardware engineers check operation after they install, most enterprises have a test environment where they install and test new software first, before moving it to production.
Hopefully none of the above is news to people, but the point is more why this is done. Not because people are incompetent, but because they are also not omniscient. It's simply unrealistic to expect coders, and designers, and architects, to get it right first time, every time. It's the same reason we have design reviews and code reviews – and even then, until you try the code out you can't be sure that there isn't something trivial that's been missed.
The thing is, just as code follows a set of instructions that get executed in specific situations, and just as the behavior of hardware follows a set of instructions that get executed in specific situations, style guides and standards provide a set of instructions that get executed in specific situations. It's just that they execute on a different platform – people.
Just as with code, it's fair to assume that the design of a given style guide or given standard may have missed something. Consider a naming standard that requires that we name software with the specific package, version number and patch level – how do you then design a high-level architecture that only requires a more generic family of that software?
This is not just idle worry. When software fails spectacularly, it diminishes the trust people place in that software. Same for hardware. And especially so for standards that aim to constrain and control the behavior of people... some of whom may well not be too keen on being constrained and controlled.
So if a given operating principle or naming standard or style guide is rolled out with much fanfare and then proves not to be fit for purpose, this is a major problem. It diminishes the credibility of the standard and those who created it – and it gives ammunition to the inevitable group who are opposed to adopting standards and the discipline that they imply.
So this is why I'm going to argue that just as software and hardware should be tested, standards and working practices should first be rolled out on a test-bed. Which then leaves the excellent question of what does a test-bed for a standard or a working practice look like? Essentially, you should test out the standard or guideline on one or more projects or initiatives – a pilot project for the standard, no less.
The ideal such 'test-bed', from what I've seen, is a project that is run by an outsider who is not necessarily sold on the idea of the new guideline, but who at least is not outright hostile. The idealized candidate is the one who manages to ask a difficult question at every meeting – the kind you always dread, even while knowing that they're raising good points. But that's the ideal candidate.
This might seem like a lot of effort, and the old rule for testing does still hold true – you have to balance the amount of testing you do against the potential consequences of problems and how easy it will be to remedy things. In a situation where the group is generally bought into the idea of the standard and it can be amended quickly, no pilot project may be necessary.
In a situation where there is a politicized environment, there is opposition to the rollout of the standard(s) or where the standard forms part of a contract (such as one outsourcing contract that I've seen) – then a pilot with neutral participants is well advised.
Ultimately, it's not possible to foresee everything every time, whether designing a system, defining a process or defining a style guide. We test the first two cases – we should test the third one as well.