Lifting the Lid on ‘Shadow Processes’


One of the significant advantages to using observation as an elicitation technique when examining processes is that we get to see how the work is really done.  Rather than relying on what has been documented, we are able to see the intricate nuances and challenges with our own eyes.   As well as seeing the ‘formal’ parts of the process, it is likely that we will see other ‘informal’ parts of the process that have never been documented.  And in doing so, we might uncover and discover some tasks and activities that have been going on below-the-radar for years, some of which might be introducing significant amounts of risk or waste into our processes.


So often, processes evolve over time rather than being consciously designed.  We may be observing a process that has never been formally analyzed or documented—and therefore the people operating it have (quite understandably) developed it and built it themselves.  Often this evolution happens against a backdrop of frequent IT system changes.  If end-users are not properly consulted, their processes are impacted, and they have to adapt them.


This leads to a dangerous pattern.  Processes are adapted in crisis mode—users and stakeholders adapt processes urgently because they have to. They have a queue of customers at the front desk, and the booking system isn’t working properly.  There isn’t necessarily time to think things through, and the focus is on creating a ‘quick fix’ or a sticking plaster (Band-Aid).  This can lead to the introduction of ‘shadow tasks’ or entire ‘shadow processes’ – activities that happen outside of the formal process, many of which are never documented.  These shadow processes often rely on custom built spreadsheets and databases which are stored on a user’s local machine—often with complex macros that were written by somebody who has long since left the organization!  I have nothing against spreadsheets when used correctly – but in some situations, left unacknowledged, these shadow processes can introduce risk.  It is tantalizingly tempting to lay the blame on the users for creating shadowy and potentially inefficient processes – but to do so would be grossly unfair.


The reality is, whenever we come across a shadow process, task or application it is fulfilling a need that the formal process or system didn’t address.   It has often been implemented in frustration, when the team involved realized that their processes and systems were inflexible and were unable to meet the customer’s needs.   These are failings not of the teams themselves, but the projects and initiatives that implemented the systems and processes in the first place.   It has often been the case that process or system changes have been implemented without sufficient thought being put into how end users would use them.


It is time, as a community of process and change professionals, that we shine the spotlight on these shadows and bring them within our remit.  We should embrace the fact that these ‘informal’ tasks and activities may well have been brought about by previous process or system implementation failures and ask questions like:


  1. What core need is this task, process or activity meeting?
  2. How could that need be better incorporated into the formal process?
  3. How do we deal with the historic data that has been recorded in the ‘informal’ way, and how do we reduce risk/increase security?
  4. How do we prevent putting our users in a situation where they have to implement local workarounds and ‘sticking plasters’ in future?


These aren’t easy conversations to have, but they bring benefit.  Making visible what was once ‘below the radar’ will help ensure that our processes are efficient not just on paper, but in reality too.


In summary:  if we find shadow processes or tasks, it is beneficial to bring them out into the daylight.  We should work collaboratively with our stakeholders to integrate the core underlying needs into the process, and work together to ensure that the underlying process is efficient and effective.