If you don’t know what the KCF is then here is probably not the best place to start and I would encourage you to go and read the overview and then return in due course.
In this article I’m going to focus on the second of the 4 tasks in the Conceptual part of the KCF, namely the Document Type Analysis.
When we developed the BCS in the first task in the KCF we ended up with a functionally-based classification taxonomy.
The objective of this task is to draw up a list of all the different types of document created or managed by the organisation and then to categorise them according to the role they play and the function they perform within the organisation. In other words, assign them to the BCS.
3 Buckets
This is not a democracy, and not all documents are created equal. Some documents are clearly more important to the organisation than others. Now there are many ways in which you can carve up the document space. One way, which I have been pushing for many years is shown below.
The idea here is that every document can be mapped to it’s own cube. We can set up processing rules which apply for each cube.
However, the model I currently favour is even simpler than that and I call it the 3 bucket method.
In the 3 bucket method, all documents are classified into 3 broad classes (the buckets). The reason that there are 3 classes is that there are 3 distinct variants on the Document Lifecycle and we need to design our SharePoint system such that it can support all 3 modes.
Managed Documents
Managed Documents (a(sometimes)ka Controlled Documents) are those set of documents that once they come into being, never (or rarely anyway) get disposed. These documents govern and guide how the organisation is supposed to function. For most businesses these will include Policies and Procedures and possibly many other subordinate and supporting document types.
In fact it is very often useful to arrange these types of documents into a hierarchy as there tends to be a distinct relationship between document types which sit at the different levels, which many organisations refer to as tiers.
Often Polices are placed at the top of the pyramid but I would argue that there are a few key managed documents that sit above Policies. The diagram below shows how Managed Documents can be arranged into 5 tiers.
Tier 0
Tier 0 documents are those that set the tone for the whole enterprise. There are likely to only be a few Tier 0 documents such as the company’s mission statement, charter and values statement as examples.
Tier 1
Tier 1 documents are typically those that set the way in which the organisation conducts business (Policies) or hopes to conduct business (Strategies). These documents govern how work is to be done, usually within functional areas. So there will be an HR Policy and an IT Strategy for example.
Tier 2
Tier 2 documents are those which govern specific processes. Usually this amounts to a set of procedure documents. So if the HR Policy determines leave entitlements an HR Leave Request Procedure document will explain how employees go about applying for leave of different types (annual, special, maternity etc.).
Unlike Tier 1 documents, which should be technology and product agnostic, Tier 2 documents are entitled to name specific applications and may include screenshots and the like.
Tier 3
Some processes will be complex and may require employees to make judgement calls. Tier 3 documents can provide detailed and specific instructions or some best practice guidance in order to help employees achieve a desired outcome.
Tier 4
Tier 4 documents are enablers. These are typically the forms that get filled out, document templates, flow charts and checklists. Note here that I am talking about blank forms, templates and checklists from which documents instances are created. An instance of a Leave Application Form is an example of a Transaction Document (described later) but the blank form itself is a Managed Document.
The reason that Managed Documents site well in a hierarchy is that they are all basically designed to govern business processes at different levels, moving from the abstract and generic to the detailed and specific as you descend the tiers.
So the company mission statement and values statement are the foundation for Policies. The Policies are propped up by documents such as Procedures which are themselves supported by Guides and Instructions. The actual mechanics of process execution often require instances of Tier 4 documents but the process itself is governed by all that sits above it.
Obviously, care needs to be taken to ensure that the governance is consistent throughout the tiers. So a Procedure shouldn’t state something that is in conflict with a Policy (for that road is paved with expensive legal bills) unless there is a clear and explicitly stated reason for doing so. In which case, the specific will tend to out-trump the generic.
But this article is not about the content of these Managed Document but rather seeks to recognise that they exits, that there are relationships between them and that they all follow a somewhat special lifecycle.
The document lifecycle for Managed Documents is special because once a Managed Document is created it tends to be created forever and is only subject to archive and disposal in the relatively rare event that the organisation chooses to embark on a radial restructuring of all its Managed Documents – which does happen occasionally (maybe every 10-20 years). Normally however, once an organisation has created say an HR Policy and Leave Application Procedure it will always have such documents.
Now, that does not mean that Managed Documents are static. In fact they should be the subject of regular periodic review to ensure that they continue to serve the business well. If they are outdated or obsolete then their value is greatly undermined.
So in summary:
- Managed Documents are critical to support and govern the ongoing and future operations of the business.
- They work on many levels from the high and conceptual to the detailed and specific.
- They are single instance products (you don’t need 2 HR policies)
- There are hierarchical relationships between managed documents which can be represented in a tier model.
- Managed Documents have a somewhat special lifecycle in that once created they never get disposed (barring a radical re-write exercise).
- Managed Documents should be subject to regular period review to ensure that they continue to meet the needs of the business, employees and external entities with which the organisation interacts.
Transient Documents
Transient Documents can be identified as those which have a short lived operational value to the organisation but have no long term strategic value and are not subject to any form of regulatory compliance.
When the useful life of a Transient Documents has expired it should be sent straight to disposal (do not pass through Go (the archive) and do not collect $200). Some classic examples of Transient Documents are:
- Calling Notice (for some event or meeting)
- Weekly canteen menu
- Meeting Agendas
- Staff Notices (they’re digging up the car park on Friday so all staff are to use the overflow)
- Calendar Events
- Management Notices (please welcome James our new IT guru)
- Hi Everyone Messages (Hi, my name is James and I’m your new IT guru, ask me about SharePoint, The Matrix and Star Trek)
- Congratulation Message
- Probably upward of 95% of all email traffic
- Social, media items such as posts on Facebook or Instagram.
- Probably upward of 99% of all txt messages and tweets (unless you are Donald Trump of course).
The problem is that there is an awful lot of Transient Documents and most of them have a very short lifespan and so end up as being noise very quickly. I have a customer who is just about to migrate from SP2010 to SP2016 and the migration was all set to include 4000+ obsolete items in their portal announcements list.
So when I see SharePoint deployments with no governance around disposal and with users complaining that they can never find what they want because there is so much crap on the system then I try and gently explain the cause and effect relationship between the two.
So in summary:
- Transient Documents have a short life-span in terms of their operational value to the business.
- There is an awful lot of Transient Documents out there (don’t believe me, look at your email in-box).
- Transient Documents are generally not subject to a management or governance process (think of Trump tweets), although corporate announcements might be subject to a release approval process.
- SharePoint is awash with Transient Documents. Think of all your announcements and calendar lists, discussion boards, bright ideas lists, issues lists and news feeds.
- If Transient Documents are not subject to a regime of rigorous disposal they become noise very quickly.
- Noise adds friction to people finding what they need to do their job.
Transaction Documents
Transaction Documents are instance documents such that they are often generated to initiate, or as part of, the execution of a business process.
For example, in order to procure something (new paper clips) a staff member has to raise a Purchase Requisition Order (PRO) as part of the Purchase Requisition Process. Note that PRO might be a Word Document, InfoPath Form, SharePoint List Item or even (heaven forbid) a piece of paper. The template for the PRO is a Managed Document but an instance of the PRO, filled in and used as part of the procurement process, is a Transaction Document.
So what makes a PRO a Transaction Document and not a Transient Document? Well, it’s usually because the PRO will be linked to a Purchase Order (PO) and ultimately an Invoice and so as part of a financial process there is a need to ensure that such transactions are recorded so that the business can be audited if need be.
So, even though the operation lifespan of a Transaction Document might be quite short it still needs to be retained for reasons of regulatory compliance or because the product has some long term value to the business.
In Australia, all business are required by law to retain all financial transaction records for a minimum of 7 years (regulatory compliance). Though, if you can believe it, I’ve been inside companies that have held on to Transaction Documents that date back to the ’80s. Usually, these end up as paper records that are shipped off to some off-site storage provider such as Iron Mountain or Recall and are never seen again. And, because no body actually knows what is held in these paper mountains then everyone is too scared to initiate disposal of anything.
So the paper mountains get larger and larger and cost the business more and more money. I’ve worked with businesses where their monthly paper mountain bill is well over $20,000! That’s an expensive overhead, and these guys have you all stitched up because they charge a fortune for you to pull anything back from their storage facility, so most organisations just go ostrich-like and pretend this issue doesn’t exist.
I’m in the wrong gig here, if I want to make money I should just set up my own paper mountain business.
Not all documents that are archived need to be retained for reasons of legal compliance. Some documents, especially photographs and these days video may be retained as part of the organisation’s history. Such documents may stay in the archive forever and never get sent for disposal.
Like Transient Documents, Transaction Documents soon become noise (and so add to the friction of search and retrieval) but unlike Transient Documents they cannot go straight to disposal, instead they need to sit an archive for a while (often years and sometime forever).
So in summary:
- Transaction Documents are generated through the execution of business processes.
- They are multiple instance documents often generated using a template or form.
- They have an operational lifespan that can be measured in weeks or months at most.
- Transaction Documents cannot be disposed of immediately and need to be retained in an archive, sometimes for many years.
- If proactive steps are not taken to govern these systems so that the Transaction Documents transition to an archive then they soon end up as noise.
- Noise adds friction to people finding what they need to do their job – did I say that already?
- Printing out Transaction Documents, throwing them in a cardboard box and arranging dispatch to an off-site paper mountain is not archiving.
- As many organisations fail to adequately record what Transaction Documents gets set to off-site storage, the paper mountains grow and grow because everyone is too scared to issue disposal instructions for fear of litigation.
- Paper mountains cost business a lot of money and without proper governance it costs them more and more each year.
- I’m in the wrong business, I should go and start a paper mountain in a lock-up somewhere.
Asymmetric Flows
Nearly every SharePoint installation I see has an asymmetric flow. The emphasis is on how to get stuff into SharePoint and no one thinks about how to get obsolete information out of SharePoint. And when, as any plumber will tell you, you have an in-flow and no out-flow then the crap just keeps piling up until one day you realise that you are living in a pig sty!
With Transient Documents the noise issue is resolved quite easily (at least conceptually its easy), you’ve just got to train users to delete stuff. However, with Transaction Documents it’s more complicated because they can’t just delete stuff, they have to send it to an archive and that sounds complicated and so sometimes it’s not done. And when it is done, it’s not done properly (just sending creates to Iron Mountain does not count).
Running an archive and creating an out-flow does not have to be complicated, it just needs to be thought through and the technology properly aligned with what needs to be achieved. And of course the system needs to be properly governed. More on this when we come back to the SharePoint implementation side of the KCF.
Filling the Buckets
Actually this step is remarkably ease. We’ve already conducted the BCS and that has revealed most of what we need.
On your journey to carving up the what the business does in terms of Functions, Activities and Transactions the types of document used in the business will naturally present themselves to you – like salmon leaping into your boat!
So you’ll end up with 3 groups of document each subject to a different document lifecycle and each type of document can be positioned somewhere in the BCS. It’s not unusual to end up with 80 document types or more. In due course these will end up as SharePoint Content Types but let’s not get ahead of ourselves.
If you find the organisation uses a document that can’t be mapped to the BCS that means that either that type of document is no longer needed or, more likely the BCS is not right and needs updating.
So the BCS and the Document Type Analysis are interlinked and during workshops and interviews with process stakeholders you’ll probably elicit both of these elements of the KCF at the same time.
Kaboodle Software 