Accessing Lessons Learned

[MSmith]

I came across an old article, originally published in the Maple Leaf and republished in the CMJ titled "Op SAGE: Knowledge in Action" (attached). It's fictional, but meant to illustrate what knowledge management could be in the CAF. It's a bit ridiculous but I think the point is valid. Although Monitor Mass and ACIMS aren't perfect, they do a decent job of the admin side of things. Where we're totally lacking is the ability to access lessons learned.

Until today I had no idea that you could access a LL database on the DWAN, no one at CTC or my unit had ever talked about it either. Even the OPME's name drop its existence, but give no link or direction on how to get there. It's a gold mine of info from past ops and what they learned, but no one seems to realize its there. I wouldn't say it's user friendly but it gets the job done. For those with DWAN access, here is a link: http://kms.mil.ca/kms/MenuSectionPage.aspx?MenuSectionKey=Lesson

Why are we not using this for AARs from each unit after every major exercise? Why do I need to request an account from my "Command/Formation/Unit Lessons Learned Officer" (I will be asking tomorrow, but im almost certain my unit does not have one of those) instead of just being able to access drafts/contribute by virtue of being logged into DWAN? Why is this not talked about more?

I hope I'm ignorant to the bigger picture and this is actually happening at levels I'm not privy to, but after spending a few hours surfing that site it really doesn't seem that way. It seems that the air force manages to submit their lessons learned after each iteration of Maple Resolve, but the site is totally devoid of any army contributions from recent serials.

Finally, I think the best feature of the site is the ability to access lessons learned from previous operations, similar to what the Op SAGE article was getting at. However, only unclassified "Lessons Synopsis Reports" are available. These give an overview of the titles of key lessons learned, but to see the substance of what was actually learned beyond a single sentence (ie, the useful part), you need to have access to secret, not enclosed references to the lessons synopsis report.

Here is a concrete example for you, from "Tanks Employment in Southern Afghanistan, 11 May 2007" (again, unclassified):

KEY LESSONS:

a. Doctrine:
iv. Breaching ops necessitate an angled approach to croplands - see para 18 ref B.
v. Improved Tank minefield extraction TTP have proven effective - see para 21 & Anx B ref B.

I would love to read about what both of those mean, but I can't, because the refs are unenclosed and secret.

Before I submit a UCR, am I totally missing something?

 

[Weinie]

I came across an old article, originally published in the Maple Leaf and republished in the CMJ titled "Op SAGE: Knowledge in Action" (attached). It's fictional, but meant to illustrate what knowledge management could be in the CAF. It's a bit ridiculous but I think the point is valid. Although Monitor Mass and ACIMS aren't perfect, they do a decent job of the admin side of things. Where we're totally lacking is the ability to access lessons learned.

Until today I had no idea that you could access a LL database on the DWAN, no one at CTC or my unit had ever talked about it either. Even the OPME's name drop its existence, but give no link or direction on how to get there. It's a gold mine of info from past ops and what they learned, but no one seems to realize its there. I wouldn't say it's user friendly but it gets the job done. For those with DWAN access, here is a link: http://kms.mil.ca/kms/MenuSectionPage.aspx?MenuSectionKey=Lesson

Why are we not using this for AARs from each unit after every major exercise? Why do I need to request an account from my "Command/Formation/Unit Lessons Learned Officer" (I will be asking tomorrow, but im almost certain my unit does not have one of those) instead of just being able to access drafts/contribute by virtue of being logged into DWAN? Why is this not talked about more?

I hope I'm ignorant to the bigger picture and this is actually happening at levels I'm not privy to, but after spending a few hours surfing that site it really doesn't seem that way. It seems that the air force manages to submit their lessons learned after each iteration of Maple Resolve, but the site is totally devoid of any army contributions from recent serials.

Finally, I think the best feature of the site is the ability to access lessons learned from previous operations, similar to what the Op SAGE article was getting at. However, only unclassified "Lessons Synopsis Reports" are available. These give an overview of the titles of key lessons learned, but to see the substance of what was actually learned beyond a single sentence (ie, the useful part), you need to have access to secret, not enclosed references to the lessons synopsis report.

Here is a concrete example for you, from "Tanks Employment in Southern Afghanistan, 11 May 2007" (again, unclassified):

KEY LESSONS:

a. Doctrine:
iv. Breaching ops necessitate an angled approach to croplands - see para 18 ref B.
v. Improved Tank minefield extraction TTP have proven effective - see para 21 & Anx B ref B.

I would love to read about what both of those mean, but I can't, because the refs are unenclosed and secret.

Before I submit a UCR, am I totally missing something?

Although it is called a LL database, that description would be generous at best. It is a compendium of observations, and recommendations. Until we actually do something to change, no lesson has been learned.

 

[MSmith]

Although it is called a LL database, that description would be generous at best. It is a compendium of observations, and recommendations. Until we actually do something to change, no lesson has been learned.

Well... I completely agree with you, but hey - at least having them be accessible would be a start. The other question is, how would these lessons be "learned?" I don't expect PAMs to change with every lesson a sub-sub-unit pulls from an op or ex, but it would be nice to be able to do what that Op SAGE article talks about and read about previous experiences if you are about to embark on a similar mission.

From the two lessons in my previous post, for example:

Should taking an angled approach to croplands be a "lesson learned" by writing it into a PAM? Probably not. I would think that would be cumbersome and doctrine would quickly balloon. However, I would like to be able to read about it prior to going into COIN ops with tanks in the future in an easily accessible place. Maybe in a separate COIN ops pam... but I think it more efficient, if you are a tank sqn OC about to deploy, to be able to quickly find and read about previous similar experience and how problems were solved. Kind of like a war diary without the narrative, just the useful bits for preparation.

vs.

"Improved Tank minefield extraction TTP have proven effective" sounds like something that should replace whatever the old TTP was in its respective PAM.

I don't buy into the notion that these observations and reccomendations are useless. But I do get what you're saying, I think.

 

[Weinie]

Well... I completely agree with you, but hey - at least having them be accessible would be a start. The other question is, how would these lessons be "learned?" I don't expect PAMs to change with every lesson a sub-sub-unit pulls from an op or ex, but it would be nice to be able to do what that Op SAGE article talks about and read about previous experiences if you are about to embark on a similar mission.

From the two lessons in my previous post, for example:

Should taking an angled approach to croplands be a "lesson learned" by writing it into a PAM? Probably not. I would think that would be cumbersome and doctrine would quickly balloon. However, I would like to be able to read about it prior to going into COIN ops with tanks in the future in an easily accessible place. Maybe in a separate COIN ops pam... but I think it more efficient, if you are a tank sqn OC about to deploy, to be able to quickly find and read about previous similar experience and how problems were solved. Kind of like a war diary without the narrative, just the useful bits for preparation.

vs.

"Improved Tank minefield extraction TTP have proven effective" sounds like something that should replace whatever the old TTP was in its respective PAM.

I don't buy into the notion that these observations and reccomendations are useless. But I do get what you're saying, I think.

Please don't think I believe these observations and recommendations are useless. On the contrary, the fact that we have not incorporated them into PAMS, TTP's, SOP's, doctrine etc borders on the negligent. But it has been my observation that we (the CAF) are so time challenged that no one has any capacity to actually build these into the system. (With one exception, CANSOFOM is extraordinary at learning from Ops)

 

[daftandbarmy]

Well... I completely agree with you, but hey - at least having them be accessible would be a start. The other question is, how would these lessons be "learned?" I don't expect PAMs to change with every lesson a sub-sub-unit pulls from an op or ex, but it would be nice to be able to do what that Op SAGE article talks about and read about previous experiences if you are about to embark on a similar mission.

From the two lessons in my previous post, for example:

Should taking an angled approach to croplands be a "lesson learned" by writing it into a PAM? Probably not. I would think that would be cumbersome and doctrine would quickly balloon. However, I would like to be able to read about it prior to going into COIN ops with tanks in the future in an easily accessible place. Maybe in a separate COIN ops pam... but I think it more efficient, if you are a tank sqn OC about to deploy, to be able to quickly find and read about previous similar experience and how problems were solved. Kind of like a war diary without the narrative, just the useful bits for preparation.

vs.

"Improved Tank minefield extraction TTP have proven effective" sounds like something that should replace whatever the old TTP was in its respective PAM.

I don't buy into the notion that these observations and reccomendations are useless. But I do get what you're saying, I think.

Why not start something up on Army.ca? (He suggested cautiously knowing full well that he hasn't checked the site to see if there's already something on here for that purpose ).

 

[Navy_Pete]

That KMS system is pretty user unfriendly, and really more of a dumpsite than functional database. There is no kind of review/editing, so basically you just have to read through a whack of stuff and try and figure out what is useful, what is a bad idea, and what is the result of someone learning a lesson that came about because they are fundamentally a muppet. Also, things get fired into the void, so nothing worse then reviewing it prior to a mission and finding 3 previous LLs from the same mission all with the same LL (which means that no one reads it to fix the problem, as well as people not reading the previous LL, so why should I bother creating our own at end of mission during the hot washup to add to the volume of detritus).

A much better option would be something like a wiki format that could be actively compiled, reviewed an updated. There are some unclass ones over on the procurement side and used to be one in the training system that were pretty good. There may be better formats (maybe like a one note?), but needs to be something really simple to add to and find stuff or it's a waste of time. A lot of LLs are context dependent, but if someone figured out a useful way to skin a cat, or picked up a useful tidbit. Even if less stuff gets into it, would probably be more useful in real terms than a massive, unweildy database of comprehensive, contextual LLs that no one has time to read.

From experience, to be effective that needs some moderation to set up a logical framework ahead of time to avoid it becoming a dumping ground. It can't be too rigid though or you need to dig through 10 layers of stupid to get to the useful bit, at which point it becomes non-useful. Alternately, someone with experience in the subject needs to actually read thought the KMS and kick out some kind of summary to the related community (which sounds like either the worlds biggest punishment posting or an absolutely awful secondary).

Knowledge management is a massive challenge though for big organizations, so don't think the CAF is really lagging. At least we recognize it's an issue and try to do something, and most units have a turnover SOP that is both easy to follow and useful. We're way ahead of most other GoC departments on that front anyway.

 

[MSmith]

Why not start something up on Army.ca? (He suggested cautiously knowing full well that he hasn't checked the site to see if there's already something on here for that purpose ).

Maybe for some informal discussion but there are people who are paid to do this for the army (not nearly enough of them), and what is the point of having a professional army if we aren't learning anything along the way? There are directives, orders, DAODs, etc that talk up the importance of the LL program but nothing actually useful

 

[FJAG]

...
Knowledge management is a massive challenge though for big organizations, so don't think the CAF is really lagging. At least we recognize it's an issue and try to do something, and most units have a turnover SOP that is both easy to follow and useful. We're way ahead of most other GoC departments on that front anyway.

KM is even a challenge for a small organization.

About a decade and a half ago I was the project manager for the JAG's Comprehensive Information Management Project which had four deliverables: a firm-wide Protected B computer network architecture; a firm-wide records management system; a firm-wide case management system and a firm-wide knowledge management system. The first two we delivered quite handily. The third bogged down due to the acquisition process for an off-the shelf, bilingual legal case management system. The fourth bogged down because of internal lethargy.

At it's most basic, the key to any valid knowledge management system has three essential parts: a system/process of recovering knowledge from the users; a process of collating and validating the knowledge; and a process of publishing/distributing the knowledge to all subsequent users as useable doctrine.

Within the CIMP we had the money to set up the process for capturing knowledge and had funding for three paralegals (one each for op law, adm law and mil justice) to collate the knowledge but fell down heavily in obtaining commitments from senior directors to validate the knowledge captured and to convert it into appropriate doctrine. The reasons why this failed were many and varied but essentially boiled down to the fact that the senior and middle leaders considered themselves and their subordinates far too busy doing day-to-day legal services to be able to devote their own time or that of their subordinates to performing the validation required and to develop the resultant doctrine to publish. To some extent there were even objections to having their subordinates spend time in capturing the knowledge required.

Proper knowledge management needs supervision and commitment from the grown-ups in any organization if it is to be of any lasting value. Without that it will always limp along as a shadow of what it could be.

 

[MSmith]

That KMS system is pretty user unfriendly, and really more of a dumpsite than functional database. There is no kind of review/editing, so basically you just have to read through a whack of stuff and try and figure out what is useful, what is a bad idea, and what is the result of someone learning a lesson that came about because they are fundamentally a muppet. Also, things get fired into the void, so nothing worse then reviewing it prior to a mission and finding 3 previous LLs from the same mission all with the same LL (which means that no one reads it to fix the problem, as well as people not reading the previous LL, so why should I bother creating our own at end of mission during the hot washup to add to the volume of detritus).

A much better option would be something like a wiki format that could be actively compiled, reviewed an updated. There are some unclass ones over on the procurement side and used to be one in the training system that were pretty good. There may be better formats (maybe like a one note?), but needs to be something really simple to add to and find stuff or it's a waste of time. A lot of LLs are context dependent, but if someone figured out a useful way to skin a cat, or picked up a useful tidbit. Even if less stuff gets into it, would probably be more useful in real terms than a massive, unweildy database of comprehensive, contextual LLs that no one has time to read.

From experience, to be effective that needs some moderation to set up a logical framework ahead of time to avoid it becoming a dumping ground. It can't be too rigid though or you need to dig through 10 layers of stupid to get to the useful bit, at which point it becomes non-useful. Alternately, someone with experience in the subject needs to actually read thought the KMS and kick out some kind of summary to the related community (which sounds like either the worlds biggest punishment posting or an absolutely awful secondary).

Knowledge management is a massive challenge though for big organizations, so don't think the CAF is really lagging. At least we recognize it's an issue and try to do something, and most units have a turnover SOP that is both easy to follow and useful. We're way ahead of most other GoC departments on that front anyway.

I think that you hit the nail on the head. Admin and routine tasks take up enough of most people's time at work that sifting through verbose documents is the last thing in their priorities. A wiki would certainly be a big step forward, but like you said there needs to be some level of review and filtering.

The Army Lessons Learned Centre ALLC does publish bulletins and summaries, but I didn't find them particularily useful to read as they mostly focused on very high level or obvious lessons that came of of a mission. The RCAC and RCIC LL bulletins from Afghanistan, while fairly long, didn't really give me the impression that we came away having learned anything significant at the tactical level on how to fight. There are certainly more informative threads on this website from personal experiences. I think the other part of the problem there is that the ALLC has a tiny staff, so it would be very difficult for them to respond to requests for info while also actually analyzing reports and creating summaries.

Which brings me to my next point. I realized yesterday that the KMS seems to be a bit obsolete now, with the army focusing on its own lessons learned organization which occasionally contributes to the KMS. There is a third layer as well, where NATO has a LL portal. However, the ALLC has some documents on its ACIMS page from a NATO LL/KM conference where the big takeaway was most LL are already shared between allied militaries via bilarteral agreements, and the NATO LL portal is just another dumping ground similar to what the KMS is to the ALLC. What a mess.

For those interested though, there are some fascinating docs available on the ALLC acims page if you do some digging. I dont have DWAN access at the moment so I cant post a link, and the ALLC acims page is criminally hard to find. Its buried within the CADTC page. Anyways, for those who remember the recent kerfuffle about China sending military delegations to observe Canadian winter warfare trg, there is a great visits report from the Canadian delegation that visited a Chinese Bn conducting their own winter warfare ex. Seriously fascinating stuff to read about the differences in military culture, conduct and doctrine, too bad it's buried so far in ACIMS that it will never see the light of day at any PD sessions. As well, there are excellent LL presentations from the French experience in Mali there as well. Too bad we are not producing something similar.

Please don't think I believe these observations and recommendations are useless. On the contrary, the fact that we have not incorporated them into PAMS, TTP's, SOP's, doctrine etc borders on the negligent. But it has been my observation that we (the CAF) are so time challenged that no one has any capacity to actually build these into the system. (With one exception, CANSOFOM is extraordinary at learning from Ops)

Wasn't trying to say that you thought they were useless, but the LL program does say something similar about how lessons aren't truely "learned" until some kind of change has been made somewhere. All im trying to say is that this information still is valuable even if no PAMS/TTPs have been changed because of it (even if they should be)

Can you shed some light on what you mean by CANSOFCOM being extraordinary at learning from ops?

 

[Weinie]

I think that you hit the nail on the head. Admin and routine tasks take up enough of most people's time at work that sifting through verbose documents is the last thing in their priorities. A wiki would certainly be a big step forward, but like you said there needs to be some level of review and filtering.

The Army Lessons Learned Centre ALLC does publish bulletins and summaries, but I didn't find them particularily useful to read as they mostly focused on very high level or obvious lessons that came of of a mission. The RCAC and RCIC LL bulletins from Afghanistan, while fairly long, didn't really give me the impression that we came away having learned anything significant at the tactical level on how to fight. There are certainly more informative threads on this website from personal experiences. I think the other part of the problem there is that the ALLC has a tiny staff, so it would be very difficult for them to respond to requests for info while also actually analyzing reports and creating summaries.

Which brings me to my next point. I realized yesterday that the KMS seems to be a bit obsolete now, with the army focusing on its own lessons learned organization which occasionally contributes to the KMS. There is a third layer as well, where NATO has a LL portal. However, the ALLC has some documents on its ACIMS page from a NATO LL/KM conference where the big takeaway was most LL are already shared between allied militaries via bilarteral agreements, and the NATO LL portal is just another dumping ground similar to what the KMS is to the ALLC. What a mess.

For those interested though, there are some fascinating docs available on the ALLC acims page if you do some digging. I dont have DWAN access at the moment so I cant post a link, and the ALLC acims page is criminally hard to find. Its buried within the CADTC page. Anyways, for those who remember the recent kerfuffle about China sending military delegations to observe Canadian winter warfare trg, there is a great visits report from the Canadian delegation that visited a Chinese Bn conducting their own winter warfare ex. Seriously fascinating stuff to read about the differences in military culture, conduct and doctrine, too bad it's buried so far in ACIMS that it will never see the light of day at any PD sessions. As well, there are excellent LL presentations from the French experience in Mali there as well. Too bad we are not producing something similar.

Wasn't trying to say that you thought they were useless, but the LL program does say something similar about how lessons aren't truely "learned" until some kind of change has been made somewhere. All im trying to say is that this information still is valuable even if no PAMS/TTPs have been changed because of it (even if they should be)

Can you shed some light on what you mean by CANSOFCOM being extraordinary at learning from ops?

Not deliberately trying to be obtuse here, but the nature of most CANSOFCOM Ops entails a degree of risk that is not resident in other orgs, accordingly, they pay a very high degree of attention to where things "could be done better."

 

[MSmith]

Not deliberately trying to be obtuse here, but the nature of most CANSOFCOM Ops entails a degree of risk that is not resident in other orgs, accordingly, they pay a very high degree of attention to where things "could be done better."

Ah. Well yes, and I've never worked outside the green army but I imagine people over there are generally more motivated and methodical as well... AARs are not sexy

 

[daftandbarmy]

One thing we tend to forget/ discount is that all of our existing tactics etc, as contained in the manuals and training we use and deliver during courses and other 'road to readiness' type preparation, are already pretty good.

These materials already tend to reflect the lessons learned, in the hardest ways imaginable, from many previous campaigns. Mainly, the things we do are really simple. It's just that doing these things well, under the most trying conditions, is the hard part. (There might be a von Clausewitz quote in there somewhere ).

I can't think of too many operational situations that I've been in where I wondered 'Gee, why wasn't I trained/prepared/ warned about this before?'. When this did occur, occasionally, I was able to get a few smart people together and figure it out, or somebody connected with me to give me the information I needed.

We can't teach everyone everything they need to know. Building great teams with good leadership and a vibrant improvement culture, with a strong learning posture, is probably the best way to ensure we're 'Utrinque Paratus'.

 

[Red_Five]

If we speak of "learning" as making adaptations to the operational environment (as opposed to learning on a course etc), we do fine as individuals and units. Its harder at the institutional level. There might be some cultural reasons for that. We are fairly tribal and place great stock on our own experience.

Putting that aside, all of the doctrine we have is based on "lessons learned." That doesn't mean that all lessons become doctrine. I was posted to the ALLC for three years (05 to 08), including a deployment as embedded Lessons Learned Liaison Officer from Feb to Aug 2006. I wrote many reports from theatre - based primarily on interviews with TF members at all ranks/appointments. They went on a wide distribution, hitting many decision-makers in the CA. Those reports were valid in their context, but they would have to go through SME review in Canada (LFDTS was the lead). That review is critical. It also takes time and effort.

I was an Admin for the LLKW (predecessor of KMS). It was challenging having a high-side with "the good stuff" and a more accessible low-side cleansed of useful details. So I guess I'm part of the problem?

 

[Navy_Pete]

One issue we have with the ages of the ships is that we are getting first time failure modes for different things, or other weird problems. We used to have periodic engineering letters where we would share stuff like that with the fleet (even if we didn't have a solution). That eventually ended up on the KMS site as a repository, but normally just got sent to every other ship plus MEPM when it was done.

It turned into a bit of a bureaucratic bit of waste because people stopped using it for what is was intended, but it was actually pretty useful when once in a while when you would see someone else with the same weird problem, and used it a few times to ping off other people to figure things out, and to flag things to LCMMs. It was canceled a few years ago so aside from trying to meet other engineers for coffees, doesn't really happen anymore.

Know there is also a high side one that was useful for finding LL documents from other ships who did similar deployments in the same regions, which included things things like what services were available (or not) in a port, what to bring with you, and where you might be able to get specific supplies easily (like some of the POLs). Those tidbits could be pretty useful, and not really related to tactics or anything, but were more of a depository of info that should probably have been recorded other places. Usually the best part was the names of key pers so you could drop them an email to ask other questions.