(We’re not going to say “best”, because, frankly, that’s a bit more arrogant than we wish to be.)
One of our colleagues had a practice of typing his thoughts out as he was having them.
Literally.
He would document his thoughts in real-time, as he was having them.
Not the I can’t believe it’s not butter type, but the thought processes his brain went through as he did his actuarial work.
If he was trying to figure out why a model’s results seemed to slide off the rails in later time periods, he might look through the variables, the inputs and the outputs, searching for anomalies. While he did that, he would write, in a Word document, “Hm, I wonder if it’s the zinkenberger table? Let’s see… Nope, not that one. That one looks good. But, maybe it’s the binkenzerger table? Oh! It’s not going past age 80, so that’s why there’s 0 values there.”
Is this documentation?
Of a sort, yes.
Is it an “actuarial document”? Does it satisfy any Actuarial Standards of Practice?
No, and no.
But is it important? Is it valuable?
Yes, and, more importantly, yes.
Documentation like this, in real time, of what the thought process was behind model development, bug fixing, and assumption management can be an essential part of providing a high-quality actuarial work product.
Here’s why good actuarial model documentation matters.
We Can Agree Documentation is Not the Most Exciting Actuarial Task
Documentation often falls to the bottom of the priority list. It comes after model building, assumption-setting, validation, and deployment, because those are the actuarial tasks which have a visible value to them. They answer the question asked; they satisfy the regulatory requirements imposed. They demonstrate the actuary’s competence, skill, and knowledge. They give us actuaries opportunity to flex our brains on all those not-quite-as-smart-as-us IT folks who spend so much time updating business requirements and process documents.
Documentation? Documentation does none of that. It’s often viewed as an unproductive time-sink, a necessary evil that should be postponed as long as possible.
It might be a couple of weeks later. Maybe months until the actuary “gets around to it” and makes sure that he’s listed what he did (if he can remember) and, occasionally, why he did it.
However, when such postponement happens, the documentation that results runs the risk of being less complete, less comprehensive, and, frankly, less usable by the next person to pick up the model in the future.
Remember, there are multiple reasons for documentation. Only one of them is usually the stated reason. All of the other benefits (and there are benefits) of documentation are the secondary, indirect benefits which come from having good documentation on hand: model hand-off happens smoothly, results can be trusted, application of the model to new areas can happen quicker.
Most Actuarial Standards of Practices (ASOPs) have documentation requirements. Like ASOP 22, Statements of Opinion Based on Asset Adequacy Analysis by Actuaries for Life or Health Insurers.
The thing is, most of those requirements are about the end product – what methods were chosen for the final analysis, the assumptions that currently exist in the model, what the actuary’s final judgment is, and so on.
These kinds of documentation requirements don’t give enough background to future users of the model (next year’s actuarial students picking up the document to figure out what the final product they’re actually making is). There is a lot of value to knowing the scope of what was considered, and rejected, for many of these aspects.
Good Documentation is Not Just What, Not Just How, But Also Why
Good documentation is a more comprehensive picture of the model, the assumptions, and the thought processes behind the choices made.
This makes your documentation not just a record of steps taken to update a database or an actuarial model (we would argue that’s not documentation but instructions), but it’s also a living record of the mindset and decision-making processes that actuaries have employed throughout.
To that end, we’d like to provide you with a few tips on how to create a culture of good documentation. If you can implement good documentation practices, you’ll find many aspects of your actuarial work improved: from assumption management to model governance.
Model Documentation Tip #1 – Set a Documentation Standard
You should have a written document about your documentation practices: why they are important, when they should be in use, who should create them, who should review them, how much detail should be included, and so on.
A “meta-document”, if you will.
This will give you standards and practices you can employ, refine, and hold professionals accountable to. Don’t be afraid to let this document evolve over time. It should, as your business, and your models, advance.
Defining this standard needs to be a priority at the highest levels. Chief Actuaries, product or division Vice Presidents, and Model Stewards needs to lead the charge by setting high expectations for themselves and their teams. That way you don’t create friction between the competing priorities of new development and documentation.
Model Documentation Tip #2 – Document Everything
Everything?
Yes, everything. From what you’re supposed to do after a meeting to that 45 minutes you spent debugging that one add-in, make some notes when you’re finished. It could be just a couple of sentences or a few paragraphs, a couple of times a day.
Again, it doesn’t have to be super-detailed. But having a record will help you when people ask you later to reconstruct your thought processes. You won’t have to divert time and mental energy away from your current projects to think about the past.
More importantly, documentation gets the information out of your head and into the record, freeing up your mind space for more important tasks, like figuring out where to go for lunch.
Model Documentation Tip #3 – Create a Final Version
After you’ve built your model, created your automated process, or set your assumptions, go back through all your in-the-moment documentation, refresh your memory, and create a final, “pretty” version for your peers and supervisors.
This will give you the opportunity to pull together all the disparate nuggets of wisdom scattered throughout however many pages you’ve been making notes on through the development and validation processes.
Doing it right away will avoid that risk of knowledge erosion.
Model Documentation Tip #4 – Have a System for Documentation Naming and Storage
A good way to put this into practice is to have a separate folder within each directory that holds documentation. That way, new employees or peers unfamiliar with your specific model structure can still find, on a consistent basis, where the documentation lives.
Plus, if you have a consistent naming convention for your files, it will make organization easier. It might not seem like a big thing, but consistency is very helpful to others.
When another professional is searching for a file and expecting the name convention “PRODUCT-TIME FRAME-TASK-VERSION”, but you’ve named it “TIME FRAME-TASK-USER-PRODUCT”, it can lead to confusion, frustration, lost time, and lack of trust that you’ve actually got the right version.
Model Documentation Tip #5 – Seek Automation
Some model elements will be automatically documented with run logs. If you are familiar with cloud-based software or storage, you know that prior versions and change logs are automatically stored. Automated change logs are a clear advantage over manual documentation methods.
If you don’t have that, look for ways to automate on your own the model building, assumption updating, and decision-making processes.
That colleague we told you about before? He designed many systems over his actuarial career. One was a series of Excel files, and he was able to design the assumption tables in such a way that he could create documentation macros to copy and paste his final assumptions into a standard format.
That documentation provided a valuable resource for his audience, who could trust his work and the model results because they could validate what he’d done and how. The automation saved him hours of mindless tasks every quarter. But it didn’t come about by accident. It was only by intentional action, knowing that he’d want to have that on hand when the model was in production, that he developed the system that became valuable.
Conclusion – Good Documentation Is Good Actuarial Practice
So, there are your 5 tips for model documentation: have a standard, document everything, create a final version, make a system for naming and storage, and look for automation. With those tips, you’ll find your actuarial model documentation created more quickly, more comprehensively, and with more value for the next actuary using your model.