Introduction
Our series on Software principles for actuaries wraps up with considerations of software development practices that actuaries should probably incorporate into their processes.
We talked about the different dimensions of how software is defined (here, here, and here), and how to apply those dimensions to actuarial work product. (Also, here, here, and here.)
This article covers some practical steps that good software developers apply. Here’s the TL; DR:
Different environments – separating what’s the official version from a testing version will force additional review and approvals, as well as giving confidence that nothing’s broken in the process.
Develop for the actual use case – most times, people only know what they already know, not what’s available, which limits their ideas of what the solution could be. Finding out what they really want to do may lead to a different solution than they envisioned.
Modularity and reuse – break down components into reusable segments to avoid multiplying the workload.
Peer review – get it early and often so it can impact future development, rather than when it’s too late to do anything about it.
Testing – do it! Do it better, too.
Documentation – do it! Do it more frequently, too.
1. Different environments
Raise a hand if you’ve ever been in this situation before:
Q: What’s the official version of that file?
A: It’s the one labeled “final V2 but without the Hinkerberger class”.
Q: I can’t find it! All I see is the ‘v1 and v4’.
A: Oh, that’s right. It’s saved under the quarterly reports directory, under the name of the guy who used to work here a decade ago.
This happens all the time. Poor discipline around keeping materials organized allows for confusion. Not to mention lost time when people are searching for the “official” version of something, whether that is a spreadsheet, a pricing model, or a set of documentation.
Software developers have been able to reduce this impact by creating different “environments” where versions of their work resides. Each environment has a specific purpose, and keeping those environments separate ensures appropriate segregation of work done in each location.
Generally, those are Development, Testing, and Production environments.
Development will be used for initial evaluations. This may also be know as “sandboxing” (i.e. playing in the sandbox) some new feature, or checking to see if you really can make something run faster. It’s not the official version, so you can work on it without screwing anything up.
Testing is the next level up. Once something’s working in Development, it can be reviewed and approved to move to Test. There, a standard set of tests can be applied to the new code to ensure everything is basically working. Again, reviews happen before anything is migrated up into the next level.
Production is the final environment, and it’s the one that end-users actually interact with. Here is where you’ll store the “official” version of everything, and it’s the thing that is most critical to get right.
Advantages of different environments
Segregating these different environments ensures no accidental changes get into any official work product. Actuaries can be free to explore without worrying they’ll screw up something downstream.
Plus, it allows for concurrent work. One professional can be trying something new while others continue their regular output without interruption.
And there’s a lot more opportunity for peer review or validation that what’s being delivered is right. Often, it seems like actuaries, with their “just do it” mentality, skip intermediate peer reviews. Requiring reviews and approvals before migration forces additional eyeballs to see everything that goes into the final product.
Disadvantages of different environments
It takes a bit more maintenance to make this happen. You may need to consider reworking your file storage hierarchy. Some actuarial departments store new official versions of their model each quarter according to the calendar year and calendar quarter. While this may help to organize results, it also requires additional model data storage, additional management, and is just a pain to keep up with each period.
A better way would be to store one official version of the model, and apply it during the various periods.That way, you could also force separate branches of the model into the Development and Testing regions at various points of time. This would ensure perpetual consistency across the various model structures and enable all the advantages of different regions.
To be fair, you may not need 3 separate environments for everything, especially for any ad-hoc or one-time use spreadsheets. We have seen this done well with two: develop/test and production. This is the minimum necessary to ensure the advantages get applied and disadvantages are avoided.
2. Develop for the actual use case, not the assumed problem
Rarely is the thing that the end user wants to do the actual thing. For example, you might say, “I want to run cash flow testing.” Well, no, what you actually want to do is perform asset adequacy analysis. Cash flow testing is your chosen method of doing that.
This is a fairly classic example that’s been used countless times. It illustrates the importance of diving deep into requirements, not just assuming you know:
The same principle exists in the actuarial world. You might start building a spreadsheet because “I want to examine COI charges on a monthly basis.” Sure, that’s a small part of what you want, but the bigger picture is what’s actually needed. In this case, what you actually need is a multiple-decrement model with monthly time steps. Once you’ve got that available, then you can segregate out COI charges by themselves. Because COI charges don’t exist in a vacuum, they are part of an integrated system.
A similar thing happened while building SLOPE. [We’ll give a longer explanation of why things look the way they do in a later blog post.] Originally we thought we wanted to reproduce some of the exact same things other systems were doing, but in a kind-of-sort-of-maybe-a-little-bit-different way. Turns out, when we dove deeper, what we really wanted was a different way of working altogether.
And that’s what software designers know. They know to design for what the user really wants, not just what they tell you they want at first. It’s when you get that question of “Hey, explain to me again just what you’re trying to do here” which forces you to reset and determine exactly what it is you really want out of the things you’re building.
The same should be true for actuarial or financial model development. Those models should be designed for the actual use case, not the assumed one. Because that end-user is often coming to you with a very specific problem, informed by a very specific set of experiences. Which may not be the deeper, underlying problem at all.
In order to effectively understand the problem, there needs to be more time up-front investigating “what do you really want to do with this thing?” Dive deeper and ask, “Yeah, but why that? And what would that do for you? What else would you want? Why does it have to be that way?”
Once you’ve gotten answers to your questions, beyond just the simple “We need to know the last quarter’s widget production numbers,” then you go build it using agile principles and make it great.
3. Modularity and reuse
KISS – Keep It Simple, Stupid and DRY – Don’t Repeat Yourself)
One essential element of programming is to keep things simple. An easy rule of thumb is “Each thing is one thing and one thing only.”
Actuaries often get into trouble with their convoluted formulas in spreadsheet cells, because they’ve impressed themselves with how elegant they can program an INDEX-MATCH inside a VLOOKUP.
That kind of multi-use causes problems when you’re trying to diagnose errors in output values and have to investigate nested functions five or six layers deep.
Instead, a better way would be to break down these multiple steps into individual components. Software development does this thing naturally, with the KISS principle as well as DRY – Don’t Repeat Yourself.
In the above example, there are multiple instances of repetitions. This model is calculating multiple sums multiple times. It would be much more efficient to calculate the various sums once, and then reference the already-calculated value.
Software programming holds this DRY principle very high, because repetition is inefficient and wasteful. Better use could be made of the programmer’s time, the reviewer’s time, the tester’s time, and the server’s run-time if all of that redundancy is eliminated. That can be accomplished by defining individual calculations once and referencing them many times.
This also leads to the idea of modularity, then, where self-contained components can be developed for specific purposes.
When modularity is built into models, then, you can swap out components without having to recreate the whole thing.
Read more about modularity in actuarial models in Non-Technical Principles of Successful Actuarial Model Design.
This allows for faster improvement opportunities because you can begin to introduce incremental advancements. As well, when you change the underlying parts of one module, you can be sure that you haven’t broken other modules. Like replacing the alternator, rather than the whole engine.
4. Peer review
Most actuaries treat peer review as an after-the-fact confirmation, rather than an ongoing part of their work. They think, I’ll create my model and get all my results, and then get it reviewed. They then do a whole load of work, perhaps 100 steps, before anything gets reviewed.
What if there was an error way back on step 20? What if not catching it for the next 80 steps means all those 80 are invalid and need to be re-worked? Wouldn’t the ultimate output be delivered sooner, and with fewer bugs, if that was caught at step 30, for example? Now you’re re-working 10 steps, rather than 80. Immediate productivity increase.
Peer review should happen during development, testing, and before and after deployment. Software engineers know this. They also know that finding problems before they are problems saves countless hours of revision and rework later.
Separate environments is just one way of encouraging adequate review. The bigger change will be in changing a mindset towards earlier, faster, smaller reviews.
Now, this is not advocating for micro-managing or being that boss.
But we are suggesting that actuaries need to get more comfortable with more frequent reviews of their work in process, rather than after everything is complete. It will pay off in better output, faster development of model enhancements, and better documentation along the way (see below).
5. Testing
Again, actuaries, by their nature of just diving in and using things, don’t have a good history with testing their work for thing that might break it. “If it’s not broken now, it probably won’t be in the future” almost seems to be the actuarial mantra.
Software developers, though, realize that there are all kinds of edge cases. What works now may not work in the future if what’s built is not ready for robust use. And you only find out if it’s robust when you test.
Good testing isn’t done by the person who’s actually building it. That person is too close. They know all the ways to trick the system into thinking it’s doing something it’s not. Instead, ask a novice to test your new model. Or someone in another department. If they can figure it out,
Also, it would be good to have a standard set of tests that are run each time, so you can benchmark what kinds of testing experiences are normal, what are not. Then you can know how to use any test results to influence future development prioritization.
Finally, you need to set an appropriate limit of error tolerance before moving forward. Most actuaries think it needs to be perfect, so they delay until it is. They test and refine and revise and retest, without understanding that sometimes it’s okay to go forward with “good enough”.
We think actuaries will need more comfort with deployment at acceptable levels, with the expectation of perpetually working on backlog development items after the model or solution is in production. That will enable faster, incremental improvements [link to agile style] of the output product.
6. Concurrent documentation
Most actuaries just work and work and work, happily getting things done. They sometimes (often?) ignore any sort of concurrent documentation. The only time documentation happens is when they come back after the project is finished. Then they struggle to recreate their thought process, or simply list what the current state is.
We get it. Documentation feels like unproductive work, so it takes time away from getting more done. And while saving time during development sounds good in theory, in practice it leads to lower-quality output.
The problem is that there is often no documentation on the thought process and attempted solutions which do not work. Future generations struggle with the same problems, or are challenged to figure out just where a number came from and why it was calculated that way. More importantly, most of the time there is no documentation at all on where this thing goes later, which can be a big problem if an unknowing user makes a change that affects everything downstream.
Concurrent documentation enables better understanding of the decisions made along the way. It eliminates the need to either try to hold all ideas in your head or recreate lost ones at the end of the project. As well, it allows for concurrent review because the reviewer can compare what was intended with what actually came out. It gives the user a chance to think about where this piece might be going, and the impact of a change to that single piece.
Also, consider the principle of “self-documenting code”. This is the idea that code should be written in such a way that it is clear to the user exactly what it is for and why.
Many times actuarial models have formulas that perform obscure calculations, transformations, or data rearrangement. But the formulas (as above) are very intricate, hard to understand, and difficult to decipher.
Better actuarial processes will consider how to add self-documenting code (and formulas in spreadsheets) to models as they are developed. Consider enhancing the intermediate documentation of your modules within either the code itself or the model setup to ensure better transfer of knowledge from one generation to the next.
Conclusion – think like a software developer, not an actuary
Note – we’re not saying “programmer” there. Programming is just one element of software development. Other aspects include segregating code (or a model) into various environments, appropriate use case design, enhanced documentation, incorporating modularity, etcetera.
Adding these principles to any actuarial model development should help reduce errors and re-work, and give actuaries fewer headaches over time.
Let’s get there together, shall we?
For more on this topic, check out the following links:
What are the best software engineering principles?
Principles and Practices of Software Development
Basic principles of good software engineering