Piotr Kononow
Many people in IT/data consider documentation as a cost. In this article, I'd like to have a look at this from the opposite angle - what is the cost of NOT having a documentation?
1. Cost of (Constant) Search of the Data
Data workers spend more time on time on searching for, and trying to make sense out of the data, than on putting that data toghether on ther report or geting valuable insights. This is because data is "hidden" among hundreds or thousands of tables and columns with strange names, unclear relationships, and cryptic codes.
Multiply this by all the people working with data and that time adds up to a significant sum each year, even in a small team.
2. Cost of Errors
Even if you get the reports and data you ask for - if people preparing it do not understand it's logic you may end up with completely incorrect results. That may lead to bad decisions. Or get you fired.
3. Cost of Reinventing the Wheel
Many analyses require tedious data preparation, cleansing, temporary data sets and custom scripts. Without communication you might end up reinventing what was previously done by someone. Someone who spent many hours (or days) on research, design, development, and testing.
4. Cost of Restoring Lost Knowledge
When things just work we asume they are easy. Think electricity. You switch the light and it is simply there. But what if your power station just decided they have had enough, you were not paying them enough, or nearby city offers more and just quit. You come home one day and there is no light, air conditioning, TV, PC, you can't charge your phone and water is coming out of your fridge. How much would it cost you to restore that know how of production of electricity? Right?
Think the same about your data - someone has that know-how and if he/she leaves it might not be easy to restore that. Documentation reduces that cost.
5. Cost Scaling Your Teams
When you want to do more with your data the solution is obious - hire more people. But new employees are not as productive as the existing ones. There is a learning curve, and without a support they just might provide zero value to your team, so in fact a negative one. That support costs experienced team members hours or days of education about the data. The best way to avoid this hustle is a convenient, easily accessible documentation. That is something that can free your current experts to do their job, and get new ones started quickly in a self-service manner.
6. Cost of Lost Opportunities and Uninformed Decisions
Think about the last bad (ok, not optimal) decision your ogranization recently took. Maybe you recently invested into development of not the right product or conquering not the right market? Could that be avoided with a little data and insights? What if people in your team had easy access to useful data - maybe it could have been avoided? Think of how much that accumulates over time - what if your competiton made less mistakes?
Documentation: It's Not Hard
Let's have a look at the cost of the documentation. Many think about dedicated people in the team or weeks, or months of work. But in fact, core documentation could be done quickly by your existing people in a day or two. And investment will pay itself quickly.
Here are some quick wins you could get under a day of work using Dataedo:
- Group tables into folders (called Subject Areas/Modules) to indicate their purpose or functional module,
- Decode unclear names into meaningful business name using titles,
- Indicate unused (or not yet used) tables - with title or custom field,
- Define relationships/foreign keys where there are no FK constraints,
- Assign architect or owner to tables with custom field - to know who created table and who to talk to about it,
- Visualize model with set of small, easily digestible ERDs.
Doing this will not take long and when you share the documentation you will see a boost in productivity and data literacy in your team.