One of the critical elements of any enterprise wide IT system is the ability to know when once important data is really no longer used and can either be purged or archived. I have the honor of speaking at Salesforce’s Dreamforce 2013 on three different subjects and of of them is about “Managing Large Data Volumes”.
In my experience, archiving often seems to be an after-thought and for good reason. If you are beginning to use a new system and there is not much data in your system, then performance is typically fine and you really don’t think about needing to purge and archive the data. Oftentimes, smaller companies can go years without any need to archive and put some of your older data aside. However if you are a larger enterprise company, this will not be the case. Eventually, you will need to archive your data. Normally, IT systems that require archiving include; slow running reports, data pulling up on the UI really slow and finally simply running out of space on your hardware (if you are not in the cloud).
Archiving large data should be an integral part of your data management policy. The first thing that must be considered is what can and cannot be archived. Typically each country has their own set of legal restrictions of how you can dispose or handle customer data, so ensure you know this first. Once you have identified the legal aspects, then you can put in business rules (what data to archive and how your company will access the data / SLA’s ect) in place and lastly the ongoing sustaining data maintenance should be considered such as how often do you need your data pushed into an archive system (monthly, weekly ect)
After all these steps above, then you should consider the technology solution for your company. Technology has become so easy nowadays with archiving data that you can choose from a variety of tools in the marketplace. My recommendation is a cloud based archiving solution and make sure you pilot the solution and can retrieve the data in a timely manner to meet the business needs of your company. Lastly, whatever solution you do chose, avoid replicating data from your current system if it is not the system of record. Another words, don’t make copies of the same data.
Finally, here is a good interview I found on data archiving from Tech Talk. Check it out:
Hector
- Tags: Archive, Dreamforce, Salesforce.com
