In a first article about the Restore Point Simulator (RPS) I talked about the history of RPS and why it was created. I want to take the time now to explain the correct input parameters. Although I added some tool-tips in 0.3.1, people are sometimes confused how it all ties together.
TL;DR? The RPS GUI maps quite directly to the Veeam interface, check the screenshots to see how, if you already understand how Veeam works.
For those that didn't read the previous article, I'll repeat the formula here which we use at Veeam to do rough estimations. Why? Because RPS is directly based on it:
D or delta is the amount of data that changes between backups. In RPS the parameter is called change rate. This is dependent on 2 parameters. First of all the frequency of backups which is in most companies "daily". Secondly the application. Don't put this parameter to low. Veeam backups up at the block level. A small update, can flag a bigger block on the VMDK level than you estimated. If you fill up a disk sequentially like a file server, you won't notice this so much because 10 sequential changes could be only flagging 1 block. However, if you are doing a lot of small random I/O on various locations, the number can quickly rise. So from my experience, a number of 5% changes is fairly optimistic while 10% is rather conservative. I personally prefer a more conservative approach.
Finally C or compression which is called "Data left after reduction". I have had tons of people discussing this parameter with me because you can interpret this one in various ways. However look at the formula and it will all make sense. The formula basically says:
For some people this feels counter-intuitive. If you prefer compression factors like 2x or 3x instead, you can easily convert those by dividing 1/(compression factor). So for example 2x would be 1/2 = 50%. If we put that in the previous formula, we would get 50 backup data = 50%*(100 total data in). So data was compressed to 2x. For 3x, you would get 1/3 or 33% =~ 35%. Again if we fill this into the formula you would get 35 = 35%*(100). Use this link if you really want to use 33%
If you want to disable compression, put data reduction to 100%.
In the formula there are 2 parameters left, F & R. Like discussed in the previous article, these one are hard to calculate, and that is exactly what RPS does for you. So for retention points, you should just define your policy. If you need 14 daily restore points, put in 14 for "Retention points" and daily for "Interval". You will see that in some case, you might end up with more restore points than you configured. This can be normal as Veeam considers your retention policy but also dependencies between Fulls and Incrementals. So don't try to adjust the retention points because you feel it miscalculated, instead take a look at the example given in previous blog article.
What also influences the amount of parameters is the style parameter. This actually maps directly to Veeam GUI. For example, "Backup Copy Job" (BCJ) should be quite easy to understand. It refers directly to a Backup Copy Job . Selecting should it should also show you BCJ specific configurations
For the other "styles" called Incremental and Reverse, the other setting map mainly to the advanced settings of a regular job except for "Retention Points"
If you selected Incremental without any active fulls or synthetic fulls, you would get a forever incremental job. It also maps directly to the GUI but more specifically to the advanced settings of a regular job. Granted, the checkboxes are under the buttons "Days" and "Months". Also there is no global "enable" checkbox for synthetic or active but do understand that checking one of the boxes enables "Active" or "Synthetic". Finally "Monthly" has preference over "Weekly". In the Veeam GUI, you can selected only one, here you can enable checkbox for both but weekly will be ignored when you enabled one Month.
So if you want a weekly synthetic backup, it would be like shown below. I did not implement "Transform", and with good reason. Until the day of today, I have received 0 requests for it.
A Monthly Active Backup would like shown below. Important here is to know this does not defines GFS policies. Those can only be defined in a Backup Copy Job. I once had a guy that checked only January because he want to have a "yearly full backup for archiving". He was quite surprised by the result. Basically you just told the engine that you want to have a yearly chain which only get resets in January.
Finally "Reverse" incremental, can be found in the same place.
That leaves only 1 parameter left, and that is the growth simulation. It is a recent addition to RPS, and personally I think it is one of the coolest things added to it. Let me explain what it does and how it works. If you need to size for the following 3 years, and you know that you have a growth rate of 10% on a yearly basis, you can just add that to the RPS. What it does is, it takes your "used data", and grows it on a daily basis via: Future Used Data = Used Data * ( 1 + 10% ) ^ (Day X/365). Thus on the last days of the simulation -after 3 years-, it would put Future Used Data = Used Data * ( 1 + 10%) ^ (1095/365).
So lets imagine you have chosen reverse incremental, 3 years simulation, 10% year on year data growth and 1000GB of data. For simplicity, let's disable compression. Calculating this manually would give you roughly 1000*(1+10%)^(1095/365) = 1000*(110%)^3 = 1000*(1,10)^3 = 1000*1,10*1,10*1,10 = 1331. Now check the Full backup in the configured example. Also interesting is that the increments from 2 days ago (retention point 3), has a smaller "Future Used data" set. It used 3y - 2 days in the formula, thus Future Used Data = 1000*(1+10%)^(1093/365) =~ 1330,30.
The differences are small in regular jobs but once you go GFS, the time growth calculation can have a dramatic impact which is hard to calculate with excel.
With this, I have discussed all parameters. Some might ask, what about "Quick Presets", what does that do? Well they are just quickly preconfigured scenario's that you can use. For example, if you want to have a monthly active backup, you can click all 12 select boxes or you can just select the matching style + monthly active full to quickly configure this scenario
If you made it this far, thanks for reading and enjoy playing with RPS.
TL;DR? The RPS GUI maps quite directly to the Veeam interface, check the screenshots to see how, if you already understand how Veeam works.
For those that didn't read the previous article, I'll repeat the formula here which we use at Veeam to do rough estimations. Why? Because RPS is directly based on it:
Backup size = C * (F*Data + R*D*Data)This formula and RPS have 3 parameters in common. Data would be the first one and it maps to "Used Size GB". To give you and example, if you would have a VM with one VMDK, the used size would be the amount of blocks the VM has already written to. So for example if you would have a thick provisioned VMDK of 50GB but you only use 20GB inside the guest, the used size would be around 20GB. A more correct definition would be what the VM is actually using when it would be thin provisioned. Because Veeam backs up at the block level, a thin provisioned disk would be exactly what Veeam needs to process during a full backup (and some meta data).
Data = sum of processed VMs size by the specific job (actually used, not provisioned)
C = average compression/dedupe ratio (depends on too many factors, compression and dedupe can be very high, but we use 50% - worst case)
F = number of full backups in retention policy (1, unless backup mode with periodic fulls is used)
R = number of rollbacks (or increments) according to retention policy (14 by default)
D = average amount of VM disk changes between cycles in percent (we use 10% right now, but will change it to 5% in v5 based on feedback... reportedly for most VMs it is just 1-2%, but active Exchange and SQL can be up to 10-20% due to transaction logs activity - so 5% seems to be good average)
D or delta is the amount of data that changes between backups. In RPS the parameter is called change rate. This is dependent on 2 parameters. First of all the frequency of backups which is in most companies "daily". Secondly the application. Don't put this parameter to low. Veeam backups up at the block level. A small update, can flag a bigger block on the VMDK level than you estimated. If you fill up a disk sequentially like a file server, you won't notice this so much because 10 sequential changes could be only flagging 1 block. However, if you are doing a lot of small random I/O on various locations, the number can quickly rise. So from my experience, a number of 5% changes is fairly optimistic while 10% is rather conservative. I personally prefer a more conservative approach.
Finally C or compression which is called "Data left after reduction". I have had tons of people discussing this parameter with me because you can interpret this one in various ways. However look at the formula and it will all make sense. The formula basically says:
Backup Data = C * (Total Data In)So C is a factor that should make "Total Data In" smaller. So the smaller the number, the smaller the backup. Compression factor is thus a percentage that tells how much data is left after it has done it job. If you define 40% (40/100) and consider a "data in" of 100, the backup data would be 40 = (40/100)*100 . If you define 60%, you actually tell the engine that you expect worse results because the end result would be 60 = (60/100)*100. So the lower the compression value, the better the compression.
For some people this feels counter-intuitive. If you prefer compression factors like 2x or 3x instead, you can easily convert those by dividing 1/(compression factor). So for example 2x would be 1/2 = 50%. If we put that in the previous formula, we would get 50 backup data = 50%*(100 total data in). So data was compressed to 2x. For 3x, you would get 1/3 or 33% =~ 35%. Again if we fill this into the formula you would get 35 = 35%*(100). Use this link if you really want to use 33%
If you want to disable compression, put data reduction to 100%.
In the formula there are 2 parameters left, F & R. Like discussed in the previous article, these one are hard to calculate, and that is exactly what RPS does for you. So for retention points, you should just define your policy. If you need 14 daily restore points, put in 14 for "Retention points" and daily for "Interval". You will see that in some case, you might end up with more restore points than you configured. This can be normal as Veeam considers your retention policy but also dependencies between Fulls and Incrementals. So don't try to adjust the retention points because you feel it miscalculated, instead take a look at the example given in previous blog article.
What also influences the amount of parameters is the style parameter. This actually maps directly to Veeam GUI. For example, "Backup Copy Job" (BCJ) should be quite easy to understand. It refers directly to a Backup Copy Job . Selecting should it should also show you BCJ specific configurations
For the other "styles" called Incremental and Reverse, the other setting map mainly to the advanced settings of a regular job except for "Retention Points"
If you selected Incremental without any active fulls or synthetic fulls, you would get a forever incremental job. It also maps directly to the GUI but more specifically to the advanced settings of a regular job. Granted, the checkboxes are under the buttons "Days" and "Months". Also there is no global "enable" checkbox for synthetic or active but do understand that checking one of the boxes enables "Active" or "Synthetic". Finally "Monthly" has preference over "Weekly". In the Veeam GUI, you can selected only one, here you can enable checkbox for both but weekly will be ignored when you enabled one Month.
So if you want a weekly synthetic backup, it would be like shown below. I did not implement "Transform", and with good reason. Until the day of today, I have received 0 requests for it.
Finally "Reverse" incremental, can be found in the same place.
That leaves only 1 parameter left, and that is the growth simulation. It is a recent addition to RPS, and personally I think it is one of the coolest things added to it. Let me explain what it does and how it works. If you need to size for the following 3 years, and you know that you have a growth rate of 10% on a yearly basis, you can just add that to the RPS. What it does is, it takes your "used data", and grows it on a daily basis via: Future Used Data = Used Data * ( 1 + 10% ) ^ (Day X/365). Thus on the last days of the simulation -after 3 years-, it would put Future Used Data = Used Data * ( 1 + 10%) ^ (1095/365).
So lets imagine you have chosen reverse incremental, 3 years simulation, 10% year on year data growth and 1000GB of data. For simplicity, let's disable compression. Calculating this manually would give you roughly 1000*(1+10%)^(1095/365) = 1000*(110%)^3 = 1000*(1,10)^3 = 1000*1,10*1,10*1,10 = 1331. Now check the Full backup in the configured example. Also interesting is that the increments from 2 days ago (retention point 3), has a smaller "Future Used data" set. It used 3y - 2 days in the formula, thus Future Used Data = 1000*(1+10%)^(1093/365) =~ 1330,30.
The differences are small in regular jobs but once you go GFS, the time growth calculation can have a dramatic impact which is hard to calculate with excel.
With this, I have discussed all parameters. Some might ask, what about "Quick Presets", what does that do? Well they are just quickly preconfigured scenario's that you can use. For example, if you want to have a monthly active backup, you can click all 12 select boxes or you can just select the matching style + monthly active full to quickly configure this scenario
