Chisels: Worth it or not?

From our last post about the new patch, we found that on average 1% quantity increases the number of maps by 1.006 times. This means that 1 chisel will increase the average number of maps by only 3%. So, are they worth it or not? Well, it’s highly dependent the expected number of drops from your map. So, let’s walk you through it:

First, use this formula:

ExpectedDrops = exp{1.89 – 0.024*Level + 0.0060*Quantity}

to find the expected number of drops from your current map. Now, multiply this value by 0.03, let’s call that ExpectedIncrease. Finally, assume the level of the map dropped is equal to the level of the map you are running. Now, is the ExpectedIncrease * Value of Expected Map better than the value of a chisel? If yes, then go for it! Chisel as much as you can! If not, then run as is.

Example:

Consider a Level 78 Map with 64% Quantity.  Then,  our ExpectedDrops = 1.49. Now, ExpectedIncrease = 1.49*0.03 = 0.0447. If we expect to get a level 78 map from this and we value it at 8 chaos, then a chisel would have to be worth less than 0.36 chaos for it to be worth it.

Second Example: White Map

Consider now you have a white level 78 map, with no Quality.  In this case, our ExpectedDrops = 1.018, ExpectedIncrease this time is fairly higher, 1.018*(0.03*20) = 0.6108. Now, if we go and roll this map, and assuming again that a 78 will drop and is valued at 8 chaos, then the chisel must be worth less than 4.8 (20%) / 4 (4 chisels used) =  1.22 chaos, which is far, far more likely.

As you can see, most likely, chiseling non-white maps will not be worth your time. For white maps, it’s worth doing if you can push the expected drop value high enough. It’s a tricky balance: since you must chisel before you know the mods you’ll have on the map, you risk having to spend chaos to reroll a map which doesn’t have the proper mods to make the chisels worth it. This means that chiseling, for the most part, is only worth it if the map level is already high enough such that the expected drop is worth the chisels, ignoring mods. At this point in Ambush, that’s probably not until level 76.

Patch Changes?: The good, the bad, and the ugly.

Today, we’re going to take a quick look at what has happened to map drop rates from the patch with updated statistical methods. I should note that the analysis this time around has much less power to detect differences due to the much lower sample size, and I’ll talk a bit more about how this may affect our analysis towards the end.  As such, I highly recommend reading the discussion section to understand the limitations of these findings.

tl;dr: Key Findings
1) The patch may have changed the way level interacts with both the average drop rate and number of map drops.
2) At level 66, we find more maps but more likely at lower levels. However, as we run higher level maps, we find less maps but we’re more likely than at pre-patch to find higher level maps.
3) Sample size is still low so please submit more data!
4) Really, we need more data!

The Data

Our post-patch data consists of 746 runs with 548 from Ambush, 78 from Hardcore, 35 from Invasion, and 85 from Standard.  As expected of a new league, the vast majority of runs are in the lower range.  A total of 1193 maps dropped from all runs.

Figure 1

For pre-patch runs, we used the same data set as the previous post, please refer to it for details.

Total Number of Map Drops

Differences between Post and Pre-patch Data

A log-linear model was used assuming a negative binomial distribution, due to over-dispersion in the data. I kept all variables in the model for the sake of controlling for as much as possible and making the test as conservative as possible. However, there was significant effect for the interaction of Level and Patch and for Patch. If you remember from the previous post, level had no affect on the number of maps previous to the patch. This means that now, after the patch, level has become a significant factor. None of the other modifiers have indicated that the patch have changed in the way they interact with the number of map drops. On a more conceptual level, what this is telling us is that at baseline (Level 66), the average number of map drops after the patch has increased from 1.01 to 1.32. However, after the patch, the number of map drops decreases for every level increase of the map ran.

Post-Patch Data Only

Running the model for post patch data only, we found that the only significant factors are Level and Quantity. The formula for the predicted average number of maps is then:

exp{1.89 – 0.024*Level + 0.0060*Quantity}

As stated previously, this means that at level 66, the baseline predicted average number of maps is 1.36 and every increase in level lowers this by 0.997 times and every % increase in quantity increases this by 1.006 times.

We do not find any other interaction and it is very possible this is due to low sample size and that we just do not have the sample size to see those effects.

Average Drop Level

Differences between Post and Pre-patch Data

I ran a normal linear regression for average drop level, excluding all runs without any map drops. Similar to above, I kept all variables in the model and again found that the interaction between level and patch was significant, as well as patch alone. Level has once again become meaningful post-patch. At baseline (Level 66, with no modifiers and no quantity),  the average drop level decreased from 66.4 pre-patch to 66.27 post-patch.  However, now after the patch, it seems each level increase now has a greater positive effect on the chances that a higher level map will drop.

Post-patch Data Only

Figure 2

Figure 2

Running the model for post patch data only, this time using a truncated normal regression model, we found that the only significant factor is Level. The formula for the predicted average number of maps is then:

max(66, 16.84 + 0.527*66)

As stated previously, this means that at level 66, the baseline predicted average map drop is 66 and every increase in level increases this by 0.527.

We do not find any other interaction and it is very possible this is due to low sample size and that we just do not have the sample size to see those effects.

Limitations

The effects we’re seeing between pre and post-patch data may very well be due to the fact that the there is barely any data for level 77 and 78 maps. Removing those data points do not change the results of the analysis.

It is important to note that the small sample size may very well cause the smaller effects of Rares/Magic/Pack Size to be obscured. If there is some interaction between level, patch, and these mods, then not accounting for them could lead to Type II error (concluding that there is a significant effect for Patch/Patch*Level when there isn’t). This is a very real possibility as it is more likely for someone to roll for better mods at higher levels than at lower levels.

Finally, it was not possible to run some interaction effects and a more in depth analysis without a larger sample size. I highly encourage everyone to submit more data so our analysis can be more powerful.

Analysis was done with SAS 9.3 and R.