Chisels: Worth it or not?

From our last post about the new patch, we found that on average 1% quantity increases the number of maps by 1.006 times. This means that 1 chisel will increase the average number of maps by only 3%. So, are they worth it or not? Well, it’s highly dependent the expected number of drops from your map. So, let’s walk you through it:

First, use this formula:

ExpectedDrops = exp{1.89 – 0.024*Level + 0.0060*Quantity}

to find the expected number of drops from your current map. Now, multiply this value by 0.03, let’s call that ExpectedIncrease. Finally, assume the level of the map dropped is equal to the level of the map you are running. Now, is the ExpectedIncrease * Value of Expected Map better than the value of a chisel? If yes, then go for it! Chisel as much as you can! If not, then run as is.

Example:

Consider a Level 78 Map with 64% Quantity.  Then,  our ExpectedDrops = 1.49. Now, ExpectedIncrease = 1.49*0.03 = 0.0447. If we expect to get a level 78 map from this and we value it at 8 chaos, then a chisel would have to be worth less than 0.36 chaos for it to be worth it.

Second Example: White Map

Consider now you have a white level 78 map, with no Quality.  In this case, our ExpectedDrops = 1.018, ExpectedIncrease this time is fairly higher, 1.018*(0.03*20) = 0.6108. Now, if we go and roll this map, and assuming again that a 78 will drop and is valued at 8 chaos, then the chisel must be worth less than 4.8 (20%) / 4 (4 chisels used) =  1.22 chaos, which is far, far more likely.

As you can see, most likely, chiseling non-white maps will not be worth your time. For white maps, it’s worth doing if you can push the expected drop value high enough. It’s a tricky balance: since you must chisel before you know the mods you’ll have on the map, you risk having to spend chaos to reroll a map which doesn’t have the proper mods to make the chisels worth it. This means that chiseling, for the most part, is only worth it if the map level is already high enough such that the expected drop is worth the chisels, ignoring mods. At this point in Ambush, that’s probably not until level 76.

Patch Changes?: The good, the bad, and the ugly.

Today, we’re going to take a quick look at what has happened to map drop rates from the patch with updated statistical methods. I should note that the analysis this time around has much less power to detect differences due to the much lower sample size, and I’ll talk a bit more about how this may affect our analysis towards the end.  As such, I highly recommend reading the discussion section to understand the limitations of these findings.

tl;dr: Key Findings
1) The patch may have changed the way level interacts with both the average drop rate and number of map drops.
2) At level 66, we find more maps but more likely at lower levels. However, as we run higher level maps, we find less maps but we’re more likely than at pre-patch to find higher level maps.
3) Sample size is still low so please submit more data!
4) Really, we need more data!

The Data

Our post-patch data consists of 746 runs with 548 from Ambush, 78 from Hardcore, 35 from Invasion, and 85 from Standard.  As expected of a new league, the vast majority of runs are in the lower range.  A total of 1193 maps dropped from all runs.

Figure 1

For pre-patch runs, we used the same data set as the previous post, please refer to it for details.

Total Number of Map Drops

Differences between Post and Pre-patch Data

A log-linear model was used assuming a negative binomial distribution, due to over-dispersion in the data. I kept all variables in the model for the sake of controlling for as much as possible and making the test as conservative as possible. However, there was significant effect for the interaction of Level and Patch and for Patch. If you remember from the previous post, level had no affect on the number of maps previous to the patch. This means that now, after the patch, level has become a significant factor. None of the other modifiers have indicated that the patch have changed in the way they interact with the number of map drops. On a more conceptual level, what this is telling us is that at baseline (Level 66), the average number of map drops after the patch has increased from 1.01 to 1.32. However, after the patch, the number of map drops decreases for every level increase of the map ran.

Post-Patch Data Only

Running the model for post patch data only, we found that the only significant factors are Level and Quantity. The formula for the predicted average number of maps is then:

exp{1.89 – 0.024*Level + 0.0060*Quantity}

As stated previously, this means that at level 66, the baseline predicted average number of maps is 1.36 and every increase in level lowers this by 0.997 times and every % increase in quantity increases this by 1.006 times.

We do not find any other interaction and it is very possible this is due to low sample size and that we just do not have the sample size to see those effects.

Average Drop Level

Differences between Post and Pre-patch Data

I ran a normal linear regression for average drop level, excluding all runs without any map drops. Similar to above, I kept all variables in the model and again found that the interaction between level and patch was significant, as well as patch alone. Level has once again become meaningful post-patch. At baseline (Level 66, with no modifiers and no quantity),  the average drop level decreased from 66.4 pre-patch to 66.27 post-patch.  However, now after the patch, it seems each level increase now has a greater positive effect on the chances that a higher level map will drop.

Post-patch Data Only

Figure 2

Figure 2

Running the model for post patch data only, this time using a truncated normal regression model, we found that the only significant factor is Level. The formula for the predicted average number of maps is then:

max(66, 16.84 + 0.527*66)

As stated previously, this means that at level 66, the baseline predicted average map drop is 66 and every increase in level increases this by 0.527.

We do not find any other interaction and it is very possible this is due to low sample size and that we just do not have the sample size to see those effects.

Limitations

The effects we’re seeing between pre and post-patch data may very well be due to the fact that the there is barely any data for level 77 and 78 maps. Removing those data points do not change the results of the analysis.

It is important to note that the small sample size may very well cause the smaller effects of Rares/Magic/Pack Size to be obscured. If there is some interaction between level, patch, and these mods, then not accounting for them could lead to Type II error (concluding that there is a significant effect for Patch/Patch*Level when there isn’t). This is a very real possibility as it is more likely for someone to roll for better mods at higher levels than at lower levels.

Finally, it was not possible to run some interaction effects and a more in depth analysis without a larger sample size. I highly encourage everyone to submit more data so our analysis can be more powerful.

Analysis was done with SAS 9.3 and R.

1000 Data Points – First Impressions

In this first look, we take a look at what modifiers affect average drop level, total number of maps dropped, and level 78 map drops. Some interesting takeaways include:

  1. Only map level (and perhaps Rares) actually influence the average level of maps dropped.
  2. However, level does not affect the number of maps dropped.
  3. Pack Size, Magic, and Rares all have the same effect on the number of maps dropped.
  4. The double boss modifier is useless when it comes to maps.
  5. Obtaining level 78 maps is only about the level of the map you’re running and the Rares modifier.
  6. It is never sustainable to only run level 78 maps.

The Data

The current data set consists of 1269 runs with 1251 from Standard and 18 from Hardcore. Of those, the majority were above map level 72. A barchart of the breakdown can be found below. A total of 2243 maps dropped from all runs.

Hist_MapLevel

Average Drop Level
I used a normal linear regression model for the average drop level and excluded all runs without a drop because it was not informative. It looks like there may be some issues with assuming Gaussian errors but I’m not fussed enough to have to code the modeling from scratch, there’s not an easy fix for this kind of data, and the conclusions were within expectations enough that I’m comfortable drawing conclusions. The only modifier that was predictive of average drop level was the level of the map. For every increase in level, the expected average map drop level increases by 0.69. You can basically write the following for expected average map drop level by map level: At baseline (level 66), the average map drop is 66.6. Then just add .68 for each level increase. At max level (78), the average map drop is 74.82.

Skyl3lazer note: My findings were slightly different, I thought that Rares were still significant, granted only with a coefficient of about 1/5th of the Level’s, or around .136 added to the Drop Level per point of Rare Monsters

Number of Map Drops
I modeled the number of map drops using a Negative Binomial model. I tried a Poisson model but found overdispersion was an issue. Significant factors were Quantity, Pack Size, Magic, and Rares.

At baseline (no mods), the average number of map drops was 1.06.

For every x% increase in the Quantity mod, the average number of map drops increases by e(0.0052*x) times or for every unit increase in the Quantity mod, we augment the average number of map drops by 1.005. So for example, the average modifier for Quantity was 74.1 and the largest was at 144. So the average number of map drops with a 74.1 quantity (and none of the other significant mods) is 1.55 and the average number of map drops with a 144 quantity is 2.22.

For every x% increase in the Pack Size mod, the average number of map drops increases by e(0.00357*x) times or for every unit increase in the magic mod, we augment the average number of map drops by 1.004. So, for example, the smallest modifier for Pack Size was 20%, average modifier for Pack Size was 36.25% and the largest was at 50%. So the respective average number of map drops for each of those are: 1.14, 1.2, 1.26 (with no other mods).

For every x% increase in the Magic mod, the average number of map drops increases by e(0.003785*x) times or for every unit increase in the magic mod, we augment the average number of map drops by 1.004. So for example, the smallest modifier for Magic was 20%, average modifier for Magic was 36.43% and the largest was at 50%. So the respective average number of map drops for each of those are: 1.14, 1.21, 1.28 (with no other mods).

For every x% increase in the Rares mod, the average number of map drops increases by e(0.003711*x) times or for every unit increase in the magic mod, we augment the average number of map drops by 1.004. So for example, the smallest modifier for Rares was 20%, average modifier for Rares was 35.37% and the largest was at 50%. So the respective average number of map drops for each of those are: 1.14, 1.21, 1.27 (with no other mods).

It is interesting to note here that in terms of the number of map drops, the % increase modifiers (Pack Size, Rares, and Magic) have the exact same effect.

In terms of getting 78 Maps
Since it is of particular interest to find the sustainability of only running 78 Maps, I also modeled for just getting 78 maps. Since, as SunnyRay from PoE forums pointed out, only level 76+ maps can drop level 78 maps, only those were used. That left us with a good 485 observations. For this, I used a Poisson model, as there was no overdispersion issues. An ordinal logistic model was also considered since the maximum number of 78 Map drops from a single run was two, however the Poisson model performed similarly with better fit. Funny enough, although I expected level to be significant, the only other factor here that mattered were Rares.

At baseline (level 76), the average number of level 78 map drops was 0.09.

For every level increase, the average number of level 78 map drops increases by e(0.68*x) times or for every unit increase in the level of the map, we augment the average number of map drops by 1.98.  So the respective average number of level 78 map drops for each eligible level are: 0.09 (level 76), 0.18 (level 77), 0.35 (level 78).

For every x% rarity increase, the average number of level 78 map drops increases by e(0.019*x) times or for every unit increase in the level of the map, we augment the average number of map drops by 1.02. So for example, the smallest modifier was 20, average modifier was 35.37 and the max modifier was 50. So the respective average number of level 78 map drops for each of those are (in a level 76 map): 0.13, 0.17, 0.23. (Just for perspective, in a level 78 map, these numbers would be: 0.51, 0.68, 0.89)

Interestingly enough, this implies that for high level maps, if you only care about getting level 78 maps, you should very much care about the Rares modifier. However, you can never expect to break even with level 78 maps and only run level 78 maps sustainably.

I’m just going to emphasize for all of these models, the rest of the mods weren’t even close to being statistically significant. Which means they are worthless for whatever we’re modeling.

Some To Do’s:

  1. Look into transformations of data as appropriate with interactions.
  2. Find most efficient leveling method that is also self-sustaining.
  3. Look into why I did not find Rares to be significant for level while Sky did not. (Model level covariate on a tiered basis.)

Backwards selection was used for variable selection. All modeling was done in R.

 

Some To Do’s:

Statistics – Intro

From time to time after gathering submissions I’ll do detailed breakdowns of the data I’ve acquired. Remember, spread this so I can get better statistics!

You can view an un-abridged, unsorted, and poorly formatted version of all of the data collected via the non-legacy submission form Here