Patch Changes?: The good, the bad, and the ugly.

Today, we’re going to take a quick look at what has happened to map drop rates from the patch with updated statistical methods. I should note that the analysis this time around has much less power to detect differences due to the much lower sample size, and I’ll talk a bit more about how this may affect our analysis towards the end.  As such, I highly recommend reading the discussion section to understand the limitations of these findings.

tl;dr: Key Findings
1) The patch may have changed the way level interacts with both the average drop rate and number of map drops.
2) At level 66, we find more maps but more likely at lower levels. However, as we run higher level maps, we find less maps but we’re more likely than at pre-patch to find higher level maps.
3) Sample size is still low so please submit more data!
4) Really, we need more data!

The Data

Our post-patch data consists of 746 runs with 548 from Ambush, 78 from Hardcore, 35 from Invasion, and 85 from Standard.  As expected of a new league, the vast majority of runs are in the lower range.  A total of 1193 maps dropped from all runs.

Figure 1

For pre-patch runs, we used the same data set as the previous post, please refer to it for details.

Total Number of Map Drops

Differences between Post and Pre-patch Data

A log-linear model was used assuming a negative binomial distribution, due to over-dispersion in the data. I kept all variables in the model for the sake of controlling for as much as possible and making the test as conservative as possible. However, there was significant effect for the interaction of Level and Patch and for Patch. If you remember from the previous post, level had no affect on the number of maps previous to the patch. This means that now, after the patch, level has become a significant factor. None of the other modifiers have indicated that the patch have changed in the way they interact with the number of map drops. On a more conceptual level, what this is telling us is that at baseline (Level 66), the average number of map drops after the patch has increased from 1.01 to 1.32. However, after the patch, the number of map drops decreases for every level increase of the map ran.

Post-Patch Data Only

Running the model for post patch data only, we found that the only significant factors are Level and Quantity. The formula for the predicted average number of maps is then:

exp{1.89 – 0.024*Level + 0.0060*Quantity}

As stated previously, this means that at level 66, the baseline predicted average number of maps is 1.36 and every increase in level lowers this by 0.997 times and every % increase in quantity increases this by 1.006 times.

We do not find any other interaction and it is very possible this is due to low sample size and that we just do not have the sample size to see those effects.

Average Drop Level

Differences between Post and Pre-patch Data

I ran a normal linear regression for average drop level, excluding all runs without any map drops. Similar to above, I kept all variables in the model and again found that the interaction between level and patch was significant, as well as patch alone. Level has once again become meaningful post-patch. At baseline (Level 66, with no modifiers and no quantity),  the average drop level decreased from 66.4 pre-patch to 66.27 post-patch.  However, now after the patch, it seems each level increase now has a greater positive effect on the chances that a higher level map will drop.

Post-patch Data Only

Figure 2

Figure 2

Running the model for post patch data only, this time using a truncated normal regression model, we found that the only significant factor is Level. The formula for the predicted average number of maps is then:

max(66, 16.84 + 0.527*66)

As stated previously, this means that at level 66, the baseline predicted average map drop is 66 and every increase in level increases this by 0.527.

We do not find any other interaction and it is very possible this is due to low sample size and that we just do not have the sample size to see those effects.

Limitations

The effects we’re seeing between pre and post-patch data may very well be due to the fact that the there is barely any data for level 77 and 78 maps. Removing those data points do not change the results of the analysis.

It is important to note that the small sample size may very well cause the smaller effects of Rares/Magic/Pack Size to be obscured. If there is some interaction between level, patch, and these mods, then not accounting for them could lead to Type II error (concluding that there is a significant effect for Patch/Patch*Level when there isn’t). This is a very real possibility as it is more likely for someone to roll for better mods at higher levels than at lower levels.

Finally, it was not possible to run some interaction effects and a more in depth analysis without a larger sample size. I highly encourage everyone to submit more data so our analysis can be more powerful.

Analysis was done with SAS 9.3 and R.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>