This is basically an extension of the original experiment. I decided to post this in similar style to the other thread about large sample sizes and crit crafting but separately because the sample size is the minimum I'd personally accept for casual testing purposes like this. Will be added to the Lorebook article later.
Across a time-cumulative large sample (four-digit minimum), application of the optional crit ingredient will result in an expected percentage that aligns with the crafter’s advertised crit probability.
METHODS & MATERIALS:
My supreme master Tinker (started at level 43, ended at level 46) tested 1000 jeweller crafting rolls across T1 apprentice through T7 Supreme, during the period of 30JUL2011 through 17SEPT2011. In every single roll, the superior level 35 jeweller tools were used along with a salt of the appropriate tier, giving a final probability of 62% for a crit. The sample was spread across 60 sittings (60 discrete logs), and each individual recorded sitting ranged from a sample size of 2 up to 44; the average individual sample was 17 per sitting.
The intended sample size was an ideal 2000, but due to a bottleneck of resources (gems and salts specifically cannot be farmed easily), I stopped at 1000. All results were logged by placing the /Standard output channel into its own tab. Results were parsed in Excel 2010 and summarily graphed in pretty colors.
Due to the higher number of sittings compared to the cooking experiment, this was difficult to graph in the usual "100 ratio" style I like, so I had to be a little creative. The row on the very bottom of the graph is the expected 62%; the final actual percent of crit-vs-normal came out to be 62.4%. From the very jagged appearance of the green/blue areas, you can see that streaks and random chance are all over the place, yet the total percent came very close to the expected value.
Because each sitting on the graph represents different actual sample sizes (the very top row is actually only two rolls, making a sample of 2), keep in mind that the "weight" for every sitting will be different and is not captured effectively in the graph. For example, a sitting of 40 rolls that gives 72.5% is going to have much more weight than a sitting of 2 rolls giving 50%.
Craft sittings were not consistent--they occurred at various hours of the day and night for over 1.5 months, with each recorded sitting varying from 2 rolls to over 40. I did not interact much with the vendors in this experiment compared to the cooking experiment and did not consistently repair tools (i.e., I forgot). The crafting window was open during every sitting, though I may or may not have had a vendor window open as well.
Crafting rolls included tokens and gems, which both stacked, and all other manner of usable final products. The crit versions were sold to fund further accumulation of necessary materials (ores, ingots, gems, salts). I didn't keep track of costs compared to the cooking experiment, but I think I came out ahead if not even. The number of rolls per jeweller tier was extremely dependent on the number of salts I had at the time per that tier, and I was unwilling to pay beyond a certain amount per salts at the auction house.
- RNG: The computerized random number generator behaves, theoretically, like a die. Even with a known percentage "crit rate," the actual results are still "random."
- Crit rate: The crit rate indicates the probability for the crafter to produce a critical success result for every individual crafting roll. The crit rate does not apply to a total cumulative sample per se. With an 83% crit rate, it is possible to fail a dozen times in a row; the probability of that happening is low, but not impossible. Please note that I conflate the terms "crit rate" and "crit probability" (the latter is accurate) because a lot of players do that, and I have to be consistent with my previous experiment.
- Channel (channeling): A timed act with the blue timer bar (such as certain class skills, or mapping, or in this case creating a single craft item); only gems and tokens can be channeled in the jeweller profession.
- Session: For consistency with my previous experiment, a crafting session is defined here as an uninterrupted series of "make all" channeling. For this jeweller experiment, sessions were mixed into sittings whenever possible to increase the sample size. For practical reasons (limited resources, limited space, etc.), I never did a complete "make all" session that would convert large amounts of material into a single product type.
- Sitting: A sitting is essentially a single recorded log. Each sitting varied in sample size, though I started out aiming for 10 per sitting. In order to prevent too much log contamination, if I started a recorded craft sitting and realized I needed more of something, I would stop recording (which creates a discrete log) and begin logging again only after I got what I needed. This means that for every login, I could have multiple recorded sittings.
- Roll: A "roll" here is defined as a single product channeling act that creates a final product(s), with the RNG applying at the end of the channel.
I do not expect Rise of Isengard to change the actual underlying critical mechanism of crafting. The only changes that I know of apply to the forthcoming Tier 7, and then mainly apply only to the prospecting (gathering and smelting) aspect.
I also assume that this experiment applies to all other crafting professions that take crit ingredients. My reasoning is that, from a programming perspective, using different code for different crit objects in different professions would be inefficient; thus, it would make sense that the underlying code behaves the same way across professions, also taking into consideration that the game is built on top of an existing, generic RNG mechanism as well.
Bottlenecks--salts and gems, but salts were the biggest bottleneck, and I don't have the patience to farm the AH for another 1000 sample. My tinker already had quite a few banked salts and gems because I was not terribly active in her jeweller profession; I might remember to make a few things on a random weekend, basically. But after a certain point, trying to find salts for the tiers where I had an excess of precious metals became a little frustrating.
Basic streak data
Max: 2 streaks of 10 crits
Max: 2 streaks of 5 noncrits (Huh.)
Mediafire link for logs.
THE END. (or is it?)