Statistical Analysis: Relationship Between Tricks and Points

[TigreLXIX]

In reviewing this post from Mick, it struck me as I looked at the image with color-coded cards that a full 75% (15 of 20) of the tricks in this hand were worth 2 points.  This seemed to be at odds with the notion that tricks should be counted as the average point value of 2.4 (sometimes rounded to 2.5) points each.  Granted, the more tricks you take, the more they will tend towards the average - but I wondered if using the average as an estimate is a good idea.  I decided to do a little more analysis.

I have 34 hand histories from various games I've played saved on my computer.  So I entered all the data into Excel and did some analysis of the point values for each trick.  For purposes of this analysis, I decided to ignore the extra 2 points awarded for the last trick - in my opinion, this is a different discussion - so the point values for the last trick were based only on the number of "pointers" (aces, tens, and kings) in the trick.

I should also note that some of the hand histories were abbreviated due to one player TRAM-ing.  In these instances, I went to the point at which TRAM was declared and virtually played out the last few tricks to determine the point values for each of these tricks.  To my recollection, there was only one hand where there was any question as to how the cards would have been played out.  In this hand, there wasn't really a big difference in terms of the points for each trick, and it wouldn't have affected the overall numbers (i.e., it would have been something like 3 points on trick 18 and 2 on trick 19 or 2 points on trick 18 and 3 on trick 19 - the overall number of tricks worth 2 and 3 points would have been the same).

So, here are the results of my analysis:

Total tricks evaluated = 680
Number of tricks with 0 points = 3 (0.4% of total)
Number of tricks with 1 point = 49 (7.2% of total)
Number of tricks with 2 points = 351 (51.6% of total)
Number of tricks with 3 points = 227 (33.4% of total)
Number of tricks with 4 points = 50 (7.4% of total)

Based on this analysis, it seems that assigning a value of 2.4 points per trick would overestimate the value of almost 60% of the tricks (0.4% + 7.2% + 51.6% = 59.2%), while underestimating the value of 40.8% of the tricks.  To me, this is a noteworthy observation - perhaps there needs to be a better way to determine overall point-taking ability of a given hand.  (I don't have the answer, but would love to explore this further.)

I further analyzed the point value of tricks based on the trick sequence.  Here is what I discovered (based again on analysis of 34 hands):

Trick 01 - average value = 2.09 points (< 2.4)
Trick 02 - average value = 2.21 points (< 2.4)
Trick 03 - average value = 2.38 points (< 2.4)
Trick 04 - average value = 2.35 points (< 2.4)
Trick 05 - average value = 2.38 points (< 2.4)
Trick 06 - average value = 2.12 points (< 2.4)
Trick 07 - average value = 2.21 points (< 2.4)
Trick 08 - average value = 2.29 points (< 2.4)
Trick 09 - average value = 2.47 points (> 2.4)
Trick 10 - average value = 2.47 points (> 2.4)
Trick 11 - average value = 2.00 points (< 2.4)
Trick 12 - average value = 2.41 points (> 2.4)
Trick 13 - average value = 2.44 points (> 2.4)
Trick 14 - average value = 2.29 points (< 2.4)
Trick 15 - average value = 2.47 points (> 2.4)
Trick 16 - average value = 2.24 points (< 2.4)
Trick 17 - average value = 2.29 points (< 2.4)
Trick 18 - average value = 2.74 points (> 2.4)
Trick 19 - average value = 3.06 points (> 2.4)
Trick 20 - average value = 3.09 points (> 2.4) (*not including 2 points for last trick)

Based on this analysis, later tricks are generally worth more than earlier tricks.  Therefore, it may not be a good strategy for the Declarer to play all his aces in the beginning of the hand, unless he has reason to do so (long side suit without good ability to control trump, mid-length side suit where aces may be forced out and trumped, or short side suit where aces may get caught "hanging").

What other observations do you have from this analysis?

(If you're interested in the Excel data - which includes just the point values for each trick and none of the other game or hand information - send me a private message with your e-mail address.)

[mickmackusa]

These observations are definitely relevant.
They give some confirmation to my secret lack of appreciation for the popular 2.4 point/trick ratio.
I would like to hear from others that insights on this topic.

My gut tells me that when we are estimating "trick:points":

• Non-trump Aces should be using 1:2
  
• Trumps might be closer to 1:2.25
  
• Other Winners might be closer to 1:3
  

Maybe we could refine how we estimate trick:points for the Trump portion of a hand.
Maybe the Aces are each worth 2, and remaining trump after Aces and Losers have been assume would be worth 2.5?

I look forward to continued discussion, because I think the jury is very much out on this topic.

Perhaps we can iron this out well enough to create an Expert's Hand Classifier!  
Tigre & Marya, we might need a larger sample size for crunching stats.
Or maybe a larger competition pool, PlayOK's hand histories are sufficiently formatted to handle this type of processing.  I have a small stockpile of games from there.

[mickmackusa]

No no no, wait.  We can't simply use any random hand histories.  We can only use select hand histories where all players make the best, most reasonable choices and make no glaring mistakes.

Individual hands will need to be screened for "correctness" before they qualify for statistical analysis.

I realize what I am saying is a somewhat grey/subjective process.
I think it will be easier to pinpoint which plays are "wrong" than which plays are "right".
Some plays are neither right nor wrong.
We may have to debate on a case by case basis.

Let's get another hand history, judge the Play phase and come to an agreement on if it is or is not a contender for analysis.

[TigreLXIX]

(03-26-2016, 05:48 AM)mickmackusa Wrote:  No no no, wait.  We can't simply use any random hand histories.  We can only use select hand histories where all players make the best, most reasonable choices and make no glaring mistakes.

Individual hands will need to be screened for "correctness" before they qualify for statistical analysis.

I realize what I am saying is a somewhat grey/subjective process.
I think it will be easier to pinpoint which plays are "wrong" than which plays are "right".
Some plays are neither right nor wrong.
We may have to debate on a case by case basis.

Let's get another hand history, judge the Play phase and come to an agreement on if it is or is not a contender for analysis.

Hmm...I think you're right, but I'm afraid it is too much of a manual process to validate a large number of hands (enough to perform a valid statistical analysis on). Maybe we should put this on the "side burner" until we have the Bot competition - then the best Bots can play a very large number of hands, and we will have good data that won't need to be validated.

[marya]

Mick said "No no no, wait.  We can't simply use any random hand histories.  We can only use select hand histories where all players make the best, most reasonable choices and make no glaring mistakes"

That might be true. It would be interesting to run bots vs themselves and record the stats for how much each trick is worth over a large sample. Then you could compare with random bots (that just played random cards, with the only requirement that they stick to the rules). I would expect to see a difference in the points vs trick number between the 2 samples, but I'm not sure.

I feel kind of bad that you went through all the work with a spreadsheet Tigre... not sure if it was fun   but I probably could add some code to do something similar at World of Card Games. It would be an aggregation over the playing styles of many, many different types of players. You'd get raw beginners mixed in with expert players mixed in with bots.

I run bot vs bot tests myself, when testing, so I could run stats over play between my bots if anyone is interested. Not sure if this excercise would be fruitful, however.

Before I make the effort to code anything up, though, I'd want to see a general discussion from interested parties about what statistics would be interesting to record. Then I can see how much work it takes to put them in place. I love doing experiments like this, but my time is limited, so I want to make sure that any experiments that I run will be worth it.

What Tigre did was to compute:

1) Number of tricks worth N points

2) Average trick value as a function of trick number

Who (if anyone) would like to see these stats for a larger data set? Any other stats that seem more relevant?

[rdwrites]

While not digressing from marya's questions, I read Tigre's post this morning and took a look at a test game from my site (that I'm testing internally) to give some additional input. I was curious about this trick counting thing (apparently similar to Spades bidding), and also how my Quality Count based bidding compares.

After taking a look at this, the first conclusion I drew is that 2.4 is an average (48/20), and has nothing to do with what tricks might actually pull. But I crunched the numbers to see what would come of it.

The next thing I found is that my numbers matched Tigre's. (Nice thought experiment, Tigre.)
Total tricks evaluated = 180
Number of tricks with 0 points = 0 (0.0% of total)
Number of tricks with 1 point = 12 (6.6% of total)
Number of tricks with 2 points = 93 (51.6% of total)
Number of tricks with 3 points = 66 (36.6% of total)
Number of tricks with 4 points = 9 (5% of total)

My interest is more in how well it predicts the winning bidder's hand though, so I looked at just those:
Total tricks evaluated = 109
Number of tricks with 0 points = 0 (0.0% of total)
Number of tricks with 1 point = 8 (7% of total)
Number of tricks with 2 points = 60 (55% of total)
Number of tricks with 3 points = 33 (30% of total)
Number of tricks with 4 points = 7 (6.4% of total)

And found they were similar. By the way, there were no sets in this game.
Continuing on with analysis of the Declarer's team tricks:
Average tricks taken: 12
times 2.4 = 29
last trick taken 67% of time for 12 points
average points taken: 30

[rdwrites]

I'll post these predictors vs actual results. My interest for QC is total points but I compared the tricks to counters pulled.

First the 2.4 results:
Aces Trump tricks 2,4 counters
   4       7        12   29     29
   5       7        16   38     39
   5       6        10   24     32
   4       7        10   24     24
   7       9        15   36     34
   5       9        13   31     31
   4       8        12   29     29
   5       6          8   19     18
   3       8         10  24     24

Does very, very well. The trick is figuring out the tricks.

This is results of my QC method:
Aces Trump QC points
   4       7      22    29
   5       7      24    41
   5       6      22    30
   4       7      22    24
   7       9      32    36
   5       9      28    33
   4       8      24    31
   5       6      22    20
   3       8      22    24

There is upside for a human if they are very very good at predicting tricks taken to bid 5 to 15 points higher than Quality Count method. Also of course lots of upside to wishfully bid too high.
But this is an area where the best humans have an advantage over the bot, and I wouldn't want it any different.

[TigreLXIX]

(03-26-2016, 03:55 PM)marya Wrote:  I feel kind of bad that you went through all the work with a spreadsheet Tigre... not sure if it was fun

You shouldn't feel bad at all, and of course it was fun! I'm a bona fide geek and love messing with data in Excel. It's actually nice to use Excel to crunch some numbers outside of my normal job.

(03-26-2016, 03:55 PM)marya Wrote:  What Tigre did was to compute:

1) Number of tricks worth N points

2) Average trick value as a function of trick number

Who (if anyone) would like to see these stats for a larger data set? Any other stats that seem more relevant?

I would definitely like to see more data on the first 2 items. I would also like to see (as rdwrites began to do) an analysis of the "expected trick count" vs. actual tricks taken by the Declarer.

[rakbeater]

ToreadorElder and I always disagreed on this. He was for a strong and fast rule of 2.5 per trick. I said that the average was 2.4 points per trick, but I never counted 2.4 points per trick, because it always depended on how many tricks you would take. Obviously, the more tricks you take in a hand, the closer to the average of 2.4/ppt you could get.

According to TE's rule of 2.5/ppt, 8 tricks won would allow you to save (20 points), but that was not always the case (unless you got the last trick). I found that 9 tricks was the number that would allow you to consistently save.

I like where this thread is going, because the refining of my bidding system is dependent on these numbers. I'm looking forward to the remainder of this research/discussion!

[rdwrites]

My thought was that 2.5 is the average of all the points, including last trick (50/20). In my sample bid winning team got last trick 2 out 3 hands. Predicting number of tricks is hard enough, predicting last trick is in there with can I run this hand out, yet even if not predictable more likely you will get it than not if you call your suit trump.

I had one hand in the test game as you can see in the stats I posted where just saved with 20 and needed last trick to do it. But the prediction was that it could be saved. (I think it was bot team that did it vs me and bot partner).