spoonitnow's 5000th Post: Game Theory and Poker

Optimal Exploitative Value Betting

Optimal exploitative ________ is a game theory term. It essentially means that we are making every play, in this case a value bet, that is not -EV. We're going to use a slightly different game that uses T cards instead of 13 so we can generalize the solution. This is nice because it doesn't change the proof at all.

Let Villain always call a value bet with n hands such that the worst hand he calls with is h_n, and that he never folds a hand that beats h_n. Then the EV of betting h_k such that 1 <= k <= n is:

(-1)(k-1)/(T-1) + (1+P)(n-k)/(T-1) + (P)(T-n)/(T-1)

And the EV of checking h_k is:

(P)(T-k)/(T-1)

The the EV of betting h_k is greater than or equal to the EV of checking h_k when:

(-1)(k-1)/(T-1) + (1+P)(n-k)/(T-1) + (P)(T-n)/(T-1) >= (P)(T-k)/(T-1)

Now we multiply through by (T-1) to get:

(-1)(k-1) + (1+P)(n-k) + (P)(T-n) >= (P)(T-k)
-k + 1 + n - k + nP - kP + PT - nP >= PT - kP
-2k + 1 + n >= 0
1 + n >= 2k
(1+n)/2 >= k

So we should bet all hands h_k such that k <= (1+n)/2.

This is a fancy-looking way to say that we should bet when we have 50% equity or more against our opponent's calling range. For example, what if our opponent calls with a T or better:

A K Q T J
1 2 3 4 5

Then n = 5, and we should bet all hands such that k <= (1+5)/2 = 3. So we should bet all hands h_k such that k <= 3, which means we're betting an Ace, King, or Queen.

Conceptually

When we're value betting the optimal exploitative frequency, then by definition we open ourselves up to adjustments. The basic adjustment our opponent could make to lower the EV of our value bets is to call less. Our two basic adjustments to protect ourselves from adjustments are either to value bet fewer hands, or to value bet more hands. Each is -EV for our range in a vacuum, and are -EV long term if our opponent never adjusts correctly.

By value betting so much, we have to bluff some non-zero amount to make life difficult on our opponent. You could do the same type of analysis above to determine the optimal exploitative bluffing frequencies, but I'm going to leave that to the reader to work out.

Originally Posted by spoonitnow

By value betting so much, we have to bluff some non-zero amount to make life difficult on our opponent. You could do the same type of analysis above to determine the optimal exploitative bluffing frequencies, but I'm going to leave that to the reader to work out.

Thanks for all of the info in this thread, this is fantastic. Here's my shot at figuring out the optimal expoitative bluffing frequency.

I'll use "j" to represent the opposite numbering of hands that Spoon used for "k": j(1) is the 2 card, J(2) is the 3 card, etc. We're assuming that this is a pure bluff, so if we are called then we lose, therefore 1 <= j <= (T-n+1).

The EV of checking h_j is:

(P)(j-1) / (T-1)

The EV of betting h_j is:

(P)(T-n-1)/(T-1) - (n)/(T-1)

We want the EV of betting to be greater than or equal to checking, so:

(P)(j-1)/(T-1) <= (P)(T-n-1)/(T-1) - (n)/(T-1)
j - 1 <= T - n - 1 - n/P
j <= T - n - n/P

In Spoon's example of the villain that calls with any T or higher, assuming the pot size is 2:

j <= 13 - 5 - 5/2
j <= 5.5

So we can bluff with any 2,3,4,5,6.

What's interesting (and hopefully was already intuitively understood before reading this far) is that Spoon's equation for value betting is only a function of the villain's calling range. But our bluffing equation is both a function of the villain's folding frequency and the size of the pot.

Or, in no-limit terms, your optimal exploitative bluffing range should be determined from the size of your bluff bet (relative to the pot) and how often the villain will fold to that bet.

Originally Posted by spoonitnow

Optimal Exploitative Value Betting
Let Villain always call a value bet with n hands such that the worst hand he calls with is h_n, and that he never folds a hand that beats h_n. Then the EV of betting h_k such that 1 <= k <= n is:

...

(1+n)/2 >= k

So we should bet all hands h_k such that k >= (1+n)/2.

There is a problem here where you invert your inequality, it should be k <= (1+n)/2

Villain has n hands in his range and h_n is the worst, so h_1 is the best, and in your example, we should bet h_k with k >= 3 so QKA (not k <= 3)

So on the same basis, EV of bluffing with h_k and n < k <= T:
P(T-n-1)/(T-1) - n/(T-1)

EV of checking is the same as before:
P(T-k)/(T-1)

So bluffing is better or equivalent to checking when:
P(T-n-1)/(T-1) - n/(T-1) >= P(T-k)/(T-1)
PT-Pn-P-n >= PT - Pk
k >= n+1+n/P

So in your example, with a deck of 13 cards and villain calls with T+ (n=5), if the pot size is:
P=1: k >= 11. We can bluff profitably with h_11, h_12 and h_13 or 4, 3 and 2 (although betting 4 is equivalent to checking it EV-wise)
P=2: k >= 8.5. We can bluff profitably with {h_9-h_13} or 6-2.

Originally Posted by daviddem

There is a problem here where you invert your inequality, it should be k <= (1+n)/2

Yeah I caught it later in the text file I'd wrote all that out in but never changed it in my post, thanks.

Optimal Exploitative Bluffing

Through a similar process as above, we determine that for a deck of T cards, Hero bluffing a hand n < k <= T is better than checking when:

k >= (n/P) + (n+1)

Interpreting this result is very important and gives a lot of insight into when we should be bluffing and when we should be checking down for showdown value in scenarios like this.

As Regards the Fundamental Theorem of Poker

Note that h_n+1 is the best hand that Villain does not call with. Also not that because P > 0 that n/P > 0, so we have k >= n+1, and note that we can never have k = n+1. What this means is that we will never be betting with h_n+1 because it's never better than checking. Note that Villain can never call with worse or fold better if we bet h_n+1.

When the Pot is Very Large

When the pot is large, intuitively we should be bluffing more often. When P is large, n/P is very small. As P approaches infinity, n/P approaches zero, and we have k >= n+1, meaning we'll be bluffing every hand worse than h_n+1.

When the Pot is Very Small

When the pot is small, intuitively we should be bluffing less often. When P is small, n/P is very small. As P approaches zero, n/P approaches infinity, and we have k >= infinity, meaning that we'll never be bluffing.

The Total Bluffing Range (Villain calls T+)

Let's take a look at the scenario in the 13-card game where our opponent is calling a bet with T+. In this game, T = 13 and n = 5:

Code:

	A		1		
	K		2		
	Q		3		
	J		4		
	T		5		
	9		6		
	8		7		P >= 5/1 = 5.00
	7		8		P >= 5/2 = 2.50
	6		9		P >= 5/3 = 1.67
	5		10		P >= 5/4 = 1.25
	4		11		P >= 5/5 = 1.00
	3		12		P >= 5/6 = 0.83
	2		13		P >= 5/7 = 0.71

The first column is the card, the second column is the k-value for each card h_k, and the third column is for what values of P we can bluff. So for betting an 8, which makes k = 7, we need P >= 5/2.

So for example, if P = 1.1, then the optimal exploitative bluffing range is betting a 4, a 3, and a 2.

As the pot size increases, it makes sense that we'll be bluffing with more hands since the reward is larger for the same risk. Similarly, as the pot size decreases, it makes sense that we'll be bluffing with fewer hands since the reward is smaller for the same risk.

The [0, 1] Half-Street Fixed-Limit Game

Each player, Hero and Villain, are given a random real number between 0 and 1, inclusive, and this serves as their hand. The player with the lowest number at showdown wins.

Hero is in position with Villain out of position. Each player posts an ante of P/2 before the hands are dealt. Each player receives their hand, Villain is forced to check in the dark. Hero then has the choice to bet 1 unit, or check to showdown. If Hero bets, then Villain has the choice to call and see a showdown or fold. At showdown, whoever has the highest card wins.

This is similar to the n-card game, but is much easier to analyze, and is much easier to adapt to "real" poker river situations. We'll be using intervals and interval notation to represent ranges. For example, [0, 0.3) would mean that a range consists of all numbers from 0 to 0.3, including 0 but not including 0.3. We'll also be using U to denote a union of two intervals, so [0, 0.2] U [0.9, 1] would mean all numbers from 0 to 0.2 and 0.9 to 1, including 0, 0.2, 0.9, and 1. For more information, see Interval (mathematics) - Wikipedia, the free encyclopedia.

Here are some questions to get us started:

1. If Villain calls with a 0.4 or lower, what hands can Hero make a +EV value bet with?
2. If each ante is 0.25 units, what is the minimal range Villain has to call a bet with so that Hero can't make a +EV bluff?
3. If Villain calls with 0.3 or lower and P = 1, what is Hero's optimal exploitative betting range?
4. What is Villain's optimal exploitative calling range against Hero's range from question 3?
5. What is Villain's unexploitable calling range (in terms of P)?
6. What is Hero's unexploitable betting range (in terms of P)?