write a custom HH converter

[sfinktah]

Similarly if someone raises to 3BB and everyone else folds the "extra" 2BB is returned, he doesn't "win" it.

Yup, you can only win what the other person loses. No argument there. The BB wins 0.5BB. That was never at issue.

It's really just an question of semantics. The total amount won never changes.

When everyone folds to the BB the part of his "bet" that wasn't called by the SB is returned, so isn't part of the pot.

Which of the following statements is untrue...
In a heads up, no rake, no limit holdem hand:
[list=]The minimum size of the pot is SB + BB
The loser relinquishes all chips he has placed in the pot
The winner takes all the chips that are in the pot
The winner wins a minumum of SB + BB[/list]

Pursuant to your example of the 3BB raise, lets consider this scenario:

Tighty post a small blind of $1
Lazy posts a big blind of $2
Brokey calls $2 and is all-in
Tighty folds

If the size of the pot in my original example was only $2, (not $3), and if Tighty and Lazy have acted exactly the same (Lazy doesn't actually act in either case), how can the size of the pot be $3 more than it was, even though only $2 has been added to it.


I think I'm starting to understand George Orwell''s 1984 a lot better now.

[WhiteRider]

Which of the following statements is untrue...
In a heads up, no rake, no limit holdem hand:

The minimum size of the pot is SB + BB
The loser relinquishes all chips he has placed in the pot
The winner takes all the chips that are in the pot
The winner wins a minumum of SB + BB

It's semantics, I agree, but techically "The minimum size of the pot is SB + BB" is wrong - the SB amount is called by the BB and the rest is returned, rather than going into the pot.

Pursuant to your example of the 3BB raise, lets consider this scenario:

Tighty post a small blind of $1
Lazy posts a big blind of $2
Brokey calls $2 and is all-in
Tighty folds

If the size of the pot in my original example was only $2, (not $3), and if Tighty and Lazy have acted exactly the same (Lazy doesn't actually act in either case), how can the size of the pot be $3 more than it was, even though only $2 has been added to it.

Here Brokey has called the BB amount, so the whole of the BB is now in the pot, not just the SB part of the BB.
If Brokey wins, the pot is $5, just as it would be if he had raised to $6 and the SB and BB both folded (his extra $4 would be returned in that case).

[sfinktah]

Since support is not forthcoming I will post what I have learned about the "bare minimum" requirements of hand history writing through trial and error.

This is extremely unfair to our support staff who do an outstanding job.

Sir, you have quoted me out of context. You failed to include the text immediately previous to that statement, which was:

With all due respect, RTFM is an acceptable answer. "Read the fracking schema" is not. It's not only unacceptable, it's nonsensical. How is a database schema going to answer this question?

I didn't start this thread. It's over 6 months old. And it was not dealt with correctly at that time. However, now that I am here, I certainly intend to finish the job.

I've written two of the hand history importers since starting here and fixed many bugs with the others, but because we parse completed hand histories we never really think about what we actually use and don't use. Also we do our parsing in three stages since the back-end is obviously reused for every site.

This will also be my second PT3 hand converter, the first can be found on the 2p2 forums. I think in providing a definitive answer to a question that seems to be of interest to multiple customer of yours - both now, and in the past - I have also qualified myself as providing support for your product above and beyond that which was previously available.

Unlike yourselves, I do not get paid for this, and am under not obligation to do so.

Furthermore almost every site has generated bad hand histories at some point in the sites lifetime. That accounts for a lot of the information we ignore or calculate ourselves.

There is no minimal format for any of the sites. We try to take all of the information and use it for cross-checking. However when we detect errors we generally trust our calculations over the hand history ones. When we find discrepancies that can't be accounted for then we throw errors.

The simple answer to all of your questions is that you must provide all of the information that any sites provides because our parsers are expecting that information. We would be extremely happy if sites used hand histories that were easy to parse and were always correct and consistent. Unfortunately that is not the case and we've had to adjust our software to compensate.

[/quote]

I have nothing but respect for the excellent work done by yourself and other programmers. I have thrown the most absurd textual content at your parsers, and it has accepted it gracefully, parsed it correctly, inferred missing information whilst discarding clearly incorrect phrasing, and yet still notified me whenever there was a mistake.

This is the reason I have so much respect.

My issue here, is that the original posters support question was not answered with even the slightest effort, and that "answer" has been read by countless individuals since then, who have likewise given up.

You have carefully worded your statement "There is no minimal format for any of the sites." However, that was not the original question. The question was "what is the minimal format for importing data into PT3."

Can you honestly say that after writing two hand history importers and fixing bugs with many of the others, that you would for even a split second, think that a PokerStars, FTP or PartyPoker history file is even in the same league of complexity compared to the Everest format?

As poker players, these are the HH files that we we see every day. They're all long, tedious, and complication to reproduce. We have no reason to think that that any other sites would provide us with anything different. I've never even heard of Everest before this week.

So we ask you - the professionals - who supply this great product, the product which we all purchased, and would purchase again, and again.
The same product we wish so desperately to integrate with.... we ask you for a tiny piece of information. "How can we most easily get our data into your product."

And now you attempt to convince us that with you wealth of experience, that you don't think it might have been pertinent to point out that the Everest format is less than half the size of a Stars (et al) history file?

Code: Select all

PokerStars Game #19754941443: Hold'em No Limit ($1/$2) - 2008/08/20 - 03:27:52 (ET)
Table 'Penelope II' 9-max Seat #5 is the button
Seat 1: swooshOU ($230.55 in chips)
Seat 2: tjy808 ($58.50 in chips)
Seat 3: madrox187 ($189 in chips)
Seat 4: laststand77 ($222.95 in chips)
Seat 5: m27275469 ($71.50 in chips)
Seat 6: hawk2702 ($171.20 in chips)
Seat 7: Hero ($428.50 in chips)
Seat 8: $imm0n$ ($200 in chips)
Seat 9: Spagiari ($193 in chips)
hawk2702: posts small blind $1
Hero: posts big blind $2
*** HOLE CARDS ***
Dealt to Hero [Jd Jc]
$imm0n$: folds
Spagiari: folds
swooshOU: folds
tjy808: folds
madrox187: folds
laststand77: folds
m27275469: folds
hawk2702: raises $4 to $6
Hero: raises $18 to $24
hawk2702: calls $18
*** FLOP *** [6s 6c 5c]
hawk2702: checks
Hero: bets $38
hawk2702: raises $109.20 to $147.20 and is all-in
Hero: calls $109.20
*** TURN *** [6s 6c 5c] [2s]
*** RIVER *** [6s 6c 5c 2s] [7d]
*** SHOW DOWN ***
hawk2702: shows [5h Ac] (two pair, Sixes and Fives)
Hero: shows [Jd Jc] (two pair, Jacks and Sixes)
Hero collected $339.40 from pot
*** SUMMARY ***
Total pot $342.40 | Rake $3
Board [6s 6c 5c 2s 7d]
Seat 1: swooshOU folded before Flop (didn't bet)
Seat 2: tjy808 folded before Flop (didn't bet)
Seat 3: madrox187 folded before Flop (didn't bet)
Seat 4: laststand77 folded before Flop (didn't bet)
Seat 5: m27275469 (button) folded before Flop (didn't bet)
Seat 6: hawk2702 (small blind) showed [5h Ac] and lost with two pair, Sixes and Fives
Seat 7: Hero (big blind) showed [Jd Jc] and won ($339.40) with two pair, Jacks and Sixes
Seat 8: $imm0n$ folded before Flop (didn't bet)
Seat 9: Spagiari folded before Flop (didn't bet)


Compared to this:

Code: Select all

EverestPoker Game #3528940933: Table Zaysan-8 - $1/$2 - No Limit Hold'em - 11:08:10 - 2008/07/29
Seat 1: RiseOf-era ($379.85)
Seat 2: MarcusAS ($193.35)
Seat 3: iL_MnK ($307.15)
Seat 4: Hero ($92.00)
Seat 5: maimuni ($223.00)
Seat 6: Matti ($220.05)
maimuni posts the small blind of $1.00
Matti posts the big blind of $2.00
The button is in seat #4
*** HOLE CARDS ***
Dealt to Hero [3h 3c]
RiseOf-era raises to $7.00
MarcusAS folds
iL_MnK folds
Hero raises to $24.00
maimuni folds
Matti folds
RiseOf-era calls $17.00
*** FLOP *** [6c 8s 10c]
RiseOf-era checks
Hero bets $35.00
RiseOf-era raises to $156.00
Hero calls $33.00, and is all-in
RiseOf-era shows [Ts 9s]
Hero shows [3h 3c]
*** TURN *** [Ah]
*** RIVER *** [10d]
RiseOf-era shows [Ts 9s]
Hero shows [3h 3c]
*** SUMMARY ***
RiseOf-era shows [Ts 9s]
Hero shows [3h 3c]
RiseOf-era collects $184.00

The was a correct answer to the OPs question. And that answer was "The Everest HH Format."

[sfinktah]

It's semantics, I agree, but techically "The minimum size of the pot is SB + BB" is wrong - the SB amount is called by the BB and the rest is returned, rather than going into the pot.

I think I'm going to have to restrain myself from further debating this, because I can see where this is progressing... I can hear AJ from HSP in my heads excitedly announcing that "if he calls this this bet, this will be the largest pot in tv history."

And I don't think I want to be involved in any debate to include TV as a credible source of fact.

Besides, although your way of thinking (indeed, perhaps the correct way) was initially counter-intuitive to me, upon reflection it is a more consistent and mathematical elegant way of describing the situation.

[sfinktah]

Similarly if someone raises to 3BB and everyone else folds the "extra" 2BB is returned, he doesn't "win" it.

Amusingly enough, the very next situation I find myself debugging.

Again, I find myself intuitively reaching for the wrong answer again... but feel much the wiser since yesterday.

I will post the incorrect hand for future readers... it doesn't hurt to collect all the information on this in one place for future developers. I might selectively edit it the thread when I'm finished, and we could get it added to the FAQ or somesuch.

Invalid pot size/winnings for hand #12339340573806 (calculated pot: $1.25; reported winnings: $0.75) (Line #593)

Code: Select all

EverestPoker Game #123: Table Berkeley - No Limit Hold'em - 09:00:23 - 2010/01/24
Seat 2: Splashy ($6.49)
Seat 4: Foldy ($22.62)
Seat 5: Moldy ($37.87)
Seat 7: Hero ($7.50)
Foldy posts the small blind of $0.25
Hero posts the big blind of $0.50
The button is in seat #4
*** HOLE CARDS ***
Dealt to Hero [8c Ad]
Splashy raises to $6.49, and is all-in
Moldy folds
Foldy folds
Hero folds
*** SUMMARY ***
Splashy collects $0.75


Intuitively I see he has won $0.75, the value of the blinds, since he has placed a considerable raise, and the everyone has folded. I fail to account for the $0.50 of that raise that was used to call the pre-existing bet of $0.50. Something that would never have happened in the old days, when people would have said "I'll see your bet of $0.50 and raise you $5.99."

[zubs1aa]

Since support is not forthcoming I will post what I have learned about the "bare minimum" requirements of hand history writing through trial and error.

This is extremely unfair to our support staff who do an outstanding job. I've written two of the hand history importers since starting here and fixed many bugs with the others, but because we parse completed hand histories we never really think about what we actually use and don't use. Also we do our parsing in three stages since the back-end is obviously reused for every site.

Furthermore almost every site has generated bad hand histories at some point in the sites lifetime. That accounts for a lot of the information we ignore or calculate ourselves.

There is no minimal format for any of the sites. We try to take all of the information and use it for cross-checking. However when we detect errors we generally trust our calculations over the hand history ones. When we find discrepancies that can't be accounted for then we throw errors.

The simple answer to all of your questions is that you must provide all of the information that any sites provides because our parsers are expecting that information. We would be extremely happy if sites used hand histories that were easy to parse and were always correct and consistent. Unfortunately that is not the case and we've had to adjust our software to compensate.

Tarix-

I understand your point. also that of sfink tho.

So if i get this right, you get a HH file in....and then parse it for inputs..do some checks.manipulations.. and then in essence transform the parsed data to PT3 format to store away right?

What sfink is asking for, I think, is (to use everest for example) a list of the elements you are parsing for. i.e. when you see an everest file you look for the following items (x,y,z) and ignore any other content. Assuming his tests were accurate, you for instance could care less about the verbiage of bet raise call.. but only parse for the player and the amount - at least that's my inference.

Knowing what fields you are ultimately populating from the parsing provides the minimal dataset. Or in fact one could call that a shema for an everest hand import.

I was for a while trying to re-write the bodog HH's from your grabbing function to translate them to something else to be able to make them readable as if they were from stars/everest someonthing else. I got as far as parsing the input file and storing all the possible data elements from a hand . However, i got hung up trying to create the output that would *look like* a stars/everest whatever.... Besides it being tricky to do, I didn't know what the likely least I could get away with doing so that PT would read it.

If I knew that PT3 was going to parse the input file and in doing so only look for a given lost of variables/fields.. it would be much much easier - i probably wouldnt have to worry about the words raise/bet call for instance as shown by sfink.

So is it too much to ask for the schema that shows what you are seeking at import? (UNless it is proprietary...)?

I know that my first instinct was to create an xml schema that represented all the elements of an incoming file from site X.. and to then transform it to match a second schema which would represent what PT was seeking. (This probably is the easiest thing to do... but I'm a noob at XML schemas and so I nixed that idea. BUt, whether one goes that route or not... if we had an IMPORT scheam (not a db schema) that'd make it far simpler a job.

Hopefully that makes sense!

p.s. I was working with tny hands.... so if anyone has any everest Tourney hand histories that I could get my hand on to go the simple route here that'd be greatly appreciated!

[tarix]

Can you honestly say that after writing two hand history importers and fixing bugs with many of the others, that you would for even a split second, think that a PokerStars, FTP or PartyPoker history file is even in the same league of complexity compared to the Everest format?

The problem with the Everest format is that it is in XML and the others are in text. While you might think that is easy the IdleMiner team had tons of trouble writing a valid hand history when they first made their hand grabber.

However, that was not the original question. The question was "what is the minimal format for importing data into PT3."

WhiteRider answered this on June 25, 2009. "You will need to convert the hand history files to match exactly the format that one of the supported sites use - if you do that then PT3 will import the hands."

Before all of your research no one else knew how little of the format we parsed. Also, if someday we needed the information you omitted we would always expect it to be there.

I would also like to point out that I personally do not get paid for answering forum posts. Furthermore when I read your updates I'm learning stuff I did not know!

[tarix]

Semantically the format you are using is a PT2 format. The real Everest format looks like this:

Code: Select all

<SESSION time="1256794204" tableName="Riga-39" id="703.44.626" type="ring" money="$" screenName="tokyochecks" game="hold-em" gametype="no-limit"/>
<HAND time="1256794242" id="5167434355" index="3" blinds="$.05/$.10" stakes="$.10/$.10">
  <SEAT position="0" name="tokyochecks" balance="1000"/>
  <SEAT position="1" name="Tsyclon" balance="553"/>
  <SEAT position="2" name="guiboull" balance="285"/>
  <SEAT position="3" name="Marito86" balance="170"/>
  <SEAT position="4" name="GetsugaTensu" balance="1041"/>
  <SEAT position="5" name="ElRulio" balance="941"/>
  <SEAT position="6" name="joelfrishman" balance="1200"/>
  <SEAT position="7" name="txspntr" balance="173"/>
  <SEAT position="8" name="vmsr83" balance="513"/>
  <SEAT position="9" name="parti74" balance="474"/>
  <DEALER position="1"/>
  <BLIND position="2" amount="5" penalty="0"/>
  <XBLIND position="3"/>
  <BLIND position="4" amount="10" penalty="0"/>
  <BLIND position="0" amount="10" penalty="0"/>
  <HOLE position="2">--</HOLE>
  <HOLE position="4">--</HOLE>
  <HOLE position="5">--</HOLE>
  <HOLE position="6">--</HOLE>
  <HOLE position="7">--</HOLE>
  <HOLE position="8">--</HOLE>
  <HOLE position="9">--</HOLE>
  <HOLE position="0">Jd</HOLE>
  <HOLE position="1">--</HOLE>
  <HOLE position="2">--</HOLE>
  <HOLE position="4">--</HOLE>
  <HOLE position="5">--</HOLE>
  <HOLE position="6">--</HOLE>
  <HOLE position="7">--</HOLE>
  <HOLE position="8">--</HOLE>
  <HOLE position="9">--</HOLE>
  <HOLE position="0">9h</HOLE>
  <HOLE position="1">--</HOLE>
  <BET position="5" amount="45"/>
  <FOLD position="6"/>
  <FOLD position="7"/>
  <FOLD position="8"/>
  <FOLD position="9"/>
  <FOLD position="0"/>
  <FOLD position="1"/>
  <FOLD position="2"/>
  <FOLD position="4"/>
  <PUSH position="5" amount="35"/>
  <SWEEP rake="0">35</SWEEP>
  <XSHOW position="5"/>
  <WIN position="5" amount="35" pot="0" potAmount="35"/>
</HAND>
<HAND time="1256794281" id="5167434686" index="4" blinds="$.05/$.10" stakes="$.10/$.10">
  <SEAT position="0" name="tokyochecks" balance="990"/>
  <SEAT position="1" name="Tsyclon" balance="553"/>
  <SEAT position="2" name="guiboull" balance="280"/>
  <SEAT position="4" name="GetsugaTensu" balance="1031"/>
  <SEAT position="5" name="ElRulio" balance="966"/>
  <SEAT position="6" name="joelfrishman" balance="1200"/>
  <SEAT position="7" name="txspntr" balance="173"/>
  <SEAT position="8" name="vmsr83" balance="513"/>
  <SEAT position="9" name="parti74" balance="474"/>
  <DEALER position="2"/>
  <BLIND position="4" amount="5" penalty="0"/>
  <BLIND position="5" amount="10" penalty="0"/>
  <HOLE position="4">--</HOLE>
  <HOLE position="5">--</HOLE>
  <HOLE position="6">--</HOLE>
  <HOLE position="7">--</HOLE>
  <HOLE position="8">--</HOLE>
  <HOLE position="9">--</HOLE>
  <HOLE position="0">Qh</HOLE>
  <HOLE position="1">--</HOLE>
  <HOLE position="2">--</HOLE>
  <HOLE position="4">--</HOLE>
  <HOLE position="5">--</HOLE>
  <HOLE position="6">--</HOLE>
  <HOLE position="7">--</HOLE>
  <HOLE position="8">--</HOLE>
  <HOLE position="9">--</HOLE>
  <HOLE position="0">2d</HOLE>
  <HOLE position="1">--</HOLE>
  <HOLE position="2">--</HOLE>
  <FOLD position="6"/>
  <BET position="7" amount="10"/>
  <BET position="8" amount="10"/>
  <FOLD position="9"/>
  <FOLD position="0"/>
  <FOLD position="1"/>
  <FOLD position="2"/>
  <BET position="4" amount="5"/>
  <BET position="5" amount="0"/>
  <SWEEP rake="2">38</SWEEP>
  <COMMUNITY>3d, 2h, Jc</COMMUNITY>
  <BET position="4" amount="0"/>
  <BET position="5" amount="0"/>
  <BET position="7" amount="0"/>
  <BET position="8" amount="0"/>
  <SWEEP rake="2">38</SWEEP>
  <COMMUNITY>Qs</COMMUNITY>
  <BET position="4" amount="10"/>
  <FOLD position="5"/>
  <FOLD position="7"/>
  <FOLD position="8"/>
  <PUSH position="4" amount="10"/>
  <SWEEP rake="2">38</SWEEP>
  <XSHOW position="4"/>
  <WIN position="4" amount="38" pot="0" potAmount="38"/>
</HAND>
</SESSION>


[sfinktah]

The problem with the Everest format is that it is in XML and the others are in text. While you might think that is easy the IdleMiner team had tons of trouble writing a valid hand history when they first made their hand grabber.

Yes, but they suffer the unfortunate obligation of having to "play by the rules." I just need a result, and I'm not shy about using a few DLL hooks to get it.

However, that was not the original question. The question was "what is the minimal format for importing data into PT3."

Well, technically the actual question was

let's say we play in a room which is not officially supported by PT3.
Is it possible/allowed to self-create an hand history converter so that, given the text file with the HHs, we can import our hands in PT3, even if the room is not supported?
I guess it is.
So the next question, is there some kind of tutorial on how to create such a converter? i.e., which is the correct syntax PT expects to get the text file, maybe an example, et cetera.

WhiteRider answered this on June 25, 2009. "You will need to convert the hand history files to match exactly the format that one of the supported sites use - if you do that then PT3 will import the hands."

And that answer is perfectly correct. And I meant no disrespect to WhiteRider in my replies. I'm sure if he had known of the legacy Everest format, he would have mentioned it. As for the "FAQ" the OP wanted, well - we're putting together some nice work right here.

Before all of your research no one else knew how little of the format we parsed. Also, if someday we needed the information you omitted we would always expect it to be there.

It was never my intent to suggest to others that further reduction of what little data is in that format would be a good idea. It is however quite useful for testing, and developing rapid prototypes.

For the record, expressing every action as a "raises to" line is not completely perfect. It causes a miscount of 1 SB when the SB limps in the pot with "... raises to BB"

I would also like to point out that I personally do not get paid for answering forum posts. Furthermore when I read your updates I'm learning stuff I did not know!

I think I have a solution to that problem. I will PM you.

[grindunumb]

I appreciate all the contributions. Some of this is over my head, but once done I hope I can follow everything. I am a prop for a bunch of small sites, and I would love to be able to review hands and see my stats. Many of the sites are so small, they will likely never be natively supported by any tracker. I hope in the end I could create a few converters since at this point most of my play is on those sites.