Upgrade to Chess.com Premium!

Convert Mega2009 to PGN?


  • 5 years ago · Quote · #1

    chessoholicalien

    Hi,

    Has anyone who has CB Mega Database 2009 managed to convert the entire thing into a single PGN file?

    I've tried 2 methods with CB10 - export to PGN and create new PGN +copy/paste - and both crash after about 60% of the export or import process. Export to text file doesn't work either, as it does not create proper PGN text.

    I've tried Chess Assistant Lite it also can't handle making the PGN.

    SCID can't do it either.

    Notepad and Wordpad can't handle the quantity of text, so I can't use them to copy/paste.

    I managed to make two PGNs containing 50% of the Mega Database 2009 each. So I need to combine them. I've tried every PGN utility I can find online, but they either don't work or they are DOS-based and I can't figure out how to use them. Using the command prompt and doing *.pgn > <new file name>.pgn produces a file which cannot be read by ChessBase or properly by SCID.

    The merge PGN utility provided on this site under Downloads also does not work.

    Can anyone either help me produce a single PGN file or at least correctly merge the two halves that I have?

    Cheers!

  • 5 years ago · Quote · #2

    goldendog

    [COMMENT DELETED]
  • 5 years ago · Quote · #3

    rooperi

    For a text editor, you can download (free) Edit Pad Lite, I dont think it has a size restriction.

    http://www.editpadpro.com/editpadlite.html.

  • 5 years ago · Quote · #4

    chessoholicalien

    Thanks guys.

    The OpenOffice didn't work. I did however find an excellent text editor that opened these huge text files very quickly: EmEditor. Unfortunately Windows couldn't handle copying the 56 million lines to the clipboard...

    Eventually I found a file merge utility called Simple File Joiner which merged the two PGNs perfectly. However, the single file is still not recognized as a valid file by ChessBase10 and SCID crashed when trying to open it.

    My reason for getting a large PGN is so I could do things in SCID that ChessBase can't do. For some reason SCID only reads PGN files and not CBH files.

    I can get SCID to open a PGN database of around 2 million games. So that's better than nothing I guess :-)

  • 5 years ago · Quote · #5

    chessoholicalien

    So I got a mail from ChessBase and they told me that PGNs are limited by size, so I guess my proposed 4-million-game PGN breached that limit.

    I'm now working on making my Mega2009 database even bigger. Will come back on that.

  • 5 years ago · Quote · #6

    chessoholicalien

    I like the player reports better. And the pieces. And auto-replay. So far :-)

  • 5 years ago · Quote · #7

    chessoholicalien

    Thanks to a member of this site I have got ChessBase Mega2009 converted in its entirety into a single SCID database - yay!! :-)

  • 5 years ago · Quote · #8

    chessoholicalien

    Sure.

    1) In ChessBase, make 2 equal-sized PGNs out of Mega2009 (then use the Check Integrity tool to make sure they are sound).

    2) Close ChessBase. Open SCID.

    3) Go to: Windows > (open) Database Switcher

    4) Open part 1 of your Mega2009 PGN.

    5) Make a new SCID database (.si4 file) and name it e.g. Megascid.

    6) Drag-and-drop Mega2009 part 1 into Megascid.

    7) Then from Database Switcher right-click on Mega2009 part 1 and close it.

    8) Open part 2 of your Mega2009 PGN.

    9) Drag-and-drop Mega2009 part 2 into Megascid.

    10) Then from Database Switcher right-click on Mega2009 part 2 and close it.

    11) Close the Database Switcher. You should now have Mega2009 as a single SCID database file! (file name: "megascid.si4")

    I have found SCID searches in Mega2009 much faster than ChessBase10 does.

  • 5 years ago · Quote · #9

    Raketonosets

    Is this PGNed database available for download somewhere?

  • 5 years ago · Quote · #10

    chessoholicalien

    Raketonosets wrote:

    Is this PGNed database available for download somewhere?


    Nope. You need to buy Mega2009 database from ChessBase first ;-)

  • 5 years ago · Quote · #11

    Raketonosets

    chessoholicalien wrote:
    Raketonosets wrote:

    Is this PGNed database available for download somewhere?


    Nope. You need to buy Mega2009 database from ChessBase first ;-)


    I am not so sure that this is correct.

    The database itself (the dataformat and its order) are undoubtedly covered by copyrights, but the information contained within the database itself is not - it is in the public domain.  The status of the collection of games is also probably of one in the public domain since it is largely composed of publicly available games.  I believe that If somebody therefore takes the Mega2009 database and exports it into the PGN format, he/she could safely make it pubicly available for download (if only because it would be impossible to prove that this collection of games was exported from Mega2009 or any other database Tongue out)

    Now, I understand that there are corporations out there who would want to sqeeze every last penny out of each one of us to maximze their profits, but we don't really need to buy into their arguments.

    Besides, corporations need to come to terms with the reality that we are living in an Internet-connected world and start to revise their business models. This guy for example has, I think, a much smarter business model:

    http://www.chessgameslinks.lars-balzer.info/data-information.html

    He offers the service of a DVD with all the databases, but also makes all the databases available online.

    As for Mega2009 is already available for download on the web anyway Wink.

    Considering all this, I would say that converting the Mega2009 into PGN (or, even better, the SCID4.0 format) and making it available to the rest of the chess players out there - in particular those who could not affod to pay such money to begin with! - is a terrific idea, no?

    RN

    PS: some good readings

    http://www.chesshistory.com/winter/extra/copyright.html
    http://www.chessvibes.com/columns/copyright-on-chess-moves-shadows-on-the-wall/
    http://www.chesscafe.com/text/kibitz17.txt

  • 5 years ago · Quote · #12

    Raketonosets

    @rigamagician: I have to admit that I did not check whether all of Lars databases were online Embarassed.  However, I do presume that he does not make a claim to the ownership of those now defunct missing from his website.  The point I was trying to make is that I think that charing users for a service (a DVD) or a technoloy (a database structure) is legally speaking legit (morally - I think that nothing should prevent people from sharing anything and everything they possibly can share, but that's me).  In an ideal world I wish CB made their full collection available for download each year for free as a sign of support and as a service to the community. I would gladly voluntarily donate a litle something to keep this service going on each year.  What I do *not* like to hear is that I *have* to pay before being able to access this info.  I would not even agree to pay 1 symbolic dollar for Megabase, on principle (but that is me, again, I am a little weird I have to admit Wink).

    @Gonnosuke: has CB actually paid each player for his/her annotations or have they collected them from public sources?

  • 5 years ago · Quote · #13

    Raketonosets

    rigamagician wrote:

    Chessbase pays grandmasters such as Robert Huebner for their annotations.  The value added to Megabase also comes from their standardization of names and such. 


    Fair enough.  Point take.  Thanks for these explanations!

  • 5 years ago · Quote · #14

    pentagram

    rigamagician wrote:

    Chessbase pays grandmasters such as Robert Huebner for their annotations.  The value added to Megabase also comes from their standardization of names and such.  There are a number of large database archives available for free: Jose, Chess Analysis Project, Icofybase, ChessDB, 1.74 Million Base, etc.


    Can you please expand more on the Chess Analysis Project? I am familiar with the other bases but not with that one.

    Is the analysis based on collected comments by analysts or is it an engine dump?

    I don't like engine dumps, imho the most useful annotation is that by a good human analyst with many comments like x is also interesting y variation is unclear etc.

  • 5 years ago · Quote · #15

    pentagram

    Gonnosuke wrote:
    pentagram wrote:
    rigamagician wrote:

    Chessbase pays grandmasters such as Robert Huebner for their annotations.  The value added to Megabase also comes from their standardization of names and such.  There are a number of large database archives available for free: Jose, Chess Analysis Project, Icofybase, ChessDB, 1.74 Million Base, etc.


    Can you please expand more on the Chess Analysis Project? I am familiar with the other bases but not with that one.

    Is the analysis based on collected comments by analysts or is it an engine dump?

    I don't like engine dumps, imho the most useful annotation is that by a good human analyst with many comments like x is also interesting y variation is unclear etc.


    It's an engine dump with limited value in my opinion.  Basically, they evaluate well known opening positions (i.e. ECO) and assign them a centipawn value based on a deep depth eval (d=21, I *think*).  It's a useful approach to dry openings but falls short when it comes to dynamic openings where the eval is often at odds with well established opening theory.


    My problem with that approach is that an engine simply cannot find all the ideas & plans tried by humans during the last 60 years in a given position during the x seconds it is evaluating the position. Furthermore I doubt in general that engines can spot deep plans no matter how much time they run, like e.g. the Karpov-Nunn plan in Be2 Najdorf (Q-side pawn advance).

     Also, again imho, a centipawn value tells very little about the position unless is is like +2.00 or -2.00.

    However these are the sort of games that have been immensely useful in my opening preparation, looking at engine centipawn vals instead of following the great variety of plans humans have tried is like castrating imagination in opening play.

    To add on top of that, I am pretty disappointed by informant's laconic annotations, I wonder what sort of publication is best for some good *human* analysis in recent games (not a personal opinion but I have heard people complaining that nowadays NIC is also an engine dump). If you have any suggestions I'm all ears.

  • 5 years ago · Quote · #16

    pentagram

    rigamagician wrote:

    I was referring to the Chess Analysis Project mainly as a huge source of PGN games.  I agree that the assessments are pretty subjective to say the least.  If you don't like Informant and New In Chess, it's hard to make constructive suggestions.


    I don't like informant, I am not a subscriber to NIC, I remember it being very good when reading it in the past (my coach was a subscriber and he was giving me issues (Ybooks) for a read a long time ago. What I have heard about NIC now is only from friends comments/random comments on the internet.

  • 5 years ago · Quote · #17

    pentagram

    rigamagician wrote:

    New In Chess used to offer to send anyone who was interested a free copy of their latest issue.  Not sure if they still do that, but if they do, it would give you a sense of what the quality is like.  I can't think of any other publications that publishes such lengthy and detailed annotations by top grandmasters (other than Informant).  The other places you could try are ICC's Chess.FM or Chessbase Magazine/Playchess Radio, but if you are unhappy with New In Chess, I doubt you'll like them much either.


    My problem with the informant is that the analysis sometimes skips alternatives, that is it is not exhaustive, sometimes the analyst just gives e.g. 13. ..Ne7!? without saying something more. Now to a GM this ..Ne7!? may be enough to make a whole game preparation but I need something more than that.

    I will check myself NIC issues to have personal opinion (as I said from the start the comments on NIC have been passed to me by others), how is the chessbase magazine?

    What I don't want is to spend my money inefficiently, I would be very disappointed to subscribe somewhere and get a Rybka dump. It may sound as a weird customer attitude but that's what I do with everything, be it software, restaurants, clothes or houses. I can understand that quality may be expensive and I am happy to pay for that, what I hate is spending money for something that turns out to not worth its price. TLDR I seek "value for money" :)

  • 5 years ago · Quote · #18

    pentagram

    Gonnosuke wrote:
    pentagram wrote:

    To add on top of that, I am pretty disappointed by informant's laconic annotations, I wonder what sort of publication is best for some good *human* analysis in recent games (not a personal opinion but I have heard people complaining that nowadays NIC is also an engine dump). If you have any suggestions I'm all ears.


    Chesspub is the answer you're looking for.  In the future, sites like Chesspub will be the primary source of opening information.  Informant and NIC are out of date before the ink is dry and sites like Chesspub are both far more topical and far more democratic.  By democratic I mean that they're willing to consider analysis from all sources, not just titled players.  The analysis is judged based on the merits of the analysis not on the title (or lack thereof) of the person who authored it.  The GM's who act as gatekeepers are also extremely accessible.  If you have a question, they're an email away and for the most part they're quite responsive.  Well worth the money in my opinion.

    For openings the King's Gambit they're already the definitive source of information.


    That sounds interesting, can I ask you how does it do with regard to the following:

    - breadth of coverage, e.g. if an interesting line appears in NIC will it be discussed in chesspub?

    - do they analyse recent game novelties? e.g. would it be a good place to discuss e.g. the latest Morozevich-Kramnik game?

     If I have understood chesspub organization correctly, there are the forums & the paid sections. Do GMs reply only if you have sunscribed to the corresponding paid section?

    Only thing I am not 100% fond of is that payed sections in chesspubliishing.com are organized in a way that you may have to subscribe to many, even if you are interested in only a 10% of their content, e.g. as a White player the KID section is useful only if your pet line is being discussed.

    ps: I no longer play KG as I decided to try and learn 1.d4 but who knows :) romance may not be dead ;)

  • 5 years ago · Quote · #19

    pentagram

    rigamagician wrote:

    New In Chess has always struck me as the most readable and human of the top quality magazines.  The GMs are chatty, and describe the psychology and ideas behind their approach.  Perhaps game collection books offer more detailed annotations, but I think it would be hard to find a better magazine for high quality annotations by top SuperGMs.

    Some of the annotators for Chessbase Magazine are major engine heads, but I think it depends a lot on the GM doing the annotating.  There tend to be a lot of lower level GMs, especially from Germany, and I find myself disagreeing with a lot of the assessments, but Chessbase does put a lot of work into making the issues interesting with little video interviews with Anand or whoever, and I usually find something of interest in each issue.


    That sounds like a good reason to get NIC. I remember it like that, but again, no-one was relying on engines then, they were very weak so people switched off their brain & switched on Fritz allot less ;) I'm happy to hear it still provides real GM analysis.

  • 5 years ago · Quote · #20

    Raketonosets

    Gonnosuke wrote:
    pentagram wrote:

    To add on top of that, I am pretty disappointed by informant's laconic annotations, I wonder what sort of publication is best for some good *human* analysis in recent games (not a personal opinion but I have heard people complaining that nowadays NIC is also an engine dump). If you have any suggestions I'm all ears.


    Chesspub is the answer you're looking for.  In the future, sites like Chesspub will be the primary source of opening information.  Informant and NIC are out of date before the ink is dry and sites like Chesspub are both far more topical and far more democratic.  By democratic I mean that they're willing to consider analysis from all sources, not just titled players.  The analysis is judged based on the merits of the analysis not on the title (or lack thereof) of the person who authored it.  The GM's who act as gatekeepers are also extremely accessible.  If you have a question, they're an email away and for the most part they're quite responsive.  Well worth the money in my opinion.

    For openings the King's Gambit they're already the definitive source of information.


    I suppose you are referring to http://www.chesspub.com, right? (sorry, I am new to the chess databases world).  They don't have an "about" on their website.  That is just a forum specialized in chess openings, right?  UndecidedEmbarassed


Back to Top

Post your reply: