SpaceEngine

SpaceEngine

Not enough ratings
GAIA DR3 Catalog Beta 0.10 RC3
   
Award
Favorite
Favorited
Unfavorite
File Size
Posted
Updated
263.666 MB
27 May @ 9:28am
29 Aug @ 11:37am
26 Change Notes ( view )

Subscribe to download
GAIA DR3 Catalog Beta 0.10 RC3

Description
Release Candidate Update:
This catalog is now distance-complete, as de-duped as it can get, and a quick check didn't turn up any obvious misclassifications. For anyone who hasn't been checking the change notes, I have a new discussions post going over the remaining plans for the near-future HERE.

RC3 Update, 09/01:
As part of my efforts to get things release-ready, I've been doing a final pass comparing my assigned values to the historical literature values. For a sample size of the 200 thousand closest stars (because that's SIMBAD's table upload size limit), I got a little over 1000 results that had existing classifications available an analysis shows the following:

Match Strength
Percentage Matched
Exact Match
16.3%
Letter and LumClass Match
31.0%
Letter Match
41.1%

Now, this looks kinda bad on paper, so I was a little worried at first, but other than one specific case involving M-type Subdwarfs that I'll have fixed for the final release it looks like my classifications are actually often more accurate. Either that or the data provided in the Gaia DR3 tables is way off, in which case, not much I can do about that.

A lot of the failed letter matches are generally around the borders, i.e. literature says G9, my classification says K0, which isn't really an issue. I have found a couple legitimate temperature discrepancies, but I'm also currently working on figuring out how to resolve those. Expect one more update in the next couple days, and probably (hopefully) final release within the week.

Overview:

This is the first publicly released beta version of my WIP effort to convert the Gaia DR3 catalog into a SpaceEngine-compatible format. I've been working on this for almost two months now, and finally feel like it's in good enough shape to warrant it.

Things to note:
  • There are 5 million stars in this version of the catalog.
  • This version of the catalog covers the entire galaxy (Or at least the parts that are covered by Gaia).
  • Given the number of stars involved, a lot of the real work is obviously automated, which can lead to some issues, and the DR3 catalog also has a few unique quirks of its own that make it difficult to just pull all the values and transfer them over.
  • This catalog is compatible with most other large catalog addons available on the workshop, but the only version of the original DR2 addon currently compatible is the 1M version due to the exponential increase in time that it takes to filter against that many stars. If you find a smaller addon that isn't compatible, let me know and I can add it easily enough.
  • Don't get too attached to any locations; the star list gets randomized with every update, and this is a beta, so updates will be fairly common.

I have an extensive description of the entire process in the Technical Documentation post in the discussions.

Now, you may be wondering, why start with 5 million when the most popular version of the DR2 addon is the 1 million star pack? The sample size, really. The more stars available, the more potential problem stars there are for people to find. In terms of the equivalent DR2 packs, this is just one step up, so I'm hoping most people's computers will be able to handle it, even if performance isn't very good. That said, even if it's just one "step" up, it's still quintupling the number of stars, so if turns out to be too much, it's easy enough to adjust, and more varieties of packs can be made available later. I reiterate, if people are finding it too hard to run (or believe that they will), let me know in the comments and I can always lower the size.



Potential Issues:

There are three main categories of issues that can be found in this catalog:
  1. Duplicate stars: for reasons I explain in the documentation, SpaceEngine's automatic duplicate merging does not work on this catalog. I did my best to filter them out, but make sure to flag any you spot.
  2. Impossible stars: either due to a model mismatch or just bad data, sometimes stars end up with combinations of values that aren't actually possible. Again, I do my best to filter them out at multiple points in the process, but I can't come up with every possible permutation of logic filters to cover every possible case off the top of my head.
  3. Incorrectly classified stars: these aren't technically always an issue, depending on how much of a variance it is. No astronomical data is ever going to be perfectly accurate at these distances, which is the reason you frequently see a ± beside a lot of values. If the classification is off by a few subclasses, that's to be expected. If it completely drops out of its "accepted" base or luminosity class, then there's a problem. There are plenty of stars with well known values that aren't already included in SpaceEngine's catalogs, so it's easy enough to just search one of the DR3 catalog stars' names in something like SIMBAD[simbad.cds.unistra.fr] and see if there's any good data on it, but manually doing this for every star would be absurd, so if you have a star you like that wasn't already in the SE catalogs, maybe check it to see if it's in the DR3 catalog and let me know if my results were accurate or not.



Progress Report:
08/29/2025—
Current Highest Range Slice: 100,000
Current Number Of Available Stars: 278,684,810
Stars Removed By Filters: 203,935,606
Usable Stars After Filtering: 74,749,204


(The previous GIF format looks real bad at this range, unfortunately.)



Bug Reporting:

My ability to look through the specifications of any given star as a result of my setup means that all I need is a name (or Source ID) and a brief description of what's wrong with it. If you spot any issues, be sure to let me know.



This work has made use of data from the European Space Agency (ESA) mission Gaia (https://www.cosmos.esa.int/gaia), processed by the Gaia Data Processing and Analysis Consortium (DPAC, https://www.cosmos.esa.int/web/gaia/dpac/consortium). Funding for the DPAC has been provided by national institutions, in particular the institutions participating in the Gaia Multilateral Agreement.
Popular Discussions View All (2)
1
11 Jul @ 2:26pm
Technical Documentation
Delta Ori
0
21 Aug @ 6:38am
Road To 1.0
Delta Ori
33 Comments
animationfan13 4 Sep @ 4:49pm 
also btw for the galaxies, i dont think u need to give them all quat, especially the far ones, there is also axis u can use for the galaxies, u can prob use data from NED. also a SDSS catalog mod would be cool, celestia had did it :)
animationfan13 3 Sep @ 6:40pm 
thanks :D
animationfan13 3 Sep @ 6:40pm 
alr ill take a look at it when i got the time
Delta Ori  [author] 3 Sep @ 1:19pm 
It's a bulk file, not a script, otherwise it would've been way harder. And probably run into filesize limits to be honest. Most star catalogs are usually in a format like votable or csv already, so it doesn't take much. All that's really necessary at a bare minimum is making sure the SpaceEngine-required fields are present. 99% of the difficulty I've had with this has been my insistence on scientifically accurate spectral classifications, which aren't included in the Gaia tables and thus need to be determined based on the existing fields first.

Eh...

Technical Documentation Link Again <– Just read that if you're really curious.
animationfan13 3 Sep @ 12:39pm 
also just wondering, but how did u make this (by that I mean converting the star data to a SE script). I thought about porting some Celestia intergalactic star mods to SE, along with adding all the real stars in the Omega Centauri cluster and more
animationfan13 3 Sep @ 12:24pm 
:o ooooooh
Delta Ori  [author] 1 Sep @ 12:42am 
Smaller packs coming soon if it's too much...
bitgen 31 Aug @ 4:09am 
ohhh MYY PC
animationfan13 26 Aug @ 8:49am 
Alr, sounds good, i was trying to figure that out too
Delta Ori  [author] 26 Aug @ 5:08am 
It took me a minute, but I think there's been a misunderstanding. Assuming I'm right, to clarify, I was referring to my earlier comment on bulk galaxies, "Only hard part would be that SE doesn't have bulk galaxy files as far as I know." I've since found out that it comes with one by default, so I know how to format it now, I'd just need to figure out how the "Quat" values work and how to convert other galaxy catalogs to use them so they're SpaceEngine compatible.

I don't actually have a new galaxy catalog to add picked out yet, whenever I get around to doing one.