SpaceEngine

SpaceEngine

Not enough ratings
GAIA DR3 Catalog Beta 0.10 RC3
   
Award
Favorite
Favorited
Unfavorite
File Size
Posted
Updated
263.666 MB
27 May @ 9:28am
29 Aug @ 11:37am
26 Change Notes ( view )

Subscribe to download
GAIA DR3 Catalog Beta 0.10 RC3

Description
Release Candidate Update:
This catalog is now distance-complete, as de-duped as it can get, and a quick check didn't turn up any obvious misclassifications. For anyone who hasn't been checking the change notes, I have a new discussions post going over the remaining plans for the near-future HERE.

RC3 Update, 09/01:
As part of my efforts to get things release-ready, I've been doing a final pass comparing my assigned values to the historical literature values. For a sample size of the 200 thousand closest stars (because that's SIMBAD's table upload size limit), I got a little over 1000 results that had existing classifications available an analysis shows the following:

Match Strength
Percentage Matched
Exact Match
16.3%
Letter and LumClass Match
31.0%
Letter Match
41.1%

Now, this looks kinda bad on paper, so I was a little worried at first, but other than one specific case involving M-type Subdwarfs that I'll have fixed for the final release it looks like my classifications are actually often more accurate. Either that or the data provided in the Gaia DR3 tables is way off, in which case, not much I can do about that.

A lot of the failed letter matches are generally around the borders, i.e. literature says G9, my classification says K0, which isn't really an issue. I have found a couple legitimate temperature discrepancies, but I'm also currently working on figuring out how to resolve those. Expect one more update in the next couple days, and probably (hopefully) final release within the week.

Overview:

This is the first publicly released beta version of my WIP effort to convert the Gaia DR3 catalog into a SpaceEngine-compatible format. I've been working on this for almost two months now, and finally feel like it's in good enough shape to warrant it.

Things to note:
  • There are 5 million stars in this version of the catalog.
  • This version of the catalog covers the entire galaxy (Or at least the parts that are covered by Gaia).
  • Given the number of stars involved, a lot of the real work is obviously automated, which can lead to some issues, and the DR3 catalog also has a few unique quirks of its own that make it difficult to just pull all the values and transfer them over.
  • This catalog is compatible with most other large catalog addons available on the workshop, but the only version of the original DR2 addon currently compatible is the 1M version due to the exponential increase in time that it takes to filter against that many stars. If you find a smaller addon that isn't compatible, let me know and I can add it easily enough.
  • Don't get too attached to any locations; the star list gets randomized with every update, and this is a beta, so updates will be fairly common.

I have an extensive description of the entire process in the Technical Documentation post in the discussions.

Now, you may be wondering, why start with 5 million when the most popular version of the DR2 addon is the 1 million star pack? The sample size, really. The more stars available, the more potential problem stars there are for people to find. In terms of the equivalent DR2 packs, this is just one step up, so I'm hoping most people's computers will be able to handle it, even if performance isn't very good. That said, even if it's just one "step" up, it's still quintupling the number of stars, so if turns out to be too much, it's easy enough to adjust, and more varieties of packs can be made available later. I reiterate, if people are finding it too hard to run (or believe that they will), let me know in the comments and I can always lower the size.



Potential Issues:

There are three main categories of issues that can be found in this catalog:
  1. Duplicate stars: for reasons I explain in the documentation, SpaceEngine's automatic duplicate merging does not work on this catalog. I did my best to filter them out, but make sure to flag any you spot.
  2. Impossible stars: either due to a model mismatch or just bad data, sometimes stars end up with combinations of values that aren't actually possible. Again, I do my best to filter them out at multiple points in the process, but I can't come up with every possible permutation of logic filters to cover every possible case off the top of my head.
  3. Incorrectly classified stars: these aren't technically always an issue, depending on how much of a variance it is. No astronomical data is ever going to be perfectly accurate at these distances, which is the reason you frequently see a ± beside a lot of values. If the classification is off by a few subclasses, that's to be expected. If it completely drops out of its "accepted" base or luminosity class, then there's a problem. There are plenty of stars with well known values that aren't already included in SpaceEngine's catalogs, so it's easy enough to just search one of the DR3 catalog stars' names in something like SIMBAD[simbad.cds.unistra.fr] and see if there's any good data on it, but manually doing this for every star would be absurd, so if you have a star you like that wasn't already in the SE catalogs, maybe check it to see if it's in the DR3 catalog and let me know if my results were accurate or not.



Progress Report:
08/29/2025—
Current Highest Range Slice: 100,000
Current Number Of Available Stars: 278,684,810
Stars Removed By Filters: 203,935,606
Usable Stars After Filtering: 74,749,204


(The previous GIF format looks real bad at this range, unfortunately.)



Bug Reporting:

My ability to look through the specifications of any given star as a result of my setup means that all I need is a name (or Source ID) and a brief description of what's wrong with it. If you spot any issues, be sure to let me know.



This work has made use of data from the European Space Agency (ESA) mission Gaia (https://www.cosmos.esa.int/gaia), processed by the Gaia Data Processing and Analysis Consortium (DPAC, https://www.cosmos.esa.int/web/gaia/dpac/consortium). Funding for the DPAC has been provided by national institutions, in particular the institutions participating in the Gaia Multilateral Agreement.
Popular Discussions View All (2)
1
11 Jul @ 2:26pm
Technical Documentation
Delta Ori
0
21 Aug @ 6:38am
Road To 1.0
Delta Ori
36 Comments
Delta Ori  [author] 8 Sep @ 12:55am 
Same with galaxies; I'm not really interested because my focus is the area around a few thousand light years of our solar system, but I've had several people ask me about doing one and it's probably within my capabilities, so I probably will make one at some point.
Delta Ori  [author] 8 Sep @ 12:55am 
The main issue with other large catalogs (like SDSS) is that most of them aren't meant for detail. They're all more focused on where the stars are rather than what the stars are. I've found that a lot of the ones that do include additional details mention that they got them by crossmatching with Gaia and using their values.

SDSS would suffer the same issue that the old Gaia DR2 addon had in that every star would be identical because there's nothing to classify them with. I started working on this because I'm writing a hard sci-fi series and used SpaceEngine to keep track of system locations and parameters, so I wanted things to be as accurately classified as possible, and doing SDSS would defeat that purpose.

That said, it would also make it really easy, so I could probably pull it off with only like a couple weeks of work if I felt like it.
animationfan13 7 Sep @ 6:28pm 
ive seen some other galaxy mods that dont really need quat, like isagi's milky way and andromeda sattelite galaxy mods.
animationfan13 4 Sep @ 4:49pm 
also btw for the galaxies, i dont think u need to give them all quat, especially the far ones, there is also axis u can use for the galaxies, u can prob use data from NED. also a SDSS catalog mod would be cool, celestia had did it :)
animationfan13 3 Sep @ 6:40pm 
thanks :D
animationfan13 3 Sep @ 6:40pm 
alr ill take a look at it when i got the time
Delta Ori  [author] 3 Sep @ 1:19pm 
It's a bulk file, not a script, otherwise it would've been way harder. And probably run into filesize limits to be honest. Most star catalogs are usually in a format like votable or csv already, so it doesn't take much. All that's really necessary at a bare minimum is making sure the SpaceEngine-required fields are present. 99% of the difficulty I've had with this has been my insistence on scientifically accurate spectral classifications, which aren't included in the Gaia tables and thus need to be determined based on the existing fields first.

Eh...

Technical Documentation Link Again <– Just read that if you're really curious.
animationfan13 3 Sep @ 12:39pm 
also just wondering, but how did u make this (by that I mean converting the star data to a SE script). I thought about porting some Celestia intergalactic star mods to SE, along with adding all the real stars in the Omega Centauri cluster and more
animationfan13 3 Sep @ 12:24pm 
:o ooooooh
Delta Ori  [author] 1 Sep @ 12:42am 
Smaller packs coming soon if it's too much...