SpaceEngine

SpaceEngine

GAIA DR3 Catalog Beta 0.10 RC3
Showing 11-20 of 26 entries
< 1  2  3 >
Update: 29 Jun @ 9:01am

V0.4.13.10
-Updated Teff-based classification logic to directly use Eric Mamajek's data table instead of a processed curve due to lack of sample size. (This is technically a reversion, but I've been using the custom curve since it was uploaded to the workshop so this is the first mention of it here.)
-Temporarily updated LumClass-specific subclass scaling back to linear for all classes for the same reason as above. This will likely change some time later on once I have some better data because Mamajek's table is specific only to dwarf stars.
-Maybe actually found and corrected the cause of the MIII issue?
-Increased range to 1300 parsecs.

Range updates are resuming at a better pace now that all of my (known) issues have been ironed out. Haven't had a chance to check on how the classification logic reversion is working out though; let me know if anything's funky.

Starting to hit a particularly dense band of stars in the input data, which has resulted in each range slice nearly doubling in size, which is therefore resulting in significantly longer processing times. I, however, am a genius, and have finally remembered after months of being unable to use my gaming computer for days on end due to RAM limitations that I have an older Work/Dedicated Server desktop with 128GB of RAM, but an extremely outdated GPU that isn't going to be needed at all for what I'm doing.

With more RAM overhead, I should be able to increase my number of parallel process workers significantly, which should speed things up quite a bit as long as the SSD in that thing is fast enough to avoid running into read-write race condition issues.

Also, I don't want to talk about the MIII issue. It was stupid. It is now less stupid.

Oh yeah, and, first O-class and Ib-class stars that survived filtering are now in the catalog. The Ib-class is listed on SIMBAD as a variable though, so I'm not sure how accurate its data is, but it apparently had a low enough RUWE score to make it in...
EDIT: Though, uh... both of those new stars aren't showing up for some reason??? I will attempt to determine why.

EDIT 2: Unfortunately, that's a false alarm, folks. Looks like they're missing because they have some bad values that made it past my filters and SpaceEngine automatically blocked them for me. I've updated the filters to catch those specific cases (and added some new classification logic that attempts to correct one of them) but both of those stars are no longer going to be of particular interest after the next update.

Update: 26 Jun @ 4:16am

V0.4.11.9
-Fixed(?) experimental rework of classification logic.
-Added additional duplicate filtering.
-Increased range to 1200 parsecs.

First things first, I may have... accidentally completely broken the main conversion script and spent several days trying to fix it before giving up and just reverting to an earlier version. Pretty much all progress was on hold during that time while I tried to figure out why a subset was leaking into a dataframe despite being cleared three different times along the way. The retry logic that blocked potentially bad models is no longer there, so there's not going to be quite as many stars recovered, but the number of problems it caused were just way too much for me to handle.

Good news, classification is finally looking pretty good. Still looks like an overabundance of MIII stars, but after trying a lot of things, I've come to the conclusion that the Gaia DR3 table maybe just happens to include a lot of them for some reason? I'll keep working on that in the background, but I'm leaning more towards it not actually being a bug anymore.

There was going to be an update to subclass assignment using some new temperature curves I was working on, but my input data being too noisy on top of the script issues has put that on hold. Might revisit in time for next range update because I've noticed some slightly incorrect classifications resulting from my current subclass curves.

In other news, in order to help the filtering script not take so long, I'm now submitting manual X-Match queries with a 1.5 arcsec error margin. This means the pipeline is no longer 100% automated, but it should clear out the last of any remaining duplicates. Hopefully. Keep an eye out just in case.

Minor addition unrelated to the actual catalog; with this update I've changed the star type breakdown in the workshop images to use the actual 5M pack rather than the entire dataset, since that's what people using it are actually going to be getting. (For clarity, the sky map density image is still the full dataset.)

Update: 21 Jun @ 12:02pm

V0.4.9.8
-Continued experimental rework of classification logic.
-Adjusted final pack creation logic to bypass randomization for low-population stars.
-Increased range to 1100 parsecs.

Still working on the new classification logic. It's still very much WIP, and I notice that there's far more luminosity class III stars than there should be, but I once again didn't have time to try and fix that for this update because the full script pipeline now takes around 6.5 hours to run.

The filters to omit stars with invalid value combinations are a little stricter in the new system, but I've added a new functionality to retry classification by discarding the bad values and pulling them from a different source model, which has actually resulted in a net increase in stars making it through the filters.

I've also managed to devise a bin system for the final output pack that sorts each spectral+luminosity class combo into their own bins, then randomizes the bins instead of the whole catalog. It then attempts to fill the pack evenly with those bins before backfilling from the larger ones. This ensures that any rarer star types that have low populations will always make it into the pack as a way to artificially increase diversity and avoid nearly every star just being KV-class.

Update: 16 Jun @ 9:39am

V0.3.8.8
-Experimental rework of classification logic.
-Increased range to 1000 parsecs.

Had to do another major rewrite of the main script because compiling the entire output file in my RAM was in hindsight perhaps not the greatest idea. Took the opportunity to overhaul a lot of the classification logic at the same time. In theory, it should be a little more scientifically accurate, but I've already noticed at least one new little issue, which unfortunately was caught too late to include a fix for it in this update. Keep an eye out just in case anything's gone horribly wrong.

Update: 12 Jun @ 12:45pm

V0.2.7.8
-Hotfix to correct broken catalog path (again).

Forgive me, for I am running on two hours of sleep and have been awake for 18 hours.

Update: 12 Jun @ 12:40pm

V0.2.7.7
-Slightly adjusted class assignment logic to avoid incorrectly assigning too many A-class subdwarfs (partially fixed).
-Increased range to 800 parsecs.

This marks the point where I simply don't have enough RAM to actually run the scripts, which required me to completely re-do the write logic of the main script to process the individual files before merging them, which I immediately introduced an error into by accidentally referencing the last dataframe instead of the final dataframe, thereby cutting 94% of all stars from the catalog.

It took me four days of attempted debugging before I finally realized that I had missed the "final_" at the very end of the script.

Probably would have had this update out sooner if not for that.

Update: 8 Jun @ 11:55am

V0.2.6.6
-Removed ~1 million faulty stars from the total catalog before selecting the 5 million for the final output that were previously allowed through.
-Increased range to 700 parsecs.

Uncovered an absolutely massive flaw with my filtering script that required almost a complete rewrite and took three whole days to properly fix. My output files still need their headers manually added by hand, but at least it's semi-functional. The filtering script also now takes several hours to run each time. Fun. Fixed that nonsense, regardless.

Update: 3 Jun @ 7:23am

V0.1.5.5
-(Finally) increased range to 600 parsecs.

Update: 1 Jun @ 12:49pm

V0.1.4.5
-Added additional Epoch-corrected HIPPARCOS catalog locational prefilter.
-Fixed a bug that was preventing the RUWE filter from working properly.
-Increased catalog range to 575 parsecs. (This was intended to be 600, but the final Apsis Supplemental file is taking forever to download and I need to go to bed.)

Update: 30 May @ 12:02am

V0.1.2.4
-Added Proper Motion filtering to avoid including stars with no PM value, since they can't have their Epoch back-projected without it.