SpaceEngine

SpaceEngine

GAIA DR3 Catalog Beta 0.10 RC3
Showing 1-10 of 26 entries
< 1  2  3 >
Update: 29 Aug @ 11:37am

V0.10.31.25
-Corrected new Class V filter.
-Increased range to 100,000 parsecs. (This added 2 new stars, both of which got filtered. About what I was expecting...)

Le Fix.

Update: 27 Aug @ 11:47pm

V0.10.30.24
-Added new filters for a specific case of unphysical Class V stars.
-Added new filters for a specific case of unphysical Class VI stars.
-Adjusted Class III hard limits to account for temperature.
-Increased range to 75,000 parsecs.

Think I caught all the current issues, so one more round of checks to make sure the new Class III limits are giving the expected results and if all is well then this is likely the final update before release.
EDIT: Scratch that; at least one more update. I did a little oopsie when trying to fix the aforementioned V-class issue, so a lot of the problem stars are actually still making it through. Fixed and already re-running, but... you know... that takes a while.

I decided to up the distance by another large chunk, but it didn't really add much, as expected. Might decide to up it to 100,000 just to have a nice-looking number at release, but it's really not worth it.

Update: 21 Aug @ 6:13am

V0.10.28.22
-Increased range to 25,000 parsecs.

Surprise?

I mentioned a few times that I was experimenting with increasing the size of the range slices I was downloading, but I realized... I've done several software and hardware upgrades since starting this project, and... Why not just put that to use and finish it?

I'm going to do another large jump to 50000 just for the sake of completeness, but I already have it downloaded and there's only around 8000-ish stars in that range since most of it is outside the galaxy.

I have a bit to say about my remaining plans going forward, but I'll stick those in a discussions post to avoid cluttering the change notes any further, as I've noticed I have a tendency to do lately, but needless to say, you can probably expect a full release in a month or two at most.

EDIT: Post is up.

Update: 18 Aug @ 3:40am

V0.10.27.22
-Fixed Subclass Assignment Issue.
-Fixed Another Temperature-Related Issue.
-Reduced Required Score For Classification By 1.
-Added Additional Guard Clauses To Account For Loosened Score Requirement.
-Increased range to 3500 parsecs.

So, I caved and loosened the filters a bit. Basically, now as long as a star sits within the absolute limits of its class and passes both the Density and Magnitude checks, it's allowed in. This once again led to some particularly poofy M-type Class II's getting misclassified as Ib's, which I've already fixed. They aren't in this version, but I've also added a set of guards for each class border just to be safe even though I didn't spot any other issues.

This update would have been out sooner, but my filtering script for some reason decided to start trying to load Linux subprocesses even though I'm running Windows and crashed, requiring me to add a line to my joblib import to force it to run Windows-compatible. Then when I tried to actually update the Steam Workshop page, the configuration tool broke. So that was fun.

In other news, experimented with upping my TAP Query size again from 50 to 100 parsecs. Seems to be fine; haven't received any angry phone calls from The CDS about my stress-testing of their servers yet. I'll try 1000 once I'm at an even 4000. If it goes through without timing out then that'll significantly increase my work speed.

Update: 8 Aug @ 3:23pm

V0.10.25.20
-Completely rewrote classification logic (Yes. Again. I... I know.)
-Tightened filters to only allow stars with high quality data.
-Integrated the COSMOS Gaia DR3 G-band Bolometric Correction Tool for any Absolute Magnitude checks. (Special thanks to Christophe Ordenovic.)
-Reintroduced LumClass-based Temperature curves. (Special thanks to the University of Northern Iowa.)
-Added Carbon Stars.
-Increased range to 3000 parsecs.
-Known Issue: Some Subclass assignments might be off. (Potentially just for subdwarfs?)
-Known Issue: Carbon Stars seem to be using some sort of placeholder values in the Gaia table, as there are far too many with the exact same Mass and Temperature for them to be accurate. Nothing I can do about that.

*Lifts Imaginary Nightvision Goggles*
"Kept you waiting, huh?"

So... Been a while. I realized that as much as I wanted to avoid it, I was slowly falling back into the trap of endless convoluted promotion and suppression clauses yet again. I actually tried *another* different classification logic style between then and now, but it just kept happening. I realized in the end that the data was just too noisy, and I had to be willing to make sacrifices to cut through that noise. Not like I'm lacking in stars, so who cares if I start rejecting almost 80% of them?

New logic uses a set of three increasingly strict levels of physical requirements for each luminosity class. Each "limits" block that a star sits within scores it points for that respective class, with higher strictness blocks being worth exponentially more. At the end, the class with the highest score wins and gets assigned.

Another "noise" issue I discovered wasn't actually an issue, just an oversight on the part of my logic. There were always stars getting thrown back and forth between class I, II, and III, and this is because they weren't actually any of those. In fact, they weren't any LumClass at all. Those were just Carbon Stars, so I updated everything to classify them appropriately rather than filtering them out for not fitting in right.

I had put everything on pause while I was working on all this (Around 65 hours of work, actually. I had to take time off my "real" job for this.), but I'll be getting back into the habit of downloading new slices. Now that the classification logic is pretty much 100% done, I'll be able to actually keep my cache files so it doesn't take ten hours to run every time.

Update: 18 Jul @ 9:16am

V0.7.22.12
-Fixed the most major-est of bugs ever laid eyes upon by mankind.
-Heavily updated classification logic (Again) (WIP).
-Updated duplicate filtering for 0.990.48.2075 Public Beta.
-Increased range to 2500 parsecs.

Okay, so, I maybe, perhaps, potentially, just might have discovered that I put a hyphen where I should have put an underscore, and half of the most important values from the Apsis tables were going completely unused and were instead being replaced by roughly-calculated values generated by fallback clauses. As far as I can tell, this has been the case since... pretty much the very start.

So...

There was a lot of stuff that needed updated, because I was creating corrective logic for problems that didn't exist this whole time. I've been on vacation for a week, and spent a lot of that working on this, but there are still a lot of issues I need to work out (I'm currently looking at a Class II with a Luminosity of 0.75). On the bright side, you'll find that there is far more variety now. While there aren't any in this update, there should actually finally be some Class I stars soon. I'm pretty sure I saw a few that were misclassified as II, but I don't have time to fix that right away as running the full script pipeline now takes around 8-ish hours if fully un-cached (which it has to be every time I make classification logic changes). Good thing I did those speed optimizations a while back or who knows how long it would be taking...

In non-catalog news, the sheer number of stars is making the linear shader display in the density images kind of useless. I've switched the RA/DEC image to sqrt and the 3D gif to square (it shows the density difference between the inward and outward facing sides better).

Update: 11 Jul @ 2:09pm

V0.6.19.11
-Completely rewrote classification logic.
-Re-added retry logic for failed classifications.
-Maybe(?) fixed an issue resulting in corrupted Gaia SourceIDs.
-Increased range to 1900 parsecs.

Hooooo boy...

So I guess to start with, that new classification logic. The old setup was growing incresingly convoluted with dozens upon dozens of somewhat-randomly ordered and sometimes-conflicting clauses. They've now all been streamlined, conflicts resolved, new logic added, and cases where a star could potentially fall into multiple categories (depending on internal physical parameters that I don't have access to) ordered in such a way as to prioritize non-main sequence just for the sake of variety since there's no real way to determine which one would be more accurate anyway.

I did some extensive testing to make sure there weren't too many errors, but as always with a major change, keep an eye out. I'll probably not make any new changes outside of fixes for a while, so I'll take some time to continue doing spot-checks on the results.

Next, the function to retry bad values by blocking the original broke a lot of things the first time I tried to do it. I made it not do that, and reintegrated it, resulting in around a 70% recovery rate, which is... suspiciously good, but the results are all passing the filters, so ahdunno...

In other news, I discovered an issue where python apparently doesn't like long integer values (like SourceIDs) and auto-converts them from Int64 to Float64, completely breaking them and resulting in star names that don't actually exist. Not sure if it was introduced during my recent round of changes or if it was a long-standing bug, so all of the star names might be different now (not that I'd expect anyone to be able to tell without having saved any).

Since I've hit a particularly dense region of stars in my TAP queries, they take a lot longer to complete, meaning I'm completing at most one per day. Because of this, I tested doubling my query distance ranges so that I can just leave them running overnight; no issues, so the range update speed has actually increased as a result.

Last thing not really related to this update, but you'll note that there's still a lack of O-class stars. There's a reason for that, which I'll put in a discussions post like the documentation soon because I'm not sure how long Steam allows your Change Notes to be...
EDIT: Merged that with the documentation post.

Update: 5 Jul @ 11:34am

V0.5.17.10
-Increased range to 1600 parsecs.

Trapped at work for the majority of this week, so not a lot of time to make improvements.

Have a range update.

Also, I've switched to using the script printout for the star type breakdown in the images. It's ugly, but far more informative.

Just look at hose MIII's...
Gross.

Update: 3 Jul @ 11:13am

V0.5.16.10
-Modified Luminosity Class assignment logic.
-Added Luminosity Class VI to the classification logic.
-Increased range to 1500 parsecs.

I gave up on the LumClass-based temperature curves; any available data is just too noisy to reasonably get a definitive result, which is probably why no one's bothered to compile a table for anything other than Class V.

As a consolation, I re-did some of the Luminosity Class assignment logic, including enabling the script to appropriately assign Class VI to subdwarfs. That bit's still brand new, so no promises it's 100% refined yet.

Also seeing a resurgence of the MIII overabundance. I shudder in horror at the thought.

And in other news, those two LumClass II stars that made it into the pack this update might actually be legit this time. They technically just barely pass all my filters, so unless I'm missing something they seem to at least be physically possible. Verdict on the accuracy of their classification is still out though; one of them has no SIMBAD entry, and the other one only has a single source, which is some random paper from the 80s, so I'm not sure their classification is any more likely to be accurate than mine is.

EDIT: Aaaaand neither the Class II's nor any of the subdwarfs are being loaded by SpaceEngine. I'm gonna have to figure out what criteria it's using for filtering, because it's clearly much stricter than mine...
EDIT 2: Nevermind; they just don't show up in the star browser for some reason, but I can search them by name just fine... Here's a free Class II on the house (Gaia DR3 4319861205637409024), but I guess you'll just have to explore to find those subdwarfs...
EDIT 3: Apparently the issue is that it for some reason doesn't think that the Gaia DR3 stars are their systems' main star despite being their systems' only star, so you can find the subdwarfs if you search specifically for objects whose parent star is a subdwarf. No idea how that works...
EDIT 4 (07/11): The Class IIs have gone missing in today's update; I'll try to figure out why and get them back in.

Update: 1 Jul @ 11:53am

V0.4.14.10
-Increased range to 1400 parsecs.

Still no LumClass-based subclass scaling; I spent literally the last twelve hours speed-optimizing the main script for the dedicated computer. It now runs three times faster, but that's still around an hour, and I had to sacrifice some float precision. That shouldn't hypothetically cause any issues, but you never know.

Speaking of, seeing some class II and Ib stars in this pack. Don't have time to check if they're legit, but don't be surprised if SpaceEngine won't load them due to any borked parameters. Overhauling an entire 850+ line script from top to bottom runs the risk of introducing all kinds of wonkiness.