So, this took some digging, but I finally found the reason for a massive slowdown - an odd one at that.
To reproduce this bug, I had only two mods loaded after the core - HugsLib, which didn't cause the problem when loaded alone, and then Humanoid Alien Races 2.0 after it.

The problem arises when a caravan runs out of other food, and has only chocolate left: until the chocolate is gone, the game speed progression is SIGNIFICANTLY slowed - so triple speed is slower than normal.

You can test this with the scenario editor - just start a new game with a bunch of chocolate and send a caravan out with just that.

HugsLib log just in case: https://gist.github.com/HugsLibRecordKeeper/d012da72074e0d08fa029eeaf5f63bbc