When removing the mod from a save first make sure that no pawns are currently inspired.
If you have pawns inspired on a map I recommend using the debug tool "End Inspiration" to remove those inspirations.
If you have any pawns that are inspired and in a caravan you will have to wait for the inspirations to de-spawn naturally or you will have to wait until they are on a map so you can end the inspirations with the debug tool.

Once all inspirations are removed it is safe to remove the mod. :)