I came here to delete my post as I had figured out but deleted too quickly before reading the answer. The game finally let me leave. I had completed my tasks but for some reason the game didn't register this fact the first few times I tried to leave.