Could someone help me understand why if I don't have any people producing food in one of my cities, I'll get a new citizen in 6 turns. But if I put one worker there, the next citizen will come in 97 turns. Shouldn't it be the other way around?

Without any workers in food production, my food is at -2.