Sax faced many brushes with death. As a child, he once fell from a height of three floors, hit his head on a stone and was believed dead. At the age of three, he drank a bowl full of acidic water, mistaking it for milk,[6] and later swallowed a pin. He received serious burns from a gunpowder explosion and once fell onto a hot cast-iron frying pan, burning his side. Several times he avoided accidental poisoning and asphyxiation from sleeping in a room where varnished furniture was drying. Another time young Sax was struck on the head by a cobblestone and fell into a river, almost dying.[7]

His mother once said that "he's a child condemned to misfortune; he won't live". His neighbors called him "little Sax, the ghost".[7]