You leave them alone until you can kill them yourself. In one playthrough I came across them as a group of 5 with the white walker dead, and in another it was a group of 80 with the white walker. Fire arrows kill the wights, and so do obsidian arrows. You'll get a valyrian steel nugget, and you go to Qohor to get it forged (50k denars). In a week, you will have a weapon that can kill the whitewalkers. Also, the nights watch archers and the wildlings archers use the arrows that would kill them.

Oh, ok. That makes sense, thank you so much :D

If you are going to Qohor to acquire a Valyrian steel weapon, be warned that the blacksmith/weapons merchant there will only accept your request if you have a high number of renown. 600, I believe.

Source: http://aworldoficeandfire.co.uk/wiki/index.php/Valyrian_Weapons

Hey there. In my case, I came across a 200+ party of Wights with NO white walker. Killed them all, went to Castle Black and the Lord commander was nowhere to be found. I looked into every Castle and town in the area and nothing. Asked his comrades of the Nightwatch, nothing.
Then it ocurred to me to go again where i founded the Wights, and found another party of them 50 strong WITH a White Walker and 6 giants. Once again killed them all an then i went back to Black Castle were Mormont were waiting for the bad news.