If you want to clear the stamp collection then in windows navigate to the folder C:\Users\<username>\AppData\Local\PostMouse\Saved\SaveGames (can enter %LocalAppData%\PostMouse\Saved\SaveGames in the address bar for explorer) and delete a file "SaveStamps.sav".

Start the game and you now will be able to collect all the stamps again.