Before you edit your saves, backup your stuff. That way you know which thumbnails go to which files before you accidentally break anything. Personally I'd recommend using Game Save Manager, and making a custom profile for Jazzpunk with the locations I provided. That way you can back up all the parts of your save at once.

The save data for Jazzpunk is split up into 3 places:

1. The text files containing your actual save files and their information. Located at:

• %APPDATA%\..\LocalLow\Necrophone Games\Jazzpunk_ Director_s Cut

2. The thumbnails you see when loading a save from the menu in-game. Located here:

• YOUR STEAM DIRECTORY\SteamApps\common\Jazzpunk\windows\Jazzpunk_Data\StreamingAssets\SaveGameScreens

3. Assorted game state information, located in the registry at:

• HKEY_CURRENT_USER\Software\Necrophone Games\Jazzpunk: Director's Cut

Your save files are made up of the text files, and the thumbnail you see in-game.
To delete a save (or just organise them like me) you need to delete or rename both parts. They're linked by the date & time section in the name of each file.

Your saves are sorted as newest date first / oldest date last. So saves will show in game with the newest at the left side of the list. If you want to change the sort order, just change the date/time in both file names of the save you want to sort (Text File & thumbnail).

If you don't care about keeping any data, and just want a fresh start with the game for whatever reason, but don't quite know how to do that with the locations I mentioned, here's[drive.google.com] a PowerShell script that will delete all of Jazzpunk's save data for you. I'd recommend deleting them yourself if possible; while the script might work for me, there's no guarantee it will work for you.
In order for it to work, you need to be on a 64bit version of Windows and Jazzpunk needs to be installed your default steam library.

You can read what the script does by opening it in notepad if you're curious. I'd suggest never running scripts downloaded from the internet without reading them first.