The dice rolls in a set pattern, going 1-4-2-6-3-5 and repeat.

Hitting the die once will make it land on it's current number. Most melee attacks will do.

For example, you want to roll a 4.
Walk up to the die so you are within melee range.
Observe it's pattern. Look for the number that comes before the number you want.
1 comes before 4. Use it as your signal to prepare.
Once the die turns from 1 to 4, hit it once.

If timed right, the die will roll a 4.

Once the die is set in motion, it's number will not change no matter how many times it gets hit, it will always land on the selected number.

Now you can get through the dice segments as you please, landing on whichever space you want.
You are the boss of the board now.