I did this, and it works for me. I can't guarantee it will work for you.
It will work when the game folder is left default. Otherwise, you will need to change pathnames.
The configuration file could probably be better. I just copy pasted part of the original file ./Match\ Rescue/MATH.conf. Some of that file produced errors, and I didn't want to copy to much or too little, but I didn't really give much thought to it. I'm sure the new conf file could be improved a lot.

The dosbox with this game is the windows version. It will not really work well with proton. You will need to install a native dosbox.
To check it you have dosbox installed and configured, open a terminal and enter:

dosbox

If dosbox does run, continue to the next step.
If dosbox doesn't run, install or reinstall dosbox and check again.

Navigate to the gamefolder
Open the terminal, and enter the commands:

rm ./*.bat
rm ./Math\ Rescue/Dosbox/dosbox.exe
touch ./start.sh
chmod +x ./start.sh
ln -s ./Math\ Rescue/Dosbox/dosbox.exe ./start.sh
touch ./math.conf

Open ./math.conf in your favorite editor
Copy and paste the following, and then save (this is part of the original ./Math\ Rescue/MATH.conf file. If you have problems with the weird characters at the bottom, copy from the original file):

[sdl]
# fullscreen: Start dosbox directly in fullscreen. (Press ALT-Enter to go back)
# fulldouble: Use double buffering in fullscreen. It can reduce screen flickering, but it can also result in a slow DOSBox.
# fullresolution: What resolution to use for fullscreen: original or fixed size (e.g. 1024x768).
# Using your monitor's native resolution with aspect=true might give the best results.
# If you end up with small window on a large screen, try an output different from surface.
# windowresolution: Scale the window to this size IF the output device supports hardware scaling.
# (output=surface does not!)
# output: What video system to use for output.
# Possible values: surface, overlay, opengl, openglnb, ddraw.
# autolock: Mouse will automatically lock, if you click on the screen. (Press CTRL-F10 to unlock)
# sensitivity: Mouse sensitivity.
# waitonerror: Wait before closing the console if dosbox has an error.
# priority: Priority levels for dosbox. Second entry behind the comma is for when dosbox is not focused/minimized.
# pause is only valid for the second entry.
# Possible values: lowest, lower, normal, higher, highest, pause.
# mapperfile: File used to load/save the key/event mappings from. Resetmapper only works with the defaul value.
# usescancodes: Avoid usage of symkeys, might not work on all operating systems.

fullscreen=TRUE
fulldouble=false
fullresolution=Fixed
windowresolution=1280x800

[autoexec]
# Lines in this section will be run at startup.
# You can put your MOUNT lines here.

@echo off
Mount C "~/.local/share/Steam/steamapps/common/Math Rescue/Math Rescue"
C:
cls

echo.
echo  Math Rescue
echo  by Apogee
echo.
echo  NOTE: To switch between windowed and full-screen press ALT+ENTER
echo.
echo  1. Episode 1
echo  2. Episode 2
echo  3. Episode 3
echo  4. Exit
echo 
choice /c1234 /s Choose your episode: /n
echo.

if errorlevel == 4 goto exit
if errorlevel == 3 goto e3
if errorlevel == 2 goto e2
if errorlevel == 1 goto e1

:e1
MR1.exe
exit

:e2
MR2.exe
exit

:e3
MR3.exe
exit

:exit
exit

Open ./start.sh in your favorite editor
Copy and paste the following, and save:

#!/bin/bash
cd ~/.local/share/Steam/steamapps/common/Math\ Rescue
dosbox -conf ./math.conf -noconsole -c