It is decided by your prescription power, if you have strong strength, it is not OK to take your glasses on and off couple times or it will make you prescription worse. But if you have weak strength, you can do that to make your eyes less strain.
I suggest wearing your eyeglasses when you need to. If you need them to read the board, then use them whenever you need to have clear distance vision. Wearing or not wearing your eyeglasses will not make your uncorrected vision better or worse.