No, it will not be so difficult. You could just wear the glasses first. Then you could wear an eye patch on it to adjust the place of it to let it suitable for the glasses. You could have a try. During the process, you should not use too much strength which could make you wear an eye patch in a mistaken way.
Ok, you should know that an eye patch is designed to help us with our sleep, it is clear that you are not going to wear eye patch with a pair of glasses on your head, that is basically impossible. Maybe you have some misunderstanding of an eye patch, with them, our sleeping journey would not be distracted. Just get some nice eye patch and take nice and sound rest.
Well, it would be better for you to wear contact lenses when you want to wear an eye patch. However, it is also possible for you to wear an eyepatch with glasses. You need to wear the eyepatch first. Adjust it to the best position and fastened it tight to your head. After that, just wear your glasses as usual.