Once, one of my colleagues told me that he couldn't know where my vision focuses. It was just like I was trying to avoid eye contact or something else or I was just absentminded. I have to admit that I didn't know what I wanted and what I wanted to do in that period. In other words, I lived with confusion. I didn't want to see anything and I didn't want to listen patiently. But after I find what I want to pursue in my life, there is nobody told me that words again.
It is normal to lose focus in your vision. After spending a long time in front of computer screens, your eyes need to relax for a while. Usually our body will stop moving when we want to relax. Maybe our eyes just choose another way to relax themselves. But if this happens when you are working or driving and you can't control it, it can be very dangerous, you'd better see an eye doctor immediately.
Do you feel relaxed when your eyes lose focus? If you do, then go ahead and do it. Our eyes need to relax too. I think this will not affect your vision. One classmate in middle school had very strange eyes. He had some vision problems too. We couldn't know where his vision focused. It looked like his vision was at sea. Maybe this is because he has very strong degrees in both eyes, so he can't focus his vision well after taking off his glasses.