If you have slight prescription (lower than -2.00) and don't glasses for reading, it's better that you don't wear your glasses whilst reading or working up close for LONG periods. If you wear your distance glasses for reading, you will need to use your eye's accommodative power more than your eyes. Some people think this can lead to increased nearsightedness.
If you are only nearsighted, then there is no need to wear them to read. But if you find no discomfort to read with glasses on, then you don't have to remove your glasses for reading. In this case, it won't ruin your vision because your eyes have no trouble accommodating for that.