Only other thing that comes to mind is a 'dirty' sensor. You can clean the 2 photocell sensors with a q-tip and water, glass cleaner, or I use isopropyl alcohol. I dab the q-tip in the alcohol and clean the front and rear sensor with one end, then dry it with the other.
If it is a dirty sensor, I would suspect the one on the face of the mirror, (facing the rear of the car). That's the one that would sense when there is light from behind and dim the mirror. If it were the front sensor (facing the windshield), the mirror is dim all day, then seems to work properly at night.
Aside from cleaning these, yup, time to replace the mirror. If I had to replace mine right now, I would spring for the extra few dollars for the Homelink Mirror. Same features as the self-dimming compass mirror, but adds the Homelink transmitter. One more thing I don't have to carry in the car, provided your garage door opener receiver supports the homelink type transmitter.