I dont like it, its opposite would be “eyesore” which is tremendously insulting in particular as both terms refer to marks of ones appearance one has little to no control over. And if there are people who are “easy on the eyes” then there are people who are “eyesores”.

Also I have never seen it used in a nonsexualizing manner.

“A sight for sore eyes” however is a goat compliment.