I'm for removing the downvote option entirely.
I haven't run into harassment issues yet as I'm fairly new to this community and quiet (mostly out of fear of getting on someone's bad side and being subjected to his/her downvote wrath...), but a fairly large forum for another MMO I play has removed this option so posts can only be rated up. People would actually stalk other players' activity just to down rate everything that person posted. It gave some people annoying notoriety so that they'd trollishly brag about being so popular that they had -300 votes on everything. It didn't help anything, and removing it has encouraged more sensitive people to actually post their ideas with less worry about ratedowns. Facebook and deviantART are both incredibly popular and only provide positive options ("Like" option or "Add to favourites/collection"). Aside from comment trolling, I think they've got it right.
If someone isn't brave enough to confront an artist about an issue in private, then that issue must not be important enough to pull that person out of his/her comfort zone.
Also, if a piece was good enough for 23,427 downloads and 182 people to like (numbers from Dwarf Red Beard at this time), what good does it do at all to know that 18 people disliked it? That is completely unnecessary information and is often just dismissed as the result of trolling, jealousy, or a misunderstanding. I know that my teenage self would rate some things incredibly critically and pick them apart out of jealousy. At this point I've learned to just ignore something if I don't like it (and that I want to punch teenage me in the face for not learning to just try to appreciate things), but I know that Minecraft has many young and/or competitive fans who may still share that same critical jealousy. I still feel it here and there, but try to turn it into "I have to try to do better," rather than "I have to berate that person's work."
Anyway, I left another Minecraft skin site to stay here because of so many random 1/5 star votes with no constructive criticism. I don't know whether the individuals did not like the skin quality, the subject of the skin, or me... even though I don't even know them. For example, I don't know if they hated Business Cat, thought my skin was incredibly ugly, or were so upset it didn't have a tail that they gave it 1 out of 5 stars. The person I created it for loved it, so I don't need to know that someone doesn't. On the other hand, it's quite warming to know if anyone else also likes it.