Does it really matter? I've known people who will remain subscribed to someone for the sole purpose of thumbing down every vid they upload. I've had it done to me as well. I don't think the poster of the video really cares much, especially after the first 2-3, when the pattern becomes apparent. Generally it makes the person doing it feel a little better, so let them do it. I doubt it's going to have any effect on your viewership, subcount, or even who manages to find your videos. I am sure if there were a few hundred people doing it, and you never got any thumbs up that there might be a few negative side effects though. As it stands I bet the only effect is to your own ego.