The technology itself may be neutral however the policies in place whereas the NSA justifies firing a hell fire missle because the target looked around the right age to be in ISIS is not.
The issue here is with drones, in general. What you are arguing for is tighter regulation of the NSA, which may or may not be a valid issue, but is not the issue at hand. The issue here is are drones, as a whole, a threat? Or does the issue stem from those who misuse them?
I can see much farther away with binoculars than I can with a drone. Why does nobody care about them? Are microphones and cameras also invasions of privacy? What about the NSA? I don't think drones should be such a controversy. Punish those who use them irresponsibly, end of story.