You talk about it like it's necessarily a bad thing, by default, for everyone. Why?
This sounds more like a uncompromising proclamation instead of thorough analysis.
The ancients had it as 'power corrupts', the abuses are plentiful and that every company that engages in these practices (and the government agencies as well) do this to ostensibly make our lives easier or keep us 'safe' is very well known and advertised. If you have evidence to the contrary feel free to share it but that's where we currently stand.
Let's assume for the sake of argument that the above events are unlikely though. When a few actors have access to the information of tens of thousands to billions of people though, this has an impact on a societal level. As jaquesm said, information is power and when one has so much information and lots of money to boot, they can begin to covertly influence policy and behavior and harass and marginalize their opponents. And they can do that directly, or by using the information of a third party, like a doctor, lawyer, religious leader, or even someone insignificant which happens to be a relative, etc. Moreover, companies can be sold, together with their databases, they can be forced to hand them over or they can be hacked. A treasure trove of data held by an otherwise principled company, might end up in the hands of an unsavory party.
Why is this a bad thing? History has shown again and again how such imbalances of power are abused. Here's a rather harmless example of data mining a mobile device + social network combined with social engineering to scam people out of money: http://toucharcade.com/2015/09/16/we-own-you-confessions-of-... If a game producer can do this, what are the pros doing?
In the modern era it is information asymmetry that we should worry about. How to prevent such a thing pragmatically is tricky.
This only works in the US and even there I have no illusions at all about the ability of a present day militia being able to fight off a trained army, it's a pacifier for overgrown toddlers. The only people that have to fear from citizens with guns are other citizens (with or without guns), the military would have absolutely no problem whatsoever dispatching those if it was decided that their lives and the resulting PR fall-out are less important than whatever objectives they were given.
> In the modern era it is information asymmetry that we should worry about.
Note that there are always provisions in the law to protect the lawmakers from having the laws applied to them.
> How to prevent such a thing pragmatically is tricky.
I think it can't be done unless you simply outlaw it wholesale and are prepared to follow up on it. And from a practical point of view this is now a rear-guard action, fall-back bit by bit and try to push back the point in time where we will have to conclude the battle was lost. This is not a problem that will simply go away, it has already gone way too far for that.
RE regulation on software engineers, Its impossible. For a software written, its PURPOSE and AUTHORS are subjective interpretations. It is much much harder to get common consensus if the software is surveillance, malware etc. So any regulation would do nothing but increase the already-so-complex-and-huge set of laws.
Well, then logical thing would be not to give anyone any power, ever.
My point is, if you take general principles and blindly apply it with "no analysis involved", you're likely to get to a pretty ridiculous state.
Don't you see any logical problems with this line of reasoning?
Just like any other tool such insights can be (and are) abused but it need not be like that.
The conclusion to reach is not to give anyone any power ever, clearly that's not feasible. The conclusion you're supposed to reach is that you can give power to people but you'll need oversight in place. Effectively you'll end up with checks and balances, pretty much the way most governments are set up.
And what history tells us - again - is that this isn't always sufficient to prevent abuses and our newspapers and other media seem to tell us that our current set of checks and balances have outlived their usefulness in the information age.
This flows from 'power corrupts' because it appears that those placed in power have - surprise - again abused their privileges.
Think of it as a warning beamed down from historical times to our present day that does not need more embellishment and is all the more powerful for its brevity, it is something so inherent in human nature that we need to be vigilant of it at all times, no matter who we end up placing trust in.
There is only one positive outcome of concentrations of power, and that is efficiency in execution. Which is extremely scary when combined with huge power.
This is really just the democracy discussion with different terms. It is well known that dictatorships are much more efficient at executing their plans. The inefficiency we voluntarily introduce when establishing and maintaining a democracy (and if you have ever been involved in democratic decisionmaking, the inefficiency can be really frustrating) is the price we pay to insure us against the efficient abuse of power as we have witnessed it countless times in human history.
I'm less pessimistic about that. I'm a big fan of gun control laws but I also think that the one positive thing that has come from the ongoing middle-east conflicts is that a determined militia can be genuinely problematic.
> Note that there are always provisions in the law to protect the lawmakers from having the laws applied to them.
To my original point about asymmetry, this is what we should be devoting our energy fighting.
> simply outlaw it wholesale
Outlaw what wholesale? I'm personally of the opinion that the long term end state will fall more on the side of honesty (combined with increased acceptance) than secrecy.
Any kind of abuse of power. The penalties for that should be severe. It's one of the few cases where I think that the penal system should be used as a means of discouragement rather than as one of education and rehabilitation.