| nettime's_risk_manager on Thu, 1 Oct 2015 17:16:19 +0200 (CEST) |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
| <nettime> Risks-Forum Digest Tue 29 Sep 2015 28.97 |
< http://catless.ncl.ac.uk/Risks/28.97.html >
RISKS-LIST: Risks-Forum Digest Tuesday 29 September 2015 Volume 28 : Issue 97
ACM FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
Peter G. Neumann, moderator, chmn ACM Committee on Computers and Public Policy
***** See last item for further information, disclaimers, caveats, etc. *****
This issue is archived at <http://www.risks.org> as
<http://catless.ncl.ac.uk/Risks/28.97.html>
The current issue can be found at
<http://www.csl.sri.com/users/risko/risks.txt>
Contents:
EPA v VW cheatware, AI & "machine learning" (Henry Baker)
Volkswagen Law (Bloomberg)
Security Standards: cars, voting, medical, critical infrastructure, etc.
(Alister Wm Macintyre)
Re: VW Scandal (Robert Schaefer)
Gaming security (Michael Albaugh)
Storing secret crypto keys in the Amazon cloud? New attack can steal them
(Ars Technica)
GCHQ operation "Karma Police" (Slashdot)
Network scientists have discovered how social networks can create
the illusion that something is common when it is actually rare (MIT)
Law Enforcement's Love/Hate Relationship with Cloud Auto Backup
(Lauren Weinstein)
Hello Barbie (The Week)
Re: U.S. and China cyber establish 'hotline' (Henry Baker)
Re: Ad-blocking (L. Mark Stone)
Abridged info on RISKS (comp.risks)
----------------------------------------------------------------------
Date: Sat, 26 Sep 2015 06:46:36 -0700
From: Henry Baker <[email protected]>
Subject: EPA v VW cheatware, AI & "machine learning"
The tech world is very excited, but also frightened, about AI & machine
learning these days; we worry about AI/machine learning algorithms replacing
doctors, lawyers, teachers, taxi drivers.
Perhaps one of the most straightforward applications of AI & machine
learning today would be a computer that "learns" how to control the
emissions of a vehicle engine so that it can pass the EPA emissions tests.
Consider the following conceptual model: a computer with a bunch (hundreds?)
of sensors and a bunch of actuators (tens?) that watches over a diesel
engine while it is being driven through a standard EPA emissions test.
The computer can sense perhaps air temperature, humidity, engine speed,
engine load, engine temperature, etc., and can control perhaps the air flow,
the fuel flow, the flow of Adblue (aka DEF/ISO 22241), etc. Sensors don't
cost very much, so there may also be sensors for the engine hood/bonnet
being open, the position of the steering wheel, etc.
We now put this system through hundreds of thousand of miles of "learning"
(millions of miles if the testing & learning can be virtualized & run in
parallel), so that the AI/machine learning algorithm learns to optimize
inputs like fuel and Adblue while still meeting EPA testing limits.
I can guarantee you that this AI/machine learning algorithm will quickly
notice that the best way to optimize for the EPA test is to "cheat" -- i.e.,
to notice that when the hood/bonnet is open and the steering wheel is
straight ahead, this would be a good time to optimize NOx and other
emissions, while during other conditions -- hood/bonnet closed and steering
wheel twisting back & forth (perhaps a curving country lane) -- emissions
aren't so important relative to performance.
(Perhaps someone -- Google might be in the best position with their work on
autonomous robots and its expertise in "machine learning" for their
self-driving cars -- is already working on such AI/machine learning
experiments for engine optimization; I'd be interested in hearing about them
if anyone can send me links.)
So is this AI/machine learning program "unethical" wrt the EPA tests ?
Should it be fined or go to jail ?
This is no longer idle speculation, as these AI/machine learning programs
are "recognizing" speech, gaits, faces, writing styles, etc. Are they also
"cheating" ?
As automobiles become more complex, and as machine learning algorithms
become more sophisticated, engine optimization computers may no longer be
"programmed" by humans using coding techniques, but will be "taught" by
following a long sequence of example situations and "learning" the correct
responses.
The DMCA may no longer be relevant to such computers, because *there is no
source code* to look at, and indeed, the *binary code* may itself simply be
a huge pile of random-looking floating point numbers in a *neural network*.
The *only* way to check such a system will be through exhaustive (!)
behavioral testing, as there won't be any source code to logically check for
"cheats" and "defeats".
I'm not trying to excuse the VW management that has already admitted to
"cheating" on the EPA tests, but as a computer scientist, I'm not so sure
where we go forward from here. We have terrific new opportunities with
electric and self-driving cars, so "optimizing" the government regulation of
diesel engines may simply be re-arranging the deck chairs on the Titanic.
------------------------------
Date: Mon, 28 Sep 2015 18:39:20 -0500
From: "Alister Wm Macintyre \(Wow\)" <[email protected]>
Subject: Volkswagen Law
Bloomberg Business Week, 28 Sep-4 Oct issue:
Germany has a law, about state ownership of corporations, called the
Volkswagen Law. Because of it, Lower Saxony owns 20% of VW, giving its
Prime Minister virtual veto power over VW. Thanks to the VW law, which the
EU has been fighting, many German companies have two boards of directors.
There is the management board, with Executives, which answers to the
Supervisory Board, run by the state, labor leaders, and shareholders.
As yet, there is no evidence which part of VW leadership structure had
anything to do with the emissions deception.
The article does not address the car hacking cover-up.
------------------------------
Date: Mon, 28 Sep 2015 14:05:05 -0500
From: "Alister Wm Macintyre \(Wow\)" <[email protected]>
Subject: Security Standards: cars, voting, medical, critical
infrastructure, etc.
I retired early this year. Previously I managed an IBM midrange system at
my day job.
The IBM OS tracked all sorts of updates, changes into a log, which we could
examine for suspicious activity, and I regularly checked it for potential
breaches, and some types of human error. We had some control over how much
activity to log, and for how many days, because of disk space constraints,
but some things, such as changes to the OS itself, we could not change the #
days below a certain point. I had occasion to alter the system date, so I
learned it was theoretically possible to erase the security log by advancing
the date, so that all contents were now past the erase date. It was
impossible to erase the fact that someone had messed with the system date.
For an OS log to be meaningful, some humans need to be able to dig out
what's important from the huge volume of frequently geeky entries.
All the passwords were changeable, but IBM knew how to bust their own
security. We found this out on the occasion of a relocation, where IBM
techs could not reassemble everything correctly, could not get into some
diagnostics, because of the security, which was on the hard drive which was
not reconnecting properly. They needed authorization from our CEO to bust
the security. Fortunately the CEO was watching the operation, so there was
no delay getting this.
If this is true across vendors, then there is a potential risk from former
and present employees of the computer vendors.
In my IBM Midrange world, there's a software monitoring package marketed
under the name "Needle in Haystack" which sends alerts to IT people, when
stuff happens which can have adverse impact on the enterprise. If the IT
people do not respond, in a reasonable time frame, the alerts go up the
management ladder.
I imagine that other platforms, outside the IBM world, ought to have similar
standards. Capture activity of a potentially suspicious nature, make it
available to relevant people in an intelligible and timely manner.
The next question is who this data belongs to, who may access it, update it
- employees, regulators, vendors of the hardware & software. I recently had
something weird happen with my auto, so I was re-reading the owner's manual.
I encountered a statement that data is captured about the vehicle
operations, and that this data belongs to the vehicle's owner.
------------------------------
Date: Mon, 28 Sep 2015 15:56:11 -0400
From: Robert Schaefer <[email protected]>
Subject: Re: VW Scandal
For those who don't have long memories, in the late 1980s, there was serious
attention to processor benchmarking by running compilers on well known
libraries and benchmark test programs' source code. It was claimed that
certain vendor's compilers (back then, compilers were often bespoke) would
recognize the library or test program, and optimize the number crunching to
the point where the benchmark test became worthless. Googling the terms
computer+benchmark+cheating shows that this is still going on today.
------------------------------
Date: Mon, 28 Sep 2015 10:37:12 -0700
From: Michael Albaugh <[email protected]>
Subject: Gaming security (Re: Murray comment on casino slots)
You might want to keep in mind the incident (possibly multiple) of slot
machines gaffed by the gaming board inspectors themselves. Back in the
1990s, but can we be sure they do not continue?
[Of course not. The gambling machines are held to a `higher' standard,
but it is still not a very high one. Remember, the best is the enemy
of the good, but the so-called good is nowhere near good enough. PGN]
------------------------------
Date: Mon, 28 Sep 2015 12:04:00 -0700
From: Lauren Weinstein <[email protected]>
Subject: Storing secret crypto keys in the Amazon cloud? New attack can
steal them
http://arstechnica.com/security/2015/09/storing-secret-crypto-keys-in-the-amazon-cloud-new-attack-can-steal-them/
Now a separate team of researchers has constructed a new method for
recovering the full private key used in a modern implementation of the
widely used RSA crypto system. Like the 2009 work, the new research
implements a CPU cache attack across two Amazon accounts that happen to be
located on the same chip or chipset. They recently used their technique to
allow one Amazon instance to recover the entire 2048-bit RSA key used by a
separate instance, which they also happened to control. The newer
technique works by probing the last level cache of the Intel Xeon
processor chipsets used by Amazon computers.
------------------------------
Date: Tue, 29 Sep 2015 12:58:41 +0200
From: Werner U <[email protected]>
Subject: GCHQ operation "Karma Police" (Slashdot)
Ars Technica's story on the revelations reported today by The Intercept that
the UK's GCHQ has been tracking World Wide Web users since 2007 with an
operation called "Karma Police" -- "a program that tracked Web browsing
habits of people around the globe in what the agency itself billed as the
'world's biggest' Internet data-mining operation, intended to eventually
track 'every visible user on the Internet.'"
http://yro.slashdot.org/story/15/09/25/2349201/gchq-tried-to-track-web-visits-of-every-visible-user-on-internet
<https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/>
<http://arstechnica.com/security/2015/09/gchq-tried-to-track-web-visits-of-every-visible-user-on-internet/>,
------------------------------
Date: Mon, 28 Sep 2015 12:00:47 -0700
From: Lauren Weinstein <[email protected]>
Subject: Network scientists have discovered how social networks can create
the illusion that something is common when it is actually rare
MIT via NNSquad
http://www.technologyreview.com/view/538866/the-social-network-illusion-that-tricks-your-mind/
Today, we get an insight into why this happens thanks to the work of
Kristina Lerman and pals at the University of Southern California. These
people have discovered an extraordinary illusion associated with social
networks which can play tricks on the mind and explain everything from why
some ideas become popular quickly to how risky or antisocial behavior can
spread so easily.
------------------------------
Date: Mon, 28 Sep 2015 20:16:15 -0700
From: Lauren Weinstein <[email protected]>
Subject: Law Enforcement's Love/Hate Relationship with Cloud Auto Backup
http://lauren.vortex.com/archive/001126.html
There's a story going around today regarding an individual who was arrested
and charged with assaulting a police officer when authorities arrived over a
noise complaint. But cellphone video recorded by the arrestee convinced a
judge that police had assaulted him, not the other way around. What's
particularly unusual in this case is that the arrestee's cellphone had
"mysteriously" vanished at the police station before any video was
discovered.
<...>
------------------------------
Date: Mon, 28 Sep 2015 18:39:20 -0500
From: "Alister Wm Macintyre \(Wow\)" <[email protected]>
Subject: Hello Barbie (The Week)
Toymaker Mattel is coming out with $75.00 Hello Barbie, a wi-fi enabled
doll. The little-girl owners shall press the belt buckle, and she'll ask
questions, like "Where do you live?" and answer the child's questions. I am
only guessing at what questions might be asked, as the list probably not yet
published where I can find it. The child's conversations with Hello Barbie
will be stored on Toy Talk servers, allegedly only to help Mattel improve
their speech recognition software. But how long before Mattel sells this
info to advertisers, or their data gets hacked? Can Hello Barbie get
software updates, so she can promote future Mattel toys, more Barbie
clothing, etc.?
------------------------------
Date: Mon, 28 Sep 2015 11:22:39 -0700
From: Henry Baker <[email protected]>
Subject: Re: U.S. and China cyber establish 'hotline' (RISKS-28.96)
Any bets about how long it will take for someone to hack this cyber 'red
phone' / 'red skype'?
------------------------------
Date: Mon, 28 Sep 2015 20:55:00 +0000 (UTC)
From: "L. Mark Stone" <[email protected]>
Subject: Re: Ad-blocking (Ross, RISKS-28.96)
Mr. Ross's remarks regarding ad-blocking software in Risks 28.94 ends with
the two questions:
"Why can I not choose to block advertisements on the Internet? What is it
about the Internet that mandates its advertisements on me, something other
media cannot do?"
Intended to highlight that the equivalent of Internet ad-blocking is allowed
with radio, television, magazines and newspapers by turning the page,
hitting the mute button or changing the channel, unfortunately the real
answer to Mr. Ross's questions is "money".
A quick Google search shows that Internet advertising is the largest single
category of advertising, outpacing even television. Online advertising can
be targeted as we know with much greater specificity than any other
advertising outlet. Neither advertisers nor online media therefore have any
incentive (and indeed extremely strong disincentives) to allow ad blocking
if they have any say-so. So why would they?
------------------------------
Date: Mon, 17 Nov 2014 11:11:11 -0800
From: [email protected]
Subject: Abridged info on RISKS (comp.risks)
The ACM RISKS Forum is a MODERATED digest. Its Usenet manifestation is
comp.risks, the feed for which is donated by panix.com as of June 2011.
=> SUBSCRIPTIONS: PLEASE read RISKS as a newsgroup (comp.risks or equivalent)
if possible and convenient for you. The mailman Web interface can
be used directly to subscribe and unsubscribe:
http://mls.csl.sri.com/mailman/listinfo/risks
Alternatively, to subscribe or unsubscribe via e-mail to mailman
your FROM: address, send a message to
[email protected]
containing only the one-word text subscribe or unsubscribe. You may
also specify a different receiving address: subscribe address= ... .
You may short-circuit that process by sending directly to either
[email protected] or [email protected]
depending on which action is to be taken.
Subscription and unsubscription requests require that you reply to a
confirmation message sent to the subscribing mail address. Instructions
are included in the confirmation message. Each issue of RISKS that you
receive contains information on how to post, unsubscribe, etc.
=> The complete INFO file (submissions, default disclaimers, archive sites,
copyright policy, etc.) is online.
<http://www.CSL.sri.com/risksinfo.html>
*** Contributors are assumed to have read the full info file for guidelines.
=> .UK users may contact <[email protected]>.
=> SPAM challenge-responses will not be honored. Instead, use an alternative
address from which you NEVER send mail!
=> SUBMISSIONS: to [email protected] with meaningful SUBJECT: line.
*** NOTE: Including the string `notsp' at the beginning or end of the subject
*** line will be very helpful in separating real contributions from spam.
*** This attention-string may change, so watch this space now and then.
=> ARCHIVES: ftp://ftp.sri.com/risks for current volume
or ftp://ftp.sri.com/VL/risks for previous VoLume
http://www.risks.org takes you to Lindsay Marshall's searchable archive at
newcastle: http://catless.ncl.ac.uk/Risks/VL.IS.html gets you VoLume, ISsue.
Lindsay has also added to the Newcastle catless site a palmtop version
of the most recent RISKS issue and a WAP version that works for many but
not all telephones: http://catless.ncl.ac.uk/w/r
<http://the.wiretapped.net/security/info/textfiles/risks-digest/> .
==> PGN's comprehensive historical Illustrative Risks summary of one liners:
<http://www.csl.sri.com/illustrative.html> for browsing,
<http://www.csl.sri.com/illustrative.pdf> or .ps for printing
is no longer maintained up-to-date except for recent election problems.
*** NOTE: If a cited URL fails, we do not try to update them. Try
browsing on the keywords in the subject line or cited article leads.
==> Special Offer to Join ACM for readers of the ACM RISKS Forum:
<http://www.acm.org/joinacm1>
------------------------------
End of RISKS-FORUM Digest 28.97
************************
# distributed via <nettime>: no commercial use without permission
# <nettime> is a moderated mailing list for net criticism,
# collaborative text filtering and cultural politics of the nets
# more info: http://mx.kein.org/mailman/listinfo/nettime-l
# archive: http://www.nettime.org contact: [email protected]