Thursday, May 28, 2009

Instinctive Computing Workshop to Explore Transformational Developments

"Instinctive computing is an emerging framework for a new kind of operating systems. Instead of making patches on an existing system, we want to make a new platform that integrates security, privacy and visual thinking in one place. Yang Cai, PhD., CyLab Ambient Intelligence Lab, 2009

CyLab News: Instinctive Computing Workshop to Explore Transformational Developments

On June 15-16, 2009, at Carnegie Mellon University CyLab, Pittsburgh, PA, USA, Yang Cai, PhD., director of the CyLab Ambient Intelligence Lab, will host an Instinctive Computing Workshop.

Instinctive computing is a computational simulation of biological and cognitive instincts. Instincts profoundly influence how we see, feel, appear, think, and act. If we want a computer to be genuinely secure, intelligent and to interact naturally with us, we must give computers the ability to recognize, understand, and even to have primitive instincts. In this workshop, we will explore transformational developments in this area, including the building blocks for instinctive computing systems and potential applications such as security, privacy, human-computer interaction, next generation networks, and product design.

Features speakers include:

Kevin Warwick, University of Reading, UK on "Robots with Biological Brains and Humans with Part Machine Brains"

Chris Urmson, Carnegie Mellon, USA on "The Urban Challenge and the Promise of Autonomous Vehicles"

David Farber, Carnegie Mellon, USA on "The Real Next Generation Internet and It's Impacts"

Howard Lipson, CERT, USA on "Built for Survival"

Michael Leyton, Rutgers University, USA on "Structure of Paintings"

Yvonne Masakowski, NAVY, USA on "Human Performance in Virtual World"

Xavier Alamán, Universidad Autónoma de Madrid, Spain on "The Intelligence of Intelligent Environments"

Themes and topics covered include --

Instinctive Systems:

Soft biometrics
Self-healing systems
Biologically inspired computing
Brain-Computer Interfaces
Swarm intelligence
Software agents
Intelligent environments
Ambient Intelligence
Augmented reality

Instinctive Cognition:

Instinctive thinking
Visual thinking
Instinctive interaction
Emotional interfaces
Animal instincts
Language instincts
Robot-Human Interaction

The program will also include numerous demo sessions and poster presentations.

For more information and to register for the conference, click here.

Instinctive Computing Workshop is co-sponsored by the National Science Foundation and Carnegie Mellon University CyLab.

For more information about Yang Cai, the Ambient Intelligence Lab and Instinctive Computing, read CyLab Research Update: Basic Instincts in the Virtual World.

Friday, May 22, 2009

CyLab Seminar Series Notes: The Evolution of A Hacking Tool, Moxie Marlinspike on SSLstrip

"Personally, I find that when looking for security vulnerabilities, a good place to start are those places where developers probably don't know what they are doing, but feel really good about the solution they implement." Moxie Marlinspike,

CyLab Seminar Series Notes: The Evolution of A Hacking Tool, Moxie Marlinspike on SSLstrip

[NOTE: CyLab's weekly seminar series provides a powerful platform for highlighting vital research. The physical audience in the auditorium is composed of Carnegie Mellon University faculty and graduate students, but CyLab's corporate partners also have access to both the live stream and the archived content via the World Wide Web. From time to time, CyBlog will whet your appetite by offering brief glimpses into these talks. Here are some of my notes from a talk delivered by independent researcher Moxie Marlinspike on 5-18-09. -- Richard Power]

In the late 1990s, as electronic commerce (and cyber crime) started heating up, I got many calls from reporters asking the same question over and over again, “Is it safe to use our credit cards over the Web?”

In response, I would point out that this was the wrong question. The right question, the bigger question, was lost on them, i.e., “What is going to happen to your credit card information at the other end of the transaction?”

The point I was driving home was that the turning over of insecure server brimming with credit card records was going to be the cyber crime of the decade, not the petty cyber theft of one credit card transaction.

As the years passed, and the aggregation of tens of thousands of credit cards on those servers turned into aggregates of hundreds of thousands of credit cards, and then in turn, millions and tens of millions of credit cards, my warning turned into an everyday fact of life in cyberspace.

But back then, for everyone one of us, who was trying to direct the attention of the public (and those paid to inform it) away from the vulnerability of a single transaction to the greater threat of digital stagecoach robberies, there were a dozen vendors and consultants waiting in line to answer those reporters in a much more simplistic (and misleading) way, “Yes, of course, it is safe, because we have SSL.”

I was reminded of this subplot in the cyber risk timeline of the last decade recently, as I watched Moxie Marlinspike’s CyLab Seminar Series presentation on “New Tricks for Defeating SSL in Practice.”

Because, unlike Moxie’s earlier contribution, SSLsniff, which issues Man-in-the-Middle (MITM) attacks on SSL, the fascinating work that Moxie talked us through in this particular presentation, SSLstrip is not really about breaking SSL, it is about getting around SSL by exploiting weaknesses in the enveloping framework in which SSL is operating.

That’s what reminded me of those vendors and consultants who wanted to put security concerns to rest by assuring the public and the press that SSL was in place. It was the wrong answer to the wrong question then, and it, and its security marketplace equivalents, still are now.

And just as the real takeaway from Moxie’s talk was not so much the attacks he demonstrated, although they are indeed compelling, it is that independent research and pure hacker kulchur is alive and well, at and elsewhere in the shadows of cyberspace.

Here are some brief excerpts from Marlinspike's talk:

In his opening remarks, Moxie observed, "The title of the talk is kind of a misnomer because I am going to be talking about SSL in the context of the Web, so the title should be 'Tricks for Defeating HTTPS in Practice.'

During his discussion of the background, that is the development of SSLsniff, Moxie shared some insight into how to succeed in such research, "Personally, I find that when looking for security vulnerabilities, a good place to start are those places where developers probably don't know what they are doing, but feel really good about the solution they implement."

Before moving on to SSLstrip, Moxie reviewed SSLsniff's ongoing relevance:

"You'd be surprised who still doesn't check basic constraints. I have promised to talk more about this in the future, and I will.

"Even when people got warning dialogs in browsers that had been fixed, most of the time they'd just click through them. There is an interesting little sub-point there, most browsers would start validating certificates, and as soon as they found a problem, they would stop. They would pop up a dialog and say, 'there is a problem, do you want to continue?' You could do these interesting tricks. You could create a totally bogus certificate that expired two minutes ago. It has invalid signature, everything about it is wrong. But the second thing that most SSL implementations did was check the time stamp, to make sure it hadn't expired. So what would happen is that it would check the name on the certificate, which of course matches whatever you are trying to intercept, and then it would check the time stamp and say, 'Oh, this expired two minutes ago.' And it would pop up a dialog to the user, saying 'There is something wrong, this ticket expired two minutes ago.' So a lot of users would be thinking "Oh, everything is fine, it's just that this ticket expired just a couple of minutes ago, they just haven't got around to issuing a new certificate yet, this should be fine,' and they would click through it.

"SSLsniff is still useful as a general MITM tool for SSL. The folks who did the MD5 hash collision stuff last December (2008) used sslsniff to hijack connections once they'd gotten a CA cert."

And in concluding his remarks on SSLsniff, Moxie gave fair warning, "There are other uses yet to be disclosed another day."

Moving on to explore what led to his development of the SSLsniff tool, Moxie drew a distinction between "browsers then and now," pointing out that in the past they employed a "postivie feedback system" to re-assure the user about the presence of security (i.e., "a number of indicators deployed to designate that a page is secure, a proliferation of little lock icons, a URL bars that turn gold") as opposed to the "negative feedback system" used now (i.e., less emphasis on sites being secure, e.g., "the proliferation of little locks has been toned down, and Firefox's gold bar is gone," increased emphasis on alerting users to problems, e.g., "a maze of hoops that users have to jump through in order to access sites with certificates that aren't signed by a CA."

In assessing the implications of this shift from the attacker's perspective, Moxie concluded: "If we trigger the negative feedback, we're screwed. If we fail to trigger the positive feedback, it's not so bad."

In looking for opportunities to apply this maxim, Moxie next looked at the relationship between SSL and HTTP.

"How do people use SSL? Nobody types 'https://' or 'http' for that matter. People generally encounter SSL either by clicking on links or through 302s, which means that people only encounter SSL through HTTP. ... We can attack both of these points through an HTTP MITM."

In rapid success- ion, Moxie led the attendees through his iteration of SSLstrip.
In the first cut, SSLstrip simply attacked HTTP instead of SSL, with promising end results, "The server never knows the difference. Everything looks secure on their end. The client doesn't display any of the disastrous warnings that we want to avoid. We see all the traffic."

"We've managed to avoid the negative feedback, but some positive feedback would be good too," Moxie said, pushing further, "People seem to like the little lock icon thing, so it'd be nice if we could get that in there too."

So in the next iteration, Marlinspike's program would respond to a favricon request for a URL it had stripped by sending back its own little padlock icon.

In his further develop- ment of the program, Moxie also dealt with the problem of sessions: "Sessions expire, and it's not always clear when or why, but they don't usually expire right in the middle of an active session. So what we do now: When we start a MITM against a network, strip all the traffic immediately, but don't touch the cookies for 5 min (or some specified length of time). As the cookies go by, make note of the active sessions. After the time is up, start killing sessions, but only new sessions that we haven't seen before. These should be the “long running” sessions that won't be seen as suspicious should they disappear."

"The results were kind of astound- ing. In 24 hours, 114 Yahoo logins, 50 Gmail logins, 42 secure posts to Ticket Master, 14 Rapidshare accounts, 13 Hotmail accounts, 9 Paypal logins, 9 LinkedIn accounts, and 3 Facebook accounts, and many more. That means in 24 hours, I got 117 email accounts, 16 credit card numbers, 7 paypal logins, over 300 other miscellaneous secure logins. So I wondered, 'That's a lot data, but what is the success rate? How many people didn't submit their data? How many people got to whatever it is they wanted to log into and then didn't log in? So I modified it a little bit and ran it again for another 24 hour period, and this time, I logged the number of people who browsed to a page which would have had secure post and then didn't post data. So how many people browsed to the g-mail login and then didn't login? How many people browsed to the paypal login and then didn't login? ... I got a comparable number of secure posts, but the question is how many people balked? Zero. In a 24 hour period, not a single person browsed to a page with a secure post and didn't post data."

When Marlinspike presented these results at Blackhat in D.C., one of the responses he received was, "Well, OK, this is a problem with 'User Education,' users are ignorant, they do not know how to use the web."

Moxie did not think that "user education" was the real issue, and rightfully so.

To demonstrate the point, he went on to deliver the presentation at Blackhat Europe in Amsterdam, and before he did he ran SSLstrip on the network at the conference, intercepted over one hundred secure logins in a thirty minute period and selected ten of the passwords collected to include on a slide for display during his talk.


Stay tuned. Marlinspike is working on more.

Among the lessons learned from this research, "Lots of times the security of HTTPS comes down to the security of HTTP, and HTTP is not secure.

Some Other CyLab Seminar Notes

CyLab Seminar Series Notes: User-Controllable Security and Privacy -- Norman Sadeh asks, "Are Expectations Realistic?"

CyLab Seminar Series: Of Frogs, Herds, Behavioral Economics, Malleable Privacy Valuations, and Context-Dependent Willingness to Divulge Personal Info

CyLab Seminar Series Notes: Why do people and corporations not invest more in security?

CyLab Research Update: Basic Instincts in the Virtual World?

For information on the benefits of partnering with CyLab, contact Gene Hambrick, CyLab Director of Corporate Relations: hambrick at

Sunday, May 17, 2009

CyLab Seminar Series Notes: User-Controllable Security and Privacy -- Norman Sadeh asks, "Are Expectations Realistic?"

"As we all realize on a daily basis, application developers have great expect- ations about what we users are capable of doing. They expect us to be able to properly configure the firewall on our home computer and virus settings on our cell phone. As enterprises move towards more agile and decentralized business practices, developers also expect us to configure increasingly complex access control policies at work. Are these expectations realistic? If they are not, how much trouble are we in and what can we do about it?"

CyLab Seminar Notes: User-Controllable Security and Privacy -- Norman Sadeh asks, "Are Expectations Realistic?"

[NOTE: CyLab's weekly seminar series provides a powerful platform for highlighting vital research. The physical audience in the auditorium is composed of Carnegie Mellon University faculty and graduate students, but CyLab's corporate partners also have access to both the live stream and the archived content via the World Wide Web. From time to time, CyBlog will wet your appetite by offering brief glimpses into these talks. Here are some of my notes from a talk delivered by Norman Sadeh on 3-16-09. Sadeh's team of collaborators in this important research includes faculty members Jason Hong, Lorrie Cranor, Lujo Bauer, Tuomas Sandholm, post docs Paul Hankes Drielsma, Eran Toch, Jinghai Rao, and PhD students Patrick Kelley, Jialiu Lin, Janice Tsai, Michael Benisch and Ram Ravichandran. -- Richard Power]

Can users be expected to effectively specify their policies? Do people even what policies they want or need? Even if they did, could they articulate these policies? What if policies evolve over time? Are we always willing to invest enough time to have perfect policies or are there important trade-offs between user burden and policy accuracy? Can we develop technologies that mitigate these potential problems and empower users to more accurately and efficiently specify security and privacy policies?

To shed some light on these compelling questions, Norman Sadeh shared some insights into data from lab and field research on mobile social networking applications.

An example is a location sharing application that uses GPS and WiFi triangulation on laptops and cell phones and allows people to share their locations with friends, families, colleagues, and ... Well, that is one of the big issues that arises in this space, who exactly are you sharing this information with? And to what extent can you control access to it?

According to Sadeh, although many such applications have been released over the past several years, adoption has been rather limited. Early on, Sadeh and his team noticed that users had great difficulty specifying location sharing privacy policies that accurately reflected their preferences.

“So what’s going on? Is it because these applications have bad user interfaces? Do people who define more privacy rules do better? Do the people who spend more time defining and refining their rules do better?” Sadeh continued. “Location sharing applications seemed to be a very good domain to study these and related issues. Because, at the end of the day, the problems are the same, whether you are trying to configure a firewall at home or at work, or you are trying to configure social networking policies. Ultimately, the question is whether we can empower users (both expert users and lay users) to specify combinations of rules that enact the behaviors they really want to enforce?”

From 2003 to 2005, Sadeh and his colleagues worked on early prototypes and did some lab studies. In 2006 and 2007, they launched the "People Finder" application, which involved a couple of hundred users in multiple pilots, with laptops and some cell phones.

In 2008, they developed their first Facebook application, Locyoution, which was piloted by over one hundred users on their laptops.

In February, 2009, Sadeh and his colleagues launched Locaccino, a new Facebook app, which could scale to hundreds of thousand of users if successful.

Data from the team’s research indicates that the problem is not bad interfaces, or the number of rules defined, or even the time spent defining and refining those rules.

But Sadeh’s work and the data he has collected through a number of pilots are providing a number of powerful insights into what it takes to better support users as they define and maintain security policies. One element of functionality that has been shown by Sadeh and his teamto have a major impact on the ability of users to specify policies they feel more comfortable with is auditing functionality:

Auditing (‘feedback’) functionality that enables users to review decisions made by the rules they have specified and ask questions such as “Which rule in my policy is responsible for this particular decision” can help users better understand the behaviors their policies give rise to

The chart on "Evaluating Usefulness of Feedback," provides a summarized view of the impact of auditing (or “feedback”) functionality on user’s comfort and, ultimately, their "willingness to share their locations with others." What you are looking at in these two charts are the total number of hours per week different users were willing to share their location with others, depending on whether they had access to feedback functionality or not.. People who had access to the auditing functionality (“Feedback” chart) started to feel more comfortable and gradually relaxed their rules, utlimately resulting in more sharing than what was observed among users who did not have access to this functionality (“No Feedback” chart).

"That makes perfect sense. You see what is going on, you gain more confidence that the system is, in fact, not leading to any sort of abuse, and is not leading to any bad scenarios, and you end up sharing your location on a more regular basis,” Sadeh explained. “This is, by the way, one of those very simply types of functionality that none of the commercial applications out there today supporting location-sharing offers. So it is not surprising that when these applications get launched, tens of thousands of people download them, but these people only end up using the application for a few days.” Current location-sharing applications are very restrictive in the types of controls they allow their users to define and provide no such feedback functionality. The end result is very little sharing. In other words, the applications are of little value.

In his remarks, Sadeh went on to explore another challenging question, "How expressive should security or privacy policies be?"

Security and privacy policies can be viewed in the light of research on mechanism design. Through recent work, Sadeh and his colleagues has looked at the benefits afforded by more expressive mechanisms or more expressive security and privacy policies, when it comes to more accurately capturing the preferences of a user or organization... ” What are the sorts of features, and the types of attributes, I will need to make available in my language to my users, so that they can end up with policies that accurately capture their intended policies?"

" You can think of a security or privacy mechanism as being some sort of function that associates different actions with different sets of conditions subject to a collection of preferences expressed by a user. Work in mechanism design typically assumes a fully rational user. In other words, given some level of expressiveness in a policy language, we would assume that our user will be able to fully take advantage of that expressiveness. ... This is what is stated in this complex formula with the arg max. The notion of efficiency is a traditional one in mechanism design. Ideally we would want our policy, or mechanism, to be as efficient as possible, namely to do the best possible job capturing our user’s ground truth preferences. If however the policy language the user is given imposes restrictions on what he or she can express, the efficiency of the resulting mechanism may be less than 100%. In other words, the user may have to make some sacrifices. For instance, you may have to decide that you will not disclose your location to a given individual at all because you don’t have the ability to accurately specify the fine conditions under which you would have been willing to do so. Instead, given the restrictions of the available policy language, you decide that you will “play it safe” and simply deny all requests for your location from that individual. In general, one can define the efficiency of a security or privacy mechanism by looking at all possible scenarios and looking at the percentage of the time when the best policy a user can define (subject to the expressiveness of the available policy language) accurately captures what the user would like to do (e.g. sharing your location versus not sharing it). However rather than doing this for a single user, we will try to do this for the entire population of users for whom the mechanism is being designed. In practice, one can approximate this by looking for a representative sample of the target user population, collect their ground truth preferences and see how we can optimally configure policies to capture their preferences subject to different restrictions in the available policy language.” –For instance, in the case of location sharing applications, we can collect people’s ground truth preferences about sharing their locations with others and examine the impact of different levels of expressiveness in the language made available to users to specify the conditions under which they are willing to disclose their location to others. This means estimating the benefits afforded by a privacy language where users can specify rules that include restrictions tied to groups of people (e.g. friends, colleagues), restrictions tied to the day or time of the request, or to where the user is at the time his or her location is requested (....or some combination of the above).

What Sadeh and his colleagues found was that such insight could be applied to the design of any security or privacy mechanism to help users take fuller advantage of the expressiveness of the language through the interface. But real users are not fully rational. There is a point where users will say, "Well, I don't care. Yes, in principle I could get a higher efficiency, i.e., policies that more closely reflect what I really want, but perhaps I am not willing to invest the time, or no matter how hard I try, beyond six or seven rules I get completely confused."

At this point, Sadeh remarked, the next natural question arises, "What about machine learning? Could machine learning help us?"

In some of the team's early experiments, using case-based reasoning, it was clear that yes, in principle, machine learning could make "a huge difference."

"You might say this is wonderful, problem solved, let's just use your game theory results, add machine learning, and we're done. So why is it that this is not the case?'

"There is a slight problem," Sadeh points out, "and that is that we are talking about privacy and security. Machine learning can be used for lots of different things, and be more accurate than we humans can be, but machine learning is not 100% accurate and there lies the potential problem. It could end up making a decision that we don't feel comfortable with at all. Even if machine-learning gives us 99% accuracy, in security or privacy the remaining 1% could be devastating: you could be giving away national security secrets, or sensitive corporate data ... “

The problem, Sadeh adds, is that machine learning is traditionally configured as a “black box” technology, i.e., users are unlikely to understand the policies they end up with.

"So we are developing different families of machine learning techniques that essentially reconcile the power of machine learning, which is unquestionable, with the principle that ultimately the user has to remain in control. If the user loses control, you have not accomplished anything. “

"Can we develop technology that incrementally suggests policy changes to users? This leads us to the concept of user-controllable policy learning. The idea is that users are willing to provide feedback. We have seen that they are actually keen to have the auditing interface; and we have also seen that they are willing to provide feedback, e.g. thumbs up or thumbs down on decisions made by their current policy. They are not necessarily going to review every decision that was made, but they are willing to go back occasionally and provide feedback. ... So what we do is take this feedback, but instead of taking over, we develop suggestions we are going to present to the user and let the user decide whether or not to accept these suggestions. You might say, 'that sounds very easy, anybody can do that.' Well, there is another problem, in order for these suggestions to be meaningful, they have to be understandable: we have to develop suggestions that a user can relate to. If your suggestion is a new decision tree with a number of different branches, the user will stare at it for a very long time and not know what to do. Instead, we tend to limit ourselves to incremental changes to user policies. We start from the policy that the user has already defined, and see if we can learn over time small, incremental variations to the policy that can be presented to the user in a way that he or she can still relate to them. When you do that, the user can make a meaningful decision, as to whether or not he likes policy changes you are suggesting and gradually improve the accurary of his or her policy. If conditions suddenly change, the user can also directly manipulate his or her policy, because he or she continues to understand it. There is no need to wait for machine learning to adapt to the new situation. So you have the best of both worlds, with users and machine learning working hand in hand.”

Yes, patents are pending.

Some References

User-Controllable Security and Privacy Project

N. Sadeh, J. Hong, L. Cranor, I. Fette, P. Kelley, M. Prabaker, and J. Rao,
"Understanding and Capturing People's Privacy Policies in a Mobile Social
Networking Application", Journal of Personal and Ubiquitous Computing

P.G.Kelley, P. Hankes Drielsma, N. Sadeh, and L.F. Cranor, "User-Controllable Learning of Security and Privacy Policies", First ACM Workshop on AISec (AISec'08), ACM CCS 2008 Conference. Oct. 2008.

J.Tsai, P. Kelley, P. Drielsma, L. Cranor, J. Hong, and N. Sadeh. Who’s Viewed You? The Impact of Feedback in a Mobile-location Application. To appear in CHI '09.

Michael Benisch, Patrick Gage Kelley, Norman Sadeh, Tuomas Sandholm, Lorrie
Faith Cranor, Paul Hankes Drielsma, and Janice Tsai. The Impact of Expressiveness on the Effectiveness of Privacy Mechanisms for Location Sharing. CMU Technical Report CMU-ISR-08-139, December 2008

Other Relevant Links

CyLab Chronicles: Wombat, the Latest CyLab Success Story

CyLab Research Update: Locaccino Enables the Watched to Watch the Watchers

CyLab Chronicles: Q&A w/ Norman Sadeh

Some Other CyLab Seminar Notes

CyLab Seminar Series: Of Frogs, Herds, Behavioral Economics, Malleable Privacy Valuations, and Context-Dependent Willingness to Divulge Personal Info

CyLab Seminar Series Notes: Why do people and corporations not invest more in security?

CyLab Research Update: Basic Instincts in the Virtual World?

For information on the benefits of partnering with CyLab, contact Gene Hambrick, CyLab Director of Corporate Relations: hambrick at

Sunday, May 10, 2009

CyLab Business Risks Forum: Gary McGraw on Online Games, Electronic Voting and Software Security

"...take a look at the work that Brian Chess and I talked about at RSA, called the BSIMM. Our insight was very simple. We said, 'You know, there are people that have been doing software security for a decade, why don't we put on our anthropology hats, go out into the field and describe what we see.' We did an observation-based model ..." Gary McGraw on Building Security In Maturity Model (BSIMM), developed by Cigital and Fortify

CyLab Business Risks Forum: Gary McGraw on Online Games, Electronic Voting and Software Security

The CyLab Business Risks Forum is an virtual group of subject matter experts in business and government I have established to further inform and enrich the online content, both for CyLab's public site and for its partners-only portal.

These subject matter experts are tapped for the monthly Business Risks Forum events that supplement the ongoing CyLab Seminar Series. Their compelling talks are accessible on-line to CyLab's corporate partners; and from time to time, I publish "CyLab Seminar Series Notes" here on CyBlog. The insights of Business Risks Forum experts are also found in my Intelligence Briefings and Culture of Security features, and in the quarterly CyLab Focus, all available via the partners-only portal.

In addition, here on CyBlog, I will be posting interviews with individual Business Risks Forum participants, e.g., this one featuring Gary McGraw, as well as virtual Roundtables engaging several participants on a common theme, e.g., CyLab Virtual Roundtable on Cyber Security News Media.

As mentioned in RSA Conference 2009: Want to Play a Game? Gary McGraw & Expert Panel Explore the "Edge of Technology," although the whole of the conference was abuzz with babble about "Cloud computing," perhaps the most important insight came from Gary McGraw, CTO for Cigital, and author of several worthy tomes. His latest book, Exploiting On-Line Games (Addison-Wesley, 2007), co-authored with Greg Hougland, was also the subject of a panel McGraw led at RSA 2009.

In his opening remarks, McGraw declared “what we are talking about is the future of software security.” There are so many people out on the exhibit hall floor hawking the so-called Cloud, “even though they have no idea what it means.” But online games are massively distributed systems. “They put nine gigabyte globs in everybody’s box.”

I made a note to follow up with McGraw for the CyLab Business Risks Forum.
We caught up recently and asked him to expand on his comment, as well as to weigh on in some other vital issues.

-- Richard Power

Richard Power, CyLab: What does your work on the exploitation of on-line games tell us about "Cloud computing"? What lessons are there to be learned?

Gary McGraw, Cigital: “Online games, like World of Warcraft (WoW), are massively distributed. They have fat clients, millions of them, and server farms that are all over the planet. And it is what, in marketing land, we are calling Web 2.0, or ‘Cloud computing,’ or flavor of the day. So if we want a case study of the future, we need look no father than on-line games. If we understand what is going on with security, and cheating, and economics, and the law, and the technology concerns, all at the same time, we have a very small but important crystal ball.”

Power: So the bigger issue that you just framed there is that these are massively distributed systems, so this is not just cheating in games, it is what is opened up to outsiders about an individual's or organization's computing space?

McGraw: Yes, and let's get one notch more specific. My work usually concerns itself with software security. And in this case, those people who architected the games overlooked a very important and fundamental security idea: trust boundaries. They forgot, or overlooked, the notion that when you put code outside the trust boundaries on your potential attacker's machine, you shouldn't be surprised when your potential attacker becomes a real attacker. In distributed systems, it is hard enough to get it right when everybody is on the right side of the trust boundaries, or there are no trust boundaries. In this case, there are incentives for bad people to make a real trust boundary, and if you do not take that into account during your design, you get yourself in trouble.

Power: Are you getting recognition on this issue from developers in the game space?

McGraw: Not so much. We are doing some work with some game companies, and helping them with software security issues. My book, though, is aimed more at software security practitioners, in general, than it is at game developers. The game developers should solve their problems, and they are interested in this work and thinking about it, but I want the lessons to apply to all sorts of others. In the front matter of the book, there are quotes from various people, and one of my favorite ones is from someone in the US Air Force, who says "Gosh, massively distributed systems are the future, and for the next 25 years, they're going to be what we have to work on, so understanding this work means getting a leg up on that knowledge you are going to need."

Power: One area of cyber security that has been of great personal interest to me for a decade is electronic voting. And I know you have done some work in this area as well. So let's move from games to perhaps the most important thing in the life of a democratic society.

McGraw: Actually, I think it is the second most important thing -- the most important thing is individual liberty, and freedom from surveillance and from "pretend security." So "No Tyranny!" is number one, and voting that works is number two.

Power: OK, the phrase is too tempting, tell us about "pretend security."

McGraw: Well, maybe not "pretend" but "ineffective." Let me give you an example: when you walk outside the door in the UK, you are on camera. And it is not clear that being on camera all the time puts a serious dent in either crime or terrorism. So what you are trading off is your own personal liberty about where you are, who you are talking to and what you are doing for some "pretend security." That sort of thing I am just vigorously opposed to. ...

Power: So then, on to number two, the sanctity of the vote. I was concerned about this before the 2000 election. In the 1990s, I had a file I called "Security Breaches Waiting to Happen," I filled it with articles I teared out of IT business magazines. Each article in the file extolled the virtues of some coming application. I kept them on file so that I could refer back to them as each of these virtues were revealed to carry hidden vices. None of them concerned me more than the notion of electronic voting, particularly because of what we know about the lack of software security that goes into the development of such applications.

McGraw: Electronic voting is not something that I have been involved in intentionally. The reason for that is that you can't escape the politics. Unfortunately, it has very little to do with either the prophecies or the technologies. Getting these machines adopted or rejected turns out to just be a political exercise. ... The work of people like Jeremy Epstein, Avi Ruben, Ed Felton and others who have been very active is very important and we should pay attention to it. To me, the most important issue of all is an audit trail that you cannot tamper with. How you get that audit trail I do not really care ... I live in Clarke County, Virginia, and Clarke County has just a few thousand people. My friend Chip was running for the School Board and he lost by one vote. Something like 612 to 613. There were maybe 14 write-in votes. There was one accumulator. The way the recount went was "well, yesterday, the register said whatever, 607, and today it also says 607." Now that is the stupidest recount I have ever heard of. That is disconcerting. Frankly, I would rather have big arguments over hanging chads, because then you have a big pile of paper to look at.

Power: Have we made any progress?

McGraw: I think we have. California has decertified the worst of the worst. That's progress. And the way California goes, the rest of the nation follows eventually. ... I do not even know my registrar's political affiliation, and it does not matter. What does matter is that she chose that machine. And when I said, "hey, you know this machine has some problems, and here is a paper my friend Avi wrote," she got really angry. I said, "this is not about you, this is about technology." So even locally, it is political. Someone has made this decision, and it makes them embarrassed to admit that it is a piece of crap. ...

Power: So our conversation has moved from online games to electronic voting, and this just hints at the scope of the overarching issue, i.e., software security. Are we progressing? Are we spinning our wheels? What should vendors be doing differently? What should academic programs be doing differently? Perhaps the most important question is whether or not this is a field that can be formalized or a more of a gestalt?

McGraw: That is a good question, and a very important one. We have made obvious and heartening progress over the last decade. You may recall I wrote a book called Building Secure Software in 1999. At that time, the goal was to convince people that software security was important and that they should be talking about it. Ten years later, everyone is talking about it, and people are starting to wonder what we should do about it. In 2006, there was a wave of "what you should do about it," including my book Software Security and Howard and Lipner's book about the SDL ... At that point, you could follow one of the religions, or make up your own religion, or whatever. But basically you were cutting new ice on how to substantiate a culture of software security inside your large company. Well, fast-forward another three years, and take a look at the work that Brian Chess and I talked about at RSA, called the BSIMM. Our insight was very simple. We said, "You know, there are people that have been doing software security for a decade, why don't we put on our anthropology hats, go out into the field and describe what we see." We did an observation-based model, we said things like "monkeys eat bananas." Notice that the phrase "monkeys eat bananas" does not say whether it is good or bad, it does not say what color the bananas are, or whether or not the monkeys eat them while running, or steal them from one another. All it says is one observable fact. We tried very hard in the BSIMM to do the same thing. So we are not making value judgments about you should do software security, what we are doing is making clear observations about how ten very successful companies that have large scale software security initiatives have done it. I think that is the next phase of moving software security from religion to science, from alchemy to science. And I am very pleased now only with the fun we had making the work, and the gracious response we got from all of the participants, but also the response from the entire community. Whenever I present the work, it is just astounding. I am hugely optimistic that we are making some great progress, and that people are recognizing that and that they are hopping on board. It is a pretty exciting time.

Power: What does Cigital do?

McGraw: We are the largest software security consulting firm on the planet. We have offices in New York, Boston, Silicon Valley and in Virginia, near Washington, D.C. We have about a hundred twenty people. We provide all sorts of software security and software assurance services. And these services start at the most strategic level with helping people to formulate and then execute large scale software security initiatives. Some of the people in the BSIMM, but not all of them, are our customers, and we helped them set up their software security initiatives. We also help people do the nuts and bolts stuff. If people need to figure out how to review ten million lines of code, we will help them pick the tools, make the tools work, make it work automatically, integrate it with their bug tracking system. We do almost everything from the strategy of how to train your people and evolve a program all the way to penetration testing and code review.

Related Posts

Silver Bullet: Gary McGraw Interviews Virgil Gligor on Software Security and Other Vital Issues

RSA Conference 2009: Want to Play a Game? Gary McGraw & Expert Panel Explore the "Edge of Technology"

Fortify & Cigital Release BSIMM -- Integrating Best Practices from Nine Software Security Initiatives

CERT Podcast Series -- An Experience-Based Maturity Model for Software Security

In Paper for White House's Melissa Hathaway, Pradeep Khosla Urges an Information-Centric Approach to Cyber Security

Marine One Preparing to Land on South Lawn of White House

Quite possibly the biggest advantage of this approach, however, is that it allows business users — not the IT department — to truly own information, the rights associated with it, and its flow across the organization. Business can take responsibility for the security of data, since they feel they own the data itself. IT’s primary role will be to focus on computing devices, servers, networks, etc. Ultimately, information-centric data security can bring about better partnerships between business and IT since the lines of responsibility for security can be clearly drawn. Dr. Pradeep Khosla, founder of Carnegie Mellon CyLab on an Information-Centric Approach to Data Protection (ISA Comments to Hathaway on Improving Information Security Architecture)

In Paper for White House's Melissa Hathaway, Pradeep Khosla Urges an Information-Centric Approach to Cyber Security

At a recent hearing of the U.S. House of Representatives subcommittee on Communications, Technology and the Internet, Larry Clinton of the Internet Security Alliance testified on cyber security challenging that confront us. You can read the transcript of his testimony, and view the C-SPAN recording of it, on the ISA's site.

Among his supporting documents, Clinton offered several policy papers submitted "as part of the process for Melissa Hathaway’s 60-day review. "Board members of the Internet Security Alliance provided policy papers addressing some of the more difficult issues for the nation," Clinton said.

One of these policy papers was authored by CyLab's founder, Pradeep Khosla.

Here is a brief excerpt with a link to the full text:

Benefits of information-centric security
There are significant advantages to the information-centric security approach.

Protect data wherever it goes. The main benefit of an information-centric approach is the ability to protect the critical asset at all times, without interruption. Unlike perimeter defenses, it protects data from both insider and outsider threats and has the added benefits of device independence, network independence, and the ability to protect data in virtual environments. The data becomes smarter and self-defending, and is therefore much easier to share and collaborate with.

Align with business flows. Since policies for data use are always with their respective data, that data can be shared and collaborated on with more confidence. Legitimate users are not restricted to certain devices or networks since the appropriate security and access policies will ensure and enforce user rights regardless of where the data is. Quite possibly the biggest advantage of this approach, however, is that it allows business users — not the IT department — to truly own information, the rights associated with it, and its flow across the organization. Business can take responsibility for the security of data, since they feel they own the data itself. IT’s primary role will be to focus on computing devices, servers, networks, etc. Ultimately, information-centric data security can bring about better partnerships between business and IT since the lines of responsibility for security can be clearly drawn.

Reduce costs and complexity. The device independence of information-centric security significantly reduces the number of separate device-centric or network-centric security solutions. Subsequently, it lowers acquisition costs, maintenance burden, and man-hours associated with integrating multiple pieces of hardware and software. The information-centric approach protects critical data assets themselves, regardless of the device or network that carries them. An organization can secure data with far fewer solutions and with far less man power.

Increase end-user transparency. A major cause of data breaches is when legitimate users, while trying to be productive, work around security restrictions. Why would they do such a thing? Because following the security practices dictated is often inconvenient and creates more work for them. A security solution should remain as transparent as possible to end users. If user workflow is not hindered or altered, there is a significantly higher chance that the security program will be followed, and hence be more effective. Information-centric security can be extremely transparent, since the protection is with the data itself. Users do not have to explicitly make decisions about valid devices, network authentications — all these policies are contained in the data itself and can be managed centrally, so they can operate as usual without interference.

The requirements of an information-centric security platform

The basics of information-centric security exist today, and they’re very well suited to commercial and federal data protection scenarios, particularly those answering the protection requirements of regulatory compliance. The following are the basic requirements for a true information-centric approach to data protection.

1. Smart data with embedded policies
Before data can be adequately protected, it needs to be easily identified and managed; you can’t hit a security target if you don’t know where it is or what it looks like. If data objects are enriched with metadata tags that carry security policies, that data can be empowered to protect, replicate, or even delete itself, as required. As a result, an information-centric security approach could enable files to communicate their vital characteristics to the devices they pass through, as well as to other data objects, throughout the cycle of their lives.

2. Universal policy language
To realize truly effective and universal information-centric security, security policies and the codes describing them need to become industry-wide standards. As data travels between servers, laptops, and removable media, the policies that govern its protection need to be enforced, regardless of the platform. To make sure those policies are interpreted uniformly throughout an enterprise, there has to be a common policy language, embedded in the data itself, that every device can understand. That way, no matter where the data moves or resides, or the security solution that protects the data, the policies governing its access remain in force.

3. User-friendly implementations
Any successful information-centric security solution needs to be transparent. That is, users shouldn’t have to modify their work habits or change their business practices in order to benefit from the security solution. If they do, the solution will almost certainly fail. Why? Because most people will reject changes imposed on their familiar work patterns and bypass new security provisions they consider a nuisance. Nor should an organization’s current software applications or computer platforms have to be upgraded as part of the security deployment; an information-centric approach is capable of working with any device, on any platform, without requiring special patches or programming. Otherwise, implementation costs will be prohibitive. Beyond that, essentially all IT environments today utilize legacy systems to some extent. That makes it difficult for administrators to justify major overhauls solely for the sake of security. But by taking an information-centric versus a device-centric approach, it is possible to create a security solution that can apply to multiple IT environments and, at the same time, avoid having to inconvenience users.

Click here to read the full text of Information Security for the Next Century: Why we need an information-centric approach to data protection by Dr. Pradeep Khosla, Carnegie Mellon CyLab (ISA Comments to Hathaway on Improving Information Security Architecture)

Wednesday, May 6, 2009

CyLab Hosts Trusted Infrastructure Workshop, with Co-Sponsors including Giants of Industry and Government

Trusted Infrastructure Workshop (TIW) 2009
Advanced Summer School on Architectures for Trustworthy Computing
June 8-12, 2009 Pittsburgh, PA

When IT infrastructure technologies fail to keep pace with emerging threats, we can no longer trust them to sustain the applications we depend on in both business and society at large. Ranging from Trusted Computing to machine virtualization, new hardware architectures, and new network security architectures, trusted infrastructure technologies attempt to place security into the very design of commercial off-the shelf technologies.

TIW 2009 is aimed at all researchers in the field of IT security with an interest in systems and infrastructure security, as well as younger master's or PhD students who are new to the field. Funding is available to support students.

CyLab is proud to be hosting TIW 2009.

CyLab's co-sponsors are the National Science Foundation (NSF), Hewlett-Packard Labs, Sun Microsystems, Seagate, IBM and Fujitsu.

The TIW 2009 program includes --

Four keynote lectures

Seven technology lectures: Trusted computing architecture, TPM module, attestation, SW-based attestation, virtualization security, network security, and trusted storage.

Four research workshops: HW security, attestation in practice, OS security, verification and formal methods.

Three hands-on labs: TPM, trusted virtualization, trusted network connect.

Several social events and networking with other researchers are planned.

Speakers will be leaders from academia, industry, and government are delivering the lectures, labs, and workshops.

CyLab will be hosting this event.

Space is limited. For details and registration, are available at the TIW 2009 web site.