From Biometrics to BSIMM , & "50 Hurricanes Hitting At Once!" -- A Report on the Sixth Annual Partners Conference

Image: CyLab Biometrics Center


From Biometrics to BSIMM, & "50 Hurricanes Hitting At Once!" -- A Report on the Sixth Annual Partners Conference

by Richard Power

Throughout 2009, I have made a point of attending some important security conferences and delivering reports on what I saw and heard, some of these reports are posted here on CyBlog and in the Intelligence Briefing section of the CyLab Partners-Only Portal. The events covered include RSA, Black Hat Briefings and USENIX Security Symposium, as well as our own SOUPS and Mobile Health Workshop. (I have included a listing of the summary reports below in “Conference Coverage.”)

It is wonderful to finish off the series with a report on the CyLab Partners Conference. It is an event accessible via invitation only, and developed as an opportunity for CyLab’s corporate partners to immerse themselves in an audacious program. The conference's agenda, like CyLab's research program itself, is sweeping in its scope and impressive in its implications.

The sixth annual CyLab Corporate Partners Conference, held on the main campus of Carnegie Mellon University (Pittsburgh, Pennsylvania) from Wednesday, October 14 to Friday, October 16, offered a deep dive into one of the world’s premier cyber security research programs. Over the span of two and a half days, attendees immersed themselves in presentations and panel discussions on a broad spectrum of research areas, including:

• Corporate Governance
• Secure Home Computing
• Usability of Security and Privacy Techniques
• Security of Cyber-Physical Systems
• Secure Mobile Systems and Networks
• Trusted Computing Platforms and Devices
• Secure Software Engineering
• Digital Forensics

The rich conference agenda also featured two keynotes, one from former White House aide Melissa Hathaway, and the other from Gary McGraw, CTO of Citigal, Inc.

In her remarks at lunch on Wednesday, Hathaway spoke of the vital role of business, government and the individual and emphasized the threat to critical infrastructure:

The specter of a "digital 9/11" is what still keeps the former U.S. acting cybersecurity czar up at night, Melissa Hathaway told a gathering of Carnegie Mellon University's CyLab corporate partners ...
To illustrate one possibility, Hathaway referred to the relatively low-level denial-of-service attacks that hit some federal Web sites for several days beginning over the July Fourth weekend.
A more powerful barrage that used more points of attack, perhaps against private-sector targets, could cause $700 billion in damage, she said.
"That's the equivalent of 50 hurricanes hitting at once," Hathaway said. Mike Cronin, Partners of Carnegie Mellon's CyLab warned that 'digital 9/11' threat growing, Pittsburgh Tribune Review, 10-15-09

At dinner on Thursday, McGraw championed the Building Security in Maturity Model (BSIMM) that McGraw's Citigal developed and is now promoting with SANS Institute, through BSIMM Begin

BSIMM is based on large-scale software security initiatives in nine enterprises: four financial services companies, three independent software vendors and two technology companies.

As McGraw remarked in his keynote, "BSIMM is not about good or bad ways to eat bananas or banana best practices. BSIMM is about observations."

The power of the observations offered by McGraw and his colleagues is in their practicality: e.g., "Ten Surprising Things," including "Nobody uses WAFs," "QA can't do software security," "PEN testing is diminishing," etc., and "Ten Things Everybody Does," including "Evangelist role,""SSG does ARA" and "good network security," etc.

BSIMM will help you know where your enterprise stands, and what direction you might want maneuver it. Perhaps most important, through BSIMM Begin, it is intended to be ongoing and participatory:

"BSIMM Begin aims to significantly broaden data collection. To keep the survey manageable, the scope has been limited to the BSIMM Level 1 activities. The goals of this survey are two-fold: to provide participants with a solid understanding of where they stand with respect to foundational software security activities; and to provide an understanding of where they stand relative to everyone else that participates. BSIMM Begin will broaden the collective understanding of what "keeping up" really means." Software Security Self-Measurement with BSIMM Begin Introduced by Cigital and The SANS Institute, 10-8-09

The BSIMM Begin survey can be accessed from the landing site: http://bsi-mm.com/begin/

For more information, read McGraw's Software [In]security: The Building Security In Maturity Model (BSIMM) in InformIT (3/16/09).

The body of the conference was devoted to updates on the diverse aspects of Cylab's bold research program.

For example, Marios Savvides, Director of Cylab’s Biometrics Center, and one of the four scientists of the Office of the Director of National Intelligence Center of Academic Excellence in S&T in Identity Sciences, delivered a report on his team's "Multi-Biometrics Research Effort."

Savvides' compelling presentation showcased how his research is tackling some of the field's most urgent and vital challenges, from Long Range Iris Recognition on the Move to Soft Biometrics to Automatic Landmarking Frontal Faces to 3-D Face Reconstruction from Single Images.

In his summary, Savvides outlined his Center's current status and goals, including:

-- Developed several key technologies of HIGHEST interest to the USG.

-- Already transitioning one technology to USG (FBI’s Universal Face Workstation)

-- Working with MIT-LL to develop Government Owned Face Recognition (GOTS-FR).

-- Working on refining and developing Iris acquisition and other technology to the USG for two more success transition stories.

"We collaborate and bridge across many USG agencies," Savvides concluded, "Our goal is to support the USG in developing key enabling technologies to deter terrorism and aid the war fighter."

The three presentations briefly cited here offer only a few glimpses into the scope of the sessions stretching over the two and a half day conference.

NOTE: A full archive of presentations, student posters, photo gallery and videos is accessible to CyLab Partners only from the Partners Portal.

Conference Coverage

A Report from the 18th USENIX Security Symposium: Android Security, Naked Keystrokes, Selling Viagra, Crying Wolf & More!

A Report from BlackHat Briefings (Las Vegas 2009): From Parking Meters to the Cloud, from SMS to Smart Grids, “Everything is Broken …”

Reflections on SOUPS 2009: Between Worlds, Cultivating Superior Cleverness, Awaiting a Shift in Consciousness

RSA Conference 2009: Summary of Posts

CyLab MRC's Martin Griss Declares,"I Do Not Want Us to be Just Another Big Consortium, I Want Us to Do Something"

NOTE: Full texts of my reports from USENIX, Blackhat and the Sixth Annual CyLab Partners Conference are available to CyLab corporate partners via the Partners Portal.

CyLab Seminar Series Notes: The Evolution of A Hacking Tool, Moxie Marlinspike on SSLstrip

"Personally, I find that when looking for security vulnerabilities, a good place to start are those places where developers probably don't know what they are doing, but feel really good about the solution they implement." Moxie Marlinspike, www.thoughtcrime.org

CyLab Seminar Series Notes: The Evolution of A Hacking Tool, Moxie Marlinspike on SSLstrip

[NOTE: CyLab's weekly seminar series provides a powerful platform for highlighting vital research. The physical audience in the auditorium is composed of Carnegie Mellon University faculty and graduate students, but CyLab's corporate partners also have access to both the live stream and the archived content via the World Wide Web. From time to time, CyBlog will whet your appetite by offering brief glimpses into these talks. Here are some of my notes from a talk delivered by independent researcher Moxie Marlinspike on 5-18-09. -- Richard Power]

In the late 1990s, as electronic commerce (and cyber crime) started heating up, I got many calls from reporters asking the same question over and over again, “Is it safe to use our credit cards over the Web?”

In response, I would point out that this was the wrong question. The right question, the bigger question, was lost on them, i.e., “What is going to happen to your credit card information at the other end of the transaction?”

The point I was driving home was that the turning over of insecure server brimming with credit card records was going to be the cyber crime of the decade, not the petty cyber theft of one credit card transaction.

As the years passed, and the aggregation of tens of thousands of credit cards on those servers turned into aggregates of hundreds of thousands of credit cards, and then in turn, millions and tens of millions of credit cards, my warning turned into an everyday fact of life in cyberspace.

But back then, for everyone one of us, who was trying to direct the attention of the public (and those paid to inform it) away from the vulnerability of a single transaction to the greater threat of digital stagecoach robberies, there were a dozen vendors and consultants waiting in line to answer those reporters in a much more simplistic (and misleading) way, “Yes, of course, it is safe, because we have SSL.”

I was reminded of this subplot in the cyber risk timeline of the last decade recently, as I watched Moxie Marlinspike’s CyLab Seminar Series presentation on “New Tricks for Defeating SSL in Practice.”

Because, unlike Moxie’s earlier contribution, SSLsniff, which issues Man-in-the-Middle (MITM) attacks on SSL, the fascinating work that Moxie talked us through in this particular presentation, SSLstrip is not really about breaking SSL, it is about getting around SSL by exploiting weaknesses in the enveloping framework in which SSL is operating.

That’s what reminded me of those vendors and consultants who wanted to put security concerns to rest by assuring the public and the press that SSL was in place. It was the wrong answer to the wrong question then, and it, and its security marketplace equivalents, still are now.

And just as the real takeaway from Moxie’s talk was not so much the attacks he demonstrated, although they are indeed compelling, it is that independent research and pure hacker kulchur is alive and well, at www.thoughtcrime.org and elsewhere in the shadows of cyberspace.

Here are some brief excerpts from Marlinspike's talk:

In his opening remarks, Moxie observed, "The title of the talk is kind of a misnomer because I am going to be talking about SSL in the context of the Web, so the title should be 'Tricks for Defeating HTTPS in Practice.'

During his discussion of the background, that is the development of SSLsniff, Moxie shared some insight into how to succeed in such research, "Personally, I find that when looking for security vulnerabilities, a good place to start are those places where developers probably don't know what they are doing, but feel really good about the solution they implement."



Before moving on to SSLstrip, Moxie reviewed SSLsniff's ongoing relevance:

"You'd be surprised who still doesn't check basic constraints. I have promised to talk more about this in the future, and I will.

"Even when people got warning dialogs in browsers that had been fixed, most of the time they'd just click through them. There is an interesting little sub-point there, most browsers would start validating certificates, and as soon as they found a problem, they would stop. They would pop up a dialog and say, 'there is a problem, do you want to continue?' You could do these interesting tricks. You could create a totally bogus certificate that expired two minutes ago. It has invalid signature, everything about it is wrong. But the second thing that most SSL implementations did was check the time stamp, to make sure it hadn't expired. So what would happen is that it would check the name on the certificate, which of course matches whatever you are trying to intercept, and then it would check the time stamp and say, 'Oh, this expired two minutes ago.' And it would pop up a dialog to the user, saying 'There is something wrong, this ticket expired two minutes ago.' So a lot of users would be thinking "Oh, everything is fine, it's just that this ticket expired just a couple of minutes ago, they just haven't got around to issuing a new certificate yet, this should be fine,' and they would click through it.

"SSLsniff is still useful as a general MITM tool for SSL. The folks who did the MD5 hash collision stuff last December (2008) used sslsniff to hijack connections once they'd gotten a CA cert."

And in concluding his remarks on SSLsniff, Moxie gave fair warning, "There are other uses yet to be disclosed another day."

Moving on to explore what led to his development of the SSLsniff tool, Moxie drew a distinction between "browsers then and now," pointing out that in the past they employed a "postivie feedback system" to re-assure the user about the presence of security (i.e., "a number of indicators deployed to designate that a page is secure, a proliferation of little lock icons, a URL bars that turn gold") as opposed to the "negative feedback system" used now (i.e., less emphasis on sites being secure, e.g., "the proliferation of little locks has been toned down, and Firefox's gold bar is gone," increased emphasis on alerting users to problems, e.g., "a maze of hoops that users have to jump through in order to access sites with certificates that aren't signed by a CA."

In assessing the implications of this shift from the attacker's perspective, Moxie concluded: "If we trigger the negative feedback, we're screwed. If we fail to trigger the positive feedback, it's not so bad."

In looking for opportunities to apply this maxim, Moxie next looked at the relationship between SSL and HTTP.

"How do people use SSL? Nobody types 'https://' or 'http' for that matter. People generally encounter SSL either by clicking on links or through 302s, which means that people only encounter SSL through HTTP. ... We can attack both of these points through an HTTP MITM."

In rapid success- ion, Moxie led the attendees through his iteration of SSLstrip.
In the first cut, SSLstrip simply attacked HTTP instead of SSL, with promising end results, "The server never knows the difference. Everything looks secure on their end. The client doesn't display any of the disastrous warnings that we want to avoid. We see all the traffic."

"We've managed to avoid the negative feedback, but some positive feedback would be good too," Moxie said, pushing further, "People seem to like the little lock icon thing, so it'd be nice if we could get that in there too."

So in the next iteration, Marlinspike's program would respond to a favricon request for a URL it had stripped by sending back its own little padlock icon.

In his further develop- ment of the program, Moxie also dealt with the problem of sessions: "Sessions expire, and it's not always clear when or why, but they don't usually expire right in the middle of an active session. So what we do now: When we start a MITM against a network, strip all the traffic immediately, but don't touch the cookies for 5 min (or some specified length of time). As the cookies go by, make note of the active sessions. After the time is up, start killing sessions, but only new sessions that we haven't seen before. These should be the “long running” sessions that won't be seen as suspicious should they disappear."

"The results were kind of astound- ing. In 24 hours, 114 Yahoo logins, 50 Gmail logins, 42 secure posts to Ticket Master, 14 Rapidshare accounts, 13 Hotmail accounts, 9 Paypal logins, 9 LinkedIn accounts, and 3 Facebook accounts, and many more. That means in 24 hours, I got 117 email accounts, 16 credit card numbers, 7 paypal logins, over 300 other miscellaneous secure logins. So I wondered, 'That's a lot data, but what is the success rate? How many people didn't submit their data? How many people got to whatever it is they wanted to log into and then didn't log in? So I modified it a little bit and ran it again for another 24 hour period, and this time, I logged the number of people who browsed to a page which would have had secure post and then didn't post data. So how many people browsed to the g-mail login and then didn't login? How many people browsed to the paypal login and then didn't login? ... I got a comparable number of secure posts, but the question is how many people balked? Zero. In a 24 hour period, not a single person browsed to a page with a secure post and didn't post data."

When Marlinspike presented these results at Blackhat in D.C., one of the responses he received was, "Well, OK, this is a problem with 'User Education,' users are ignorant, they do not know how to use the web."

Moxie did not think that "user education" was the real issue, and rightfully so.

To demonstrate the point, he went on to deliver the presentation at Blackhat Europe in Amsterdam, and before he did he ran SSLstrip on the network at the conference, intercepted over one hundred secure logins in a thirty minute period and selected ten of the passwords collected to include on a slide for display during his talk.

Bravo.

Stay tuned. Marlinspike is working on more.

Among the lessons learned from this research, "Lots of times the security of HTTPS comes down to the security of HTTP, and HTTP is not secure.

Some Other CyLab Seminar Notes

CyLab Seminar Series Notes: User-Controllable Security and Privacy -- Norman Sadeh asks, "Are Expectations Realistic?"

CyLab Seminar Series: Of Frogs, Herds, Behavioral Economics, Malleable Privacy Valuations, and Context-Dependent Willingness to Divulge Personal Info

CyLab Seminar Series Notes: Why do people and corporations not invest more in security?

CyLab Research Update: Basic Instincts in the Virtual World?

For information on the benefits of partnering with CyLab, contact Gene Hambrick, CyLab Director of Corporate Relations: hambrick at andrew.cmu.edu

CyLab Seminar Series Notes: User-Controllable Security and Privacy -- Norman Sadeh asks, "Are Expectations Realistic?"

"As we all realize on a daily basis, application developers have great expect- ations about what we users are capable of doing. They expect us to be able to properly configure the firewall on our home computer and virus settings on our cell phone. As enterprises move towards more agile and decentralized business practices, developers also expect us to configure increasingly complex access control policies at work. Are these expectations realistic? If they are not, how much trouble are we in and what can we do about it?"

CyLab Seminar Notes: User-Controllable Security and Privacy -- Norman Sadeh asks, "Are Expectations Realistic?"

[NOTE: CyLab's weekly seminar series provides a powerful platform for highlighting vital research. The physical audience in the auditorium is composed of Carnegie Mellon University faculty and graduate students, but CyLab's corporate partners also have access to both the live stream and the archived content via the World Wide Web. From time to time, CyBlog will wet your appetite by offering brief glimpses into these talks. Here are some of my notes from a talk delivered by Norman Sadeh on 3-16-09. Sadeh's team of collaborators in this important research includes faculty members Jason Hong, Lorrie Cranor, Lujo Bauer, Tuomas Sandholm, post docs Paul Hankes Drielsma, Eran Toch, Jinghai Rao, and PhD students Patrick Kelley, Jialiu Lin, Janice Tsai, Michael Benisch and Ram Ravichandran. -- Richard Power]

Can users be expected to effectively specify their policies? Do people even what policies they want or need? Even if they did, could they articulate these policies? What if policies evolve over time? Are we always willing to invest enough time to have perfect policies or are there important trade-offs between user burden and policy accuracy? Can we develop technologies that mitigate these potential problems and empower users to more accurately and efficiently specify security and privacy policies?

To shed some light on these compelling questions, Norman Sadeh shared some insights into data from lab and field research on mobile social networking applications.

An example is a location sharing application that uses GPS and WiFi triangulation on laptops and cell phones and allows people to share their locations with friends, families, colleagues, and ... Well, that is one of the big issues that arises in this space, who exactly are you sharing this information with? And to what extent can you control access to it?

According to Sadeh, although many such applications have been released over the past several years, adoption has been rather limited. Early on, Sadeh and his team noticed that users had great difficulty specifying location sharing privacy policies that accurately reflected their preferences.

“So what’s going on? Is it because these applications have bad user interfaces? Do people who define more privacy rules do better? Do the people who spend more time defining and refining their rules do better?” Sadeh continued. “Location sharing applications seemed to be a very good domain to study these and related issues. Because, at the end of the day, the problems are the same, whether you are trying to configure a firewall at home or at work, or you are trying to configure social networking policies. Ultimately, the question is whether we can empower users (both expert users and lay users) to specify combinations of rules that enact the behaviors they really want to enforce?”

From 2003 to 2005, Sadeh and his colleagues worked on early prototypes and did some lab studies. In 2006 and 2007, they launched the "People Finder" application, which involved a couple of hundred users in multiple pilots, with laptops and some cell phones.

In 2008, they developed their first Facebook application, Locyoution, which was piloted by over one hundred users on their laptops.

In February, 2009, Sadeh and his colleagues launched Locaccino, a new Facebook app, which could scale to hundreds of thousand of users if successful.

Data from the team’s research indicates that the problem is not bad interfaces, or the number of rules defined, or even the time spent defining and refining those rules.

But Sadeh’s work and the data he has collected through a number of pilots are providing a number of powerful insights into what it takes to better support users as they define and maintain security policies. One element of functionality that has been shown by Sadeh and his teamto have a major impact on the ability of users to specify policies they feel more comfortable with is auditing functionality:


Auditing (‘feedback’) functionality that enables users to review decisions made by the rules they have specified and ask questions such as “Which rule in my policy is responsible for this particular decision” can help users better understand the behaviors their policies give rise to

The chart on "Evaluating Usefulness of Feedback," provides a summarized view of the impact of auditing (or “feedback”) functionality on user’s comfort and, ultimately, their "willingness to share their locations with others." What you are looking at in these two charts are the total number of hours per week different users were willing to share their location with others, depending on whether they had access to feedback functionality or not.. People who had access to the auditing functionality (“Feedback” chart) started to feel more comfortable and gradually relaxed their rules, utlimately resulting in more sharing than what was observed among users who did not have access to this functionality (“No Feedback” chart).

"That makes perfect sense. You see what is going on, you gain more confidence that the system is, in fact, not leading to any sort of abuse, and is not leading to any bad scenarios, and you end up sharing your location on a more regular basis,” Sadeh explained. “This is, by the way, one of those very simply types of functionality that none of the commercial applications out there today supporting location-sharing offers. So it is not surprising that when these applications get launched, tens of thousands of people download them, but these people only end up using the application for a few days.” Current location-sharing applications are very restrictive in the types of controls they allow their users to define and provide no such feedback functionality. The end result is very little sharing. In other words, the applications are of little value.

In his remarks, Sadeh went on to explore another challenging question, "How expressive should security or privacy policies be?"

Security and privacy policies can be viewed in the light of research on mechanism design. Through recent work, Sadeh and his colleagues has looked at the benefits afforded by more expressive mechanisms or more expressive security and privacy policies, when it comes to more accurately capturing the preferences of a user or organization... ” What are the sorts of features, and the types of attributes, I will need to make available in my language to my users, so that they can end up with policies that accurately capture their intended policies?"

" You can think of a security or privacy mechanism as being some sort of function that associates different actions with different sets of conditions subject to a collection of preferences expressed by a user. Work in mechanism design typically assumes a fully rational user. In other words, given some level of expressiveness in a policy language, we would assume that our user will be able to fully take advantage of that expressiveness. ... This is what is stated in this complex formula with the arg max. The notion of efficiency is a traditional one in mechanism design. Ideally we would want our policy, or mechanism, to be as efficient as possible, namely to do the best possible job capturing our user’s ground truth preferences. If however the policy language the user is given imposes restrictions on what he or she can express, the efficiency of the resulting mechanism may be less than 100%. In other words, the user may have to make some sacrifices. For instance, you may have to decide that you will not disclose your location to a given individual at all because you don’t have the ability to accurately specify the fine conditions under which you would have been willing to do so. Instead, given the restrictions of the available policy language, you decide that you will “play it safe” and simply deny all requests for your location from that individual. In general, one can define the efficiency of a security or privacy mechanism by looking at all possible scenarios and looking at the percentage of the time when the best policy a user can define (subject to the expressiveness of the available policy language) accurately captures what the user would like to do (e.g. sharing your location versus not sharing it). However rather than doing this for a single user, we will try to do this for the entire population of users for whom the mechanism is being designed. In practice, one can approximate this by looking for a representative sample of the target user population, collect their ground truth preferences and see how we can optimally configure policies to capture their preferences subject to different restrictions in the available policy language.” –For instance, in the case of location sharing applications, we can collect people’s ground truth preferences about sharing their locations with others and examine the impact of different levels of expressiveness in the language made available to users to specify the conditions under which they are willing to disclose their location to others. This means estimating the benefits afforded by a privacy language where users can specify rules that include restrictions tied to groups of people (e.g. friends, colleagues), restrictions tied to the day or time of the request, or to where the user is at the time his or her location is requested (....or some combination of the above).

What Sadeh and his colleagues found was that such insight could be applied to the design of any security or privacy mechanism to help users take fuller advantage of the expressiveness of the language through the interface. But real users are not fully rational. There is a point where users will say, "Well, I don't care. Yes, in principle I could get a higher efficiency, i.e., policies that more closely reflect what I really want, but perhaps I am not willing to invest the time, or no matter how hard I try, beyond six or seven rules I get completely confused."

At this point, Sadeh remarked, the next natural question arises, "What about machine learning? Could machine learning help us?"

In some of the team's early experiments, using case-based reasoning, it was clear that yes, in principle, machine learning could make "a huge difference."

"You might say this is wonderful, problem solved, let's just use your game theory results, add machine learning, and we're done. So why is it that this is not the case?'

"There is a slight problem," Sadeh points out, "and that is that we are talking about privacy and security. Machine learning can be used for lots of different things, and be more accurate than we humans can be, but machine learning is not 100% accurate and there lies the potential problem. It could end up making a decision that we don't feel comfortable with at all. Even if machine-learning gives us 99% accuracy, in security or privacy the remaining 1% could be devastating: you could be giving away national security secrets, or sensitive corporate data ... “

The problem, Sadeh adds, is that machine learning is traditionally configured as a “black box” technology, i.e., users are unlikely to understand the policies they end up with.

"So we are developing different families of machine learning techniques that essentially reconcile the power of machine learning, which is unquestionable, with the principle that ultimately the user has to remain in control. If the user loses control, you have not accomplished anything. “

"Can we develop technology that incrementally suggests policy changes to users? This leads us to the concept of user-controllable policy learning. The idea is that users are willing to provide feedback. We have seen that they are actually keen to have the auditing interface; and we have also seen that they are willing to provide feedback, e.g. thumbs up or thumbs down on decisions made by their current policy. They are not necessarily going to review every decision that was made, but they are willing to go back occasionally and provide feedback. ... So what we do is take this feedback, but instead of taking over, we develop suggestions we are going to present to the user and let the user decide whether or not to accept these suggestions. You might say, 'that sounds very easy, anybody can do that.' Well, there is another problem, in order for these suggestions to be meaningful, they have to be understandable: we have to develop suggestions that a user can relate to. If your suggestion is a new decision tree with a number of different branches, the user will stare at it for a very long time and not know what to do. Instead, we tend to limit ourselves to incremental changes to user policies. We start from the policy that the user has already defined, and see if we can learn over time small, incremental variations to the policy that can be presented to the user in a way that he or she can still relate to them. When you do that, the user can make a meaningful decision, as to whether or not he likes policy changes you are suggesting and gradually improve the accurary of his or her policy. If conditions suddenly change, the user can also directly manipulate his or her policy, because he or she continues to understand it. There is no need to wait for machine learning to adapt to the new situation. So you have the best of both worlds, with users and machine learning working hand in hand.”

Yes, patents are pending.

Some References

User-Controllable Security and Privacy Project

N. Sadeh, J. Hong, L. Cranor, I. Fette, P. Kelley, M. Prabaker, and J. Rao,
"Understanding and Capturing People's Privacy Policies in a Mobile Social
Networking Application", Journal of Personal and Ubiquitous Computing.

P.G.Kelley, P. Hankes Drielsma, N. Sadeh, and L.F. Cranor, "User-Controllable Learning of Security and Privacy Policies", First ACM Workshop on AISec (AISec'08), ACM CCS 2008 Conference. Oct. 2008.

J.Tsai, P. Kelley, P. Drielsma, L. Cranor, J. Hong, and N. Sadeh. Who’s Viewed You? The Impact of Feedback in a Mobile-location Application. To appear in CHI '09.

Michael Benisch, Patrick Gage Kelley, Norman Sadeh, Tuomas Sandholm, Lorrie
Faith Cranor, Paul Hankes Drielsma, and Janice Tsai. The Impact of Expressiveness on the Effectiveness of Privacy Mechanisms for Location Sharing. CMU Technical Report CMU-ISR-08-139, December 2008

Other Relevant Links

CyLab Chronicles: Wombat, the Latest CyLab Success Story

CyLab Research Update: Locaccino Enables the Watched to Watch the Watchers

CyLab Chronicles: Q&A w/ Norman Sadeh

Some Other CyLab Seminar Notes

CyLab Seminar Series: Of Frogs, Herds, Behavioral Economics, Malleable Privacy Valuations, and Context-Dependent Willingness to Divulge Personal Info

CyLab Seminar Series Notes: Why do people and corporations not invest more in security?

CyLab Research Update: Basic Instincts in the Virtual World?

For information on the benefits of partnering with CyLab, contact Gene Hambrick, CyLab Director of Corporate Relations: hambrick at andrew.cmu.edu

CyLab Seminar Series: Of Frogs, Herds, Behavioral Economics, Malleable Privacy Valuations, and Context-Dependent Willingness to Divulge Personal Info

[NOTE: CyLab's weekly seminar series provides a powerful platform for the highlighting vital research. The physical audience in the auditorium is composed of Carnegie Mellon University faculty and graduate students, but CyLab's corporate partners also have access via the World Wide Web. On a frequent basis, CyBlog will wet your appetite by offering brief glimpses into these talks. Here are my notes from a talk delivered by Alessandro Acquisti on 4-6-09. -- Richard Power]

The boiling frog story states that a frog can be boiled alive if the water is heated slowly enough — it is said that if a frog is placed in boiling water, it will jump out, but if it is placed in cold water that is slowly heated, it will never jump out. Wikipedia

CyLab Seminar Series Notes: Of Frogs & Herds, Behavioral Economics, Malleable Privacy Valuations, & Context-Dependent Willingness to Divulge Personal Info

Carnegie Mellon University CyLab researcher Alessandro Acquisti, Assistant Professor of Information Technology and Public Policy at Carnegie Mellon’s H. John Heinz III College, always warns his students not to trust Wikipedia; nevertheless, this Boiling Frog story, whether apocryphal or not, provides a useful foil for some fascinating research.

Working with collaborators Leslie John and George Loewenstein, Acquisti has been delving into the mysteries of privacy and security from a behavioral economics perspective. Aquisti and his colleagues see a great application for this particular discipline in the exploration of privacy and security decision-making.

“Behavioral economics is a field of economics that combines psychology plus more traditional economic thinking to understand why people really make decisions, and why certain decisions are sometimes sub-optimal, inconsistent and paradoxical,” Acquisti explains.

Acquisiti’s seminar focused on three studies his team had conducted:

Study 1: The “frog” effect (or lack thereof) on information disclosure.
Study 2: The “herding” effect on information disclosure.
Study 3: The effect of framing on privacy values.

“We feel we are uncovering something novel and peculiar to the privacy area.”

These notes will focus on Study 1.

The research on the “frog” effect explores the impact of privacy intrusions on the propensity to disclose, and in particular, what A sees as “a crucial question, one of the most interesting questions, and one of the most difficult to answer, “Do privacy intrusions (and how we react them) alert or rather desensitize individuals to privacy concerns”

“We live in a society in which every week in the media there is some new event, e.g., exposing personal data on millions of consumers, NSA is spying on domestic communications, passport records of important people are being accessed illegally, etc. Does all of this exposure to privacy intrusions make people believe that well, there is no privacy any longer, so I stop caring; there is so much information out there that there is nothing I can do about it? Or, in fact, is it the opposite, there is so much discussion and so much evidence of intrusions that it will create the opposite effect, at a certain moment, subjects start saying this is too much, enough is enough, and start reacting?”

“This is a difficult question to answer. Because you have to combine longitudinal data and effect, the age effect and the cohort effect, e.g., ‘Do you people use Facebook so much because they are young (age effect) or because they are born in a certain culture (cohort effect)?’ And in the absence of a longitudinal study that tracks people over ten, twenty or thirty years, what we did was simulate a scenario of privacy intrusions by creating a survey of questions with different levels of sensitivity. The sensitivity of the questions ranged from the tame, e.g., ‘Have you ever failed to turn the lights out at home or at work when you left?’ to the intrusive, e.g., ‘Have you ever had sex with the current husband, wife or partner of a friend?’

The survey included ten tame questions, ten moderately intrusive questions, and ten intrusive questions. The design of the survey included randomly assigning respondents to eight different conditions, e.g., the order of the questions, i.e., from tame to intrusive or from intrusive to tame as well as in pseudo-random or sudden order. It was framed as a survey on “ethical behavior.’ Another important factor was when respondents were asked for identifying information, i.e., at the beginning or at the end. “As you can imagine, people were much more willing to give an e-mail address before seeing the survey then after seeing what the survey was about.” The pool of respondents consisted of online readers of the New York Times (NYT), and the survey was linked to from the blog of a NYT op-ed columnist.

“We manipulated the order in which questions were presented to survey participants. Some subjects would see a survey which started with very tame questions and then increasingly became very intrusive. While other subjects started from the very intrusive questions and then went down to questions of lower and lower sensitivity.”


Two hypotheses were tested: the “Frog” hypothesis and the “Coherent Arbitrariness” hypothesis. The “Frog” hypothesis says that people will admit to sensitive behavior more often when they get “warmed up” by getting the tame questions before the more intrusive ones. The “Coherent Arbitrariness” says that people will admit to sensitive behavior less often when they “warmed up” by the survey, because their expectations about the intrusiveness of the survey will be established early on.

The “Coherent Arbitrariness” hypothesis was the one supported.

The “Frog” hypothesis was strongly rejected by the data.

Subjects in the increasing condition admitted to sensitive, moderate, and tame behaviors less often than subjects in other conditions

Subjects in the decreasing condition admitted to sensitive behaviors more often than subjects in other conditions

Bottom line: starting a survey with tame questions, then increasing their intrusiveness, inhibits information disclosure. Sensitive behaviors were more frequently admitted to when asked first.

To read a CyLab Chronicles Q&A with Alessandro Acquisti, click here.

For information on the benefits of partnering with CyLab, contact Gene Hambrick, CyLab Director of Corporate Relations: hambrick at andrew.cmu.edu

Spotlight On: Programming Techniques Used as an Insider Attack Tool

How they strike
Nine of the insiders in these cases inserted malicious code with the intent of causing harm to their organization or to individuals. Six of the insiders used logic bombs to carry out their attacks. Other attacks methods included
• social engineering
• sabotaging backup tapes
• compromising accounts
• deleting and modifying log files
• unauthorized access
• intentionally deploying a virus on customer systems
Spotlight On: Programming Techniques Used as an Insider Attack Tool

Spotlight On: Programming Techniques Used as an Insider Attack Tool

Spotlight On: is a quarterly report issued by the CERT Insider Threat Team.

The Insider Threat Team receives significant funding from CyLab.

As one of their benefits, CyLab's corporate partners receive each issue of Spotlight On three months prior to its public release.

So as Programming Techniques Used as an Insider Attack Tool is released to the public, CyLab's partners are now moving on to Malicious Insiders with Ties to the Internet Underground Community, which we will post here on CyBlog in 3Q09.

Spotlight On: Programming Techniques Used as an Insider Attack Tool includes analysis of numerous cases.

Similarities across Cases
While the number of cases analyzed for this article is limited, there are similarities worth noting. The majority of these cases were IT Sabotage cases,1 which follow the escalation patterns documented in CERT’s MERIT model.2 The MERIT model is a system dynamics model of the insider IT sabotage problem that elaborates complex interactions in the domain and unintended consequences of organizational policies, practices, technology, and culture on insider behavior.
In each of the fifteen cases, changes made by the insider may have been detected prior to the malicious code being deployed had the organization had change controls in place to detect unauthorized modifications to critical systems and software. Some of the organizations did use configuration management tools to track and log changes to critical software. However, either the tools did not prohibit software from being released without approval from a trusted second person, or the organization failed to audit the change control logs for unauthorized changes. Programming Techniques Used as an Insider Attack Tool

Spotlight On: Programming Techniques Used as an Insider Attack Tool also articulates a number of practices to help in mitigating this particular aspect of the insider threat

See also the third edition of CERT's Common Sense Guide to Prevention and Detection of Insider Threats, and its empirically-based insider threat risk assessment diagnostic.

To read a CyLab Chronicles Q&A with CERT Insider Threat Team leader Dawn Capelli, click here.

For information on the benefits of partnering with CyLab, contact Gene Hambrick, CyLab Director of Corporate Relations: hambrick at andrew.cmu.edu

-- Richard Power

CyLab Seminar Series Notes: Why do people and corporations not invest more in security?

CyLab Seminar Series Notes: Why do people and corporations not invest more in security? Nicolas Christin on "Understanding User Investments & Response to Security Threats"

[NOTE: CyLab's weekly seminar series provides a powerful platform for the highlighting vital research. The physical audience in the auditorium is composed of Carnegie Mellon University faculty and graduate students, but CyLab's corporate partners also have access via the World Wide Web. Frequently, CyBlog will wet your appetite by offering brief glimpses into these talks. Here are my notes from a talk on Understanding User Investments and Response to Security Threats delivered by Nicolas Christin on 1-19-09. -- Richard Power]

Why do people and corporations not invest more in security?

After all, CyLab researcher Nicolas Christin notes, there are compelling reasons for spending more on security: users claim they have an interest in secure practices, security technology is, by and large, inexpensively available (e.g., PGP, SSL, AES), and financial losses can be very costly.

The thesis of this research is that economics can help understand and change user behavior.

There are several reasons why Christin and his fellow researchers believe in their thesis:

"First of all, this is the 21st Century, everybody is on the network," Christin explains, "all computers are connected one way or another.

"It is also a competitive environment, e.g., competing Internet service providers (ISPs), competing content providers, even within a single organization you may have different divisions competing for funds.

"In addition, we have strong externalities, i.e., the security of one person affects the whole network, or at least a significant number of other users; for example, when the Code Red Worm or the I Love You virus started to propagate on the Internet, they were passed along by end users who failed to properly secure their systems ... so who should pay for security? The people who financially suffer the most from security problems, or the people who are causing these security problems to flourish in the first place? That is an interesting question...

"Another reason we think economics is a very good complement to technology is that [today] criminals themselves are, by and large, very rational, they are in it for the money."

However, unlike most cyber criminals, end users, whether corporations or individuals, are not, in general, "perfectly rational," nor are they random. "We need to find a way of modeling their behavior" Christin remarks, 'so that we can then impact it."

Using modeling methodology, which includes formal analysis, experimental research and field data measurement, Christin and his fellow researchers are in pursuit of "an abstraction that captures as much as possible the salient features of a host of different security situations"

"When you have a fairly reasonable model," according to Christin, "you can use it to test intervention mechanisms before deploying them in practice." For example, you might ask, what would be the impact of passing some particular law? "You test it on your model, and you can make recommendations to a policymaker, and you have something to substantiate your argument, which hopefully makes it more compelling."

"We have tried to look at simple security games, where people are playing against each other, with an exogenous attacker, so all the people we are looking at are basically defending against a common set of threats. We have separated them into a finite number of canonical security games ... They cover a reasonable range of security situations."

"We decouple security strategies into investments to protect yourself (e.g., setting up a firewall) and self-insurance coverage (e.g., archiving data for back up)."

("Most of the research in the economics of security assumes that you have a single security variable," Christin explains, "but we think that this is a little too rough, and that at least we should consider the two different things that people can do.)

"We also consider those network externalities, i.e., a choice by one person on the network affects other participants on the network."



In the course of his seminar, Christin broke down the elements of the general model as depicted in this post:

"You have your 'Expected utility, which is essentially the amount of money that you can expect to have after the attack has or has not taken place. It is simply the 'Initial endowment' (that is, the money you start out with) minus the potential losses you are going to face, and minus the 'Protection investment.' Say our player invests in security, maybe he buys an anti-virus program: that is going to be captured by this protection cost si(normalized to (0-1) hence the scaling variable bi).

"Then there is 'Insurance purchased,' the amount of insurance that you have purchased, whether it is literally insurance with a provider, or simply back ups, i.e., anything that allows you to recover after a security attack has been successful. So the security expenses are broken down in protection and insurance expenses.'Expected loss' is that lose you would expect to face if you didn't institute any security at all. ...

"The expected loss is mitigated by your security expenses, as you can see from the formula. The insurance expenses only depend on that individual player, which means that if your return on investment doesn't depend on what other people are doing. That is good news for most people.

"But the thing that is going to throw us off and that makes this research interesting is that when it comes to protection, we have these externalities here, 'Network protection level (public good)' ... somehow hidden in that function ... where the level of protection that the individual player picks is part of the level of protection it is going to receive, but the level of protection that the other people pick is also going to impact the bottom line, the 'Expected utility.'

"So, the overall utility H depends on all the players in the network. This is where the actions of others impacts my welfare, and this is the critical point we have to model."

To read a relevant paper, which includes a complete analysis, click on J. Grossklags, N. Christin, and J. Chuang. Secure or Insure? A Game-Theoretic Analysis of Information Security Games. WWW'08.

To read a CyLab Chronicles Q&A with Nicolas Christin, click here.

For information on the benefits of partnering with CyLab, contact Gene Hambrick, CyLab Director of Corporate Relations: hambrick at andrew.cmu.edu

CyLab Partners Speak Out on the Benefits of Partnernship

“We couldn’t be happier with our membership than we are.” Christopher Martin, Bosch

“With our external collaborations, we are seeking two things -- the best and the brightest people, and the best and the brightest ideas. That’s what brings us to CyLab.” Dennis Shou, Symantec

“That Carnegie Mellon brand goes along way, we advertise our association with CyLab on our web site, and as a cyber security company, we look at it as a strong halo effect on our brand.” Michael Concordia, BitArmor

“As we look at technology and innovation, there are always unintended consequences, CyLab acts as that branch or arm that allows us to respond proactively to those unintended consequences.” Jay Srini, University of Pennsylvania Medical Center (UPMC)

CyLab Partners Speak Out on the Benefits of Partnernship

CyLab is a world-class academic research program, with a unique multi-disciplinary approach, pursuing a vital goal: "Confidence in a Networked World."

Along the way, "CyLab harnesses the future to secure the present."

And what do we offer leaders in industry, technology and government?

"Partnering with CyLab sharpens the cutting edge."

In our ongoing efforts to communicate the reality of CyLab, how we chose to articulate our own mission is, of course, important; but the insights of our corporate partners are, arguably, at least as important.

So here are four brief videos featuring CyLab partners answering four vital questions. Take a few moments to listen to these voices of experiences.

For more information on the CyLab Partners program, click here.

-- Richard Power

Why Partner with CyLab?



What is the Future of CyLab?



What are the Benefits of Partnership?



What is the CyLab Differential?

CyLab on You Tube: "CyLab is a unique organization that covers a space that nobody else does in the world today."

CyLab on You Tube

"CyLab is a unique organization that covers a space that nobody else does in the world today ... Our researchers have a mission to work on problems that others don't, that the industry hasn't addressed yet. We are two or three steps ahead of industry." Virgil Gligor, CyLab Co-Director

Take a few moments to become acquainted with some of the voices and faces of Carnegie Mellon University CyLab by viewing this recent video --

CyLab Research Update: Basic Instincts in the Virtual World?

"Instinctive computing is an emerging framework for a new kind of operating systems. Instead of making patches on an existing system, we want to make a new platform that integrates security, privacy and visual thinking in one place. Yang Cai, PhD., CyLab Instinctive Computing Lab, 2009

CyLab Research Update: Basic Instincts in the Virtual World?

On-line access to the CyLab weekly seminar series is one of the benefits of the CyLab Partners program. This access enables CyLab's corporate partners to expose their own teams to the latest developments in our ongoing research program.

The research being conducted at CyLab is both breathtaking in its vision and powerful in its practicality.

From time to time, CyBlog will offer you a glimpse behind the curtain.

Here is your first peek --

Are there digital pheromones?

In nature, pheromones are used for identification, alarm, trail and information, but a CyLab team is applying the concept to cyberspace.

"Whenever we do a google search we leave a trail, we actually leave our digital pheromones," says Yang Cai, founder of CyLab's Ambient Intelligence Lab. "This pheromone metaphor will combine a lot of elements together, e.g., digital, analog, physical and on-line community. It is a new way to think about different technologies, e.g., positioning, wireless networks, sensing, search, database retrieval. We can integrate a lot of technologies under this concept."

In a recent CyLab partners program seminar, Yang Cai gave a mind-expanding talk on "Instinctive Computing," and the concept of "digital pheromones" was just one aspect of his presentation.

"Instinctive Computing is a rethinking of overall computing, AI and network technologies and a new paradigm for the integrated security and privacy," according to Cai. "Instinctive Computing is a biologically and cognitively inspired computing that minimizes information overhead and maximizes security, privacy, efficiency and reliability. ... Five years ago, at the birth of Cylab, the founding director Pradeep Khosla pointed out that the ultimate goal of security research here is to catalyze the revolutionary technologies for next generation computing and networking. Instinctive Computing is a brand new field created in Cylab."

"Recently, in the field of Cybernetics and AI, there have been quite a few studies about Subconsciousness, e.g., Perceptual Intelligence (Pentland) looks at the perceptual models of humans and animals, and Affective Computing (Picard/Minsky) proposes an emotional machine. According to them, emotions plays a major role in human decision-making and control a lot of our mental resources during decision-making. There is even one PhD. thesis on daydreaming, i.e., how to create a script that simulates daydreaming. It is very unique research. Here at our lab, we are trying to build an instinctive operating system. It is an ambitious goal, but we are trying to build it from very small pieces."

Cai's research is focused on developing technologies in three areas of "Instinctive Computing": Soft Biometrics, Videometrics and Intelligence.

Soft Biometrics: "Soft biometrics is not meant to replace conventional biometrics, but to compliment and assist the traditional methods. The idea behind Soft Biometrics is that in our daily life we do not look at people's irises or fingerprints. We normally very vaguely look at proportion, color, height, gesture, etc. This kind of fuzzy input could be used to identify a person, or discover a pattern. Soft Biometrics would be good for fast-screening. It is non-invasive, because you can do it from videos. And it is also affordable, because a lot of video is free."

Videometrics: "There is a lot of video, but not enough people to look at it all. Most of the video is just thrown away. Here we try to retrieve those videos by words and eye-gazing, so the network will only send the sensory data that the operator is interested in, and the rest will be in low resolution. So we have a multi-resolution video stream. It saves a lot of bandwidth. We tested this on a mobile phone, and sent only the face in high resolution and the rest of the image on low resolution; and this reduced the size of the image sent from a 220K to only 2K. It is a big saving in bandwidth. We also applied this to surveillance videos, and the reduction is significant. So this is very practical for a digital video network, because the big problem is the scalability problem. You have very sophisticated, high resolution cameras but you do not have the bandwidth to pass this to the command center."

Intelligence: We are working with several companies on this project. We try to analyze the sensor data. NASA, for example, has something like fourteen years of data on the ocean and eighteen years of satellite data, but most of the data just sits in the server. There is no time to look at it. We are doing data mining to look at it, and create visualization tools to help the analysts look at it in a very quick way. We really need to see the patterns. This is called spatial-temporal data mining, and this will be very meaningful."

Cai and his team are producing promising results, including:

"The visual instinct-based object segmentation yields robust and fast results."

"The multi-resolution video stream can reduce the network bandwidth significantly."

"We found that a highly selective security system can reduce the concerns of privacy."

"Finally, a security system may be usedful for healthcare research or affordable diagnoses."

If you are interested in learning more, click here to find out about the Instinctive Computing Workshop that Cai is hosting on June 15-16, 2009 at Carnegie Mellon University CyLab in Pittsburgh, Pennsylvania.