Trust in Computer Systems and the Cloud. Mike Bursell
Читать онлайн книгу.
economists call externalities. An externality is when there is a cost or benefit to a party who did not choose to incur that cost or benefit.55 Certainly, in the case of the US government's concerns about Huawei telecommunications equipment, any back-door type of behaviour would count as a cost, even if that cost were not directly economic. Let us consider computer systems more generally. Sometimes actions might be performed by an entity (the trustee) without any intention of harm—that is, cost—to the party trusting it (the trustor); but if the trustor does not know about these actions, they have no way to evaluate any possible impact. In this case, the trustor needs to make explicit requirements either to exclude specific actions or even to require that no other actions will be performed beyond the expected ones. This second course of action may seem like the obvious one to take but is actually very difficult.
Many applications, when running, will perform actions that are not core to the functioning of the program itself, which we might call side effects. At the API level, there is a more formal use of this phrase, where actions are performed on data or variables that are not “local” to the function or operator being called. The general case where non-core actions are performed provides us with enough real concern. Two typical examples, which are recurring problems for IT security, will serve to illustrate the problem.
It is a truism that computer programs do not always function as they are designed. For that reason, log files are often collected to allow those who are tasked with managing the programs to understand any problems and maybe to feed back to those who designed and wrote the programs any bugs that are identified. Such logging is generally associated with the actions of the application; but, equally, logging may be performed on the data that is being entered, manipulated, and generated or on user logins and interactions, to allow someone auditing the application and its usage to track how it is being used.56 The danger with which we are concerned is that information is being recorded in logs that should not be. This situation may mean that those who have legitimate access to a particular set of logs also get access to information that is not appropriate. A well-known example of this would be application logs designed to help a developer debug a payment application that logs credit card details, exposing them to the developer. The user of the site has a trust relationship to the application where they should expect that such information is not exposed but has no knowledge of the fact that this logging is taking place nor the ability to control or stop it.
Similar problems can occur with backup files. These differ from log files in that they are not intended for consumption by anything other than the application that may need to recover in the event of a problem, but the files need to contain enough data and information for the application to recover all of the state that it needs to continue operation. There is definitely a possible cost here to the user of an application if these files are accessed by unauthorised parties, but at least in this case, there is a possible benefit, too: the application can continue to be used. The question is whether this benefit outweighs the possible cost and, more specifically, whether the trustor even has the ability to make a choice as to whether backup files are stored or has enough information to make an informed choice as to whether they should be. While backup files on a local system are typically accessible to a user—though not always, nor always advertised—the likelihood of this being the case for remote or multi-user systems is significantly reduced. It would be good—that is, in my interests—if I, as trustor, were given the option to back up my own data in this case and insist that any backups generated by the trustee be anonymised or have any critical data removed. But even if I can insist on this, the chances that I can realistically enforce it are very low.
This is, in fact, another example of security economics: the backups are put in place not for my benefit but for the benefit of the entity operating the application or service I am using. Even if I have visibility into the actions they are performing, I have little or no chance or opportunity to influence them in my favour. Sometimes, despite the inability of individuals to have an impact on the practices of those whose services they are using, governments or other regulatory bodies put in place measures that force service providers to adopt practices that do benefit individuals. Good examples of this in the area we have been describing are the European Union's General Data Protection Regulation (GDPR) and the State of California's California Consumer Privacy Act (CCPA), both of which force service providers to protect consumers' data and put in place measures to prevent it from being misused. A slightly weaker type of protection, but one that can help, is the establishment of industry standards aimed at promoting good practice. Historically, however, standards have ended up benefiting industry players—service providers—rather than consumers or customers, who rarely have much—if any—representation on standards bodies.
In our definition of trust, we started with the following statement:
"Trust is the assurance that one entity holds that another will perform particular actions according to a specific expectation.
It turns out that establishing that assurance can be more difficult than might be expected and that the performance of actions may also need to specify the non-performance of other actions to ensure that we can fully understand what behaviours we, the trustor, are trusting the other entity, the trustee, to perform. In the next chapter, we will examine trust in even more detail, the impact of different forms of trust, how trust is expressed, and some of the alternatives that may be appropriate in certain contexts.
Notes
1 1 Harper, 2014, p. 10.
2 2 Mayers et al., 1995, cited in Harper p. 230.
3 3 Baier, 1986, referenced in Cheshire, p. 55.
4 4 Hardin, 2001.
5 5 Cheshire, 2001, p. 52.
6 6 Dasgupta, 1988.
7 7 Gambetta, 1988.
8 8 There is a wider question about whether money is real and whether bank accounts truly transfer anything when balances are updated, but such discussions are beyond the scope of this book.
9 9 Open source software “is software with source code that anyone can inspect, modify, and enhance”— Opensource.com (2021).
10 10 The definition of what exactly counts as “rational” behaviour may be in itself an issue of debate.
11 11 Axelrod, 1990.
12 12 Heap and Varoufakis, 1995.
13 13 Rathburn, 2011, p. 24.
14 14 Rathburn, 2011, p. 28.
15 15 Schneier, 2012.
16 16 Hobbes 1651; 1996.
17 17 Rousseau, 1762.
18 18