Bayesian & non Bayesian approaches to trust and Wang & Vassileva's equation

Main article: Technical overview of the ratings system

How Bayesian Approaches Restrict our Thinking, particularly on Trust

If we adopt a non-Bayesian approach, it opens the door to many possible ways that Probability and Trust can be assigned. If you’d prefer to skip to that, just go to the next section. This section is somewhat of an essay on how/why, in a Bayesian approach, our thinking becomes more limited.

Bayes restricts us in both a hard technical sense and more generally in how we think about the information network. The Bayes eqn requires prior and posterior probabilities to make updates with. These probabilities are, presumably, the result of rigorous experimental evidence produced by others (ie experts). This isn’t, of course, strictly necessary but it is the way we usually think about it.

The probabilities can then be modified with trust but, in this context, trust is generally thought of as the competence and honesty of the person performing the test and both are assumed to be relatively high. Trust has less to do with competence in Bayesian approaches (vs averaging approaches) because they only require that the source run a test and report the result correctly. There’s competence in that, to be sure, but it involves little judgement or in-depth knowledge.

Incidentally, in previous posts, the competence was modeled as the random part of Trust (${\textstyle T_{r}}$) and the honesty modeled as Lies and Bias (${\textstyle T_{l}}$ and ${\textstyle T_{b}}$). Implicitly (but not necessarily) left out of the competence is the more important issue of the quality of the underlying experimental evidence. This is done because we assume, correctly or not, that such evidence has the backing of experts, has been peer-reviewed, etc. It is, furthermore, difficult for the layperson to independently verify this type of information. We assume, in effect, that the probabilities are right, in and of themselves, and that only the respondent can mess them up by reporting them incorrectly (or falsely).

Here’s a simple example. If we have a family gathering and ask everyone to take Covid tests before the event, we usually trust that everyone will competently follow the simple instructions and report correctly whether they saw one line or two. We also trust that they will do this honestly. We don’t question the underlying testing that was done to validate the test. However, if during our get together we ask everyone what the best economic system is, or form of government, we will likely get plenty of debate questioning the reasoning, sources, and methods folks used to arrive at their answers. Questions will be raised, in effect, about the tests people are using to determine their answer. Our Trust methodology changes, as it should, as does our method for combining everyone’s opinion (averaging instead of Bayes).

Furthermore in a Bayesian approach, because Trust is limited, we tend to limit our view of who assigns it. Indeed, so far, the requestor of information is 100% responsible for assigning Trust to the respondent it is directly querying. If you have had interactions with someone you feel pretty confident that you can assign a trust to them on questions of basic honesty and competence. This might work if all the respondent has to do is report the result of a test. However, if you then engage in a discussion with that person about weighty topics you’ll often have more doubts about how far they can be trusted, not because they are suddenly dishonest but because they may not know alot about the subject.

You might then find it useful to engage others in determining to what extent the source can be trusted on a particular question.

How Non-Bayesian Trust Approaches Might Work

The weighted average discussion last time brought out the notion that if we are not using Bayes we have a great deal of flexibility in what types of trust information could be provided, and who provides it.

Answers do not necessarily have to be given in terms of probabilities. The source would simply report an answer (True / False) and leave it to the requestor (or others) to evaluate its probability (based on Trust or some component thereof). In a system like this we could simply count up all the True answers and all the False answers and display that. A percentage True/False could be calculated and the higher number would be the “winner”.

An equation to perform this could simply use our current equations (Bayes or averaging) with 100% as the respondent’s probability. In an averaging approach this leads to the same result as simply averaging the answers. We will skip a demonstration of this because it is obvious.

A remaining issue is that of assigning Trust. We would still want trust to modify probabilities, even if they start at 100%. For instance, sources could assign a confidence to their own answers. This could be treated mathematically like Sapienza’s trust-modified probability but it differs qualitatively in the sense that now we recognize that the probability is more nuanced than our Bayesian thinking suggested. In particular it recognizes that confidence in an answer is itself a judgement that might benefit from multiple views.

More generally then, Trust, or let’s say Confidence in someone’s knowledge, can be assigned by multiple parties. The source can provide a confidence and we can evaluate that confidence on the basis of our trust. If we think they have an inflated view of their confidence (a common problem) we can adjust it downward with our own view using an averaging technique. We could weight each confidence to favor either party in performing the average.

Wang and Vassileva's Trust Equation

Indeed we could envision the entire community weighing in on the Trust of the source (ie respondent). A paper by Wang and Vassileva does just this and provides an Equation 3 for dealing with this:

Here we take Equation 3 and add two terms to it to represent our situation: Our source’s trust (confidence) in it’s own answer and the requesting node’s trust in the source. The paper’s apparent presumption was that a) the requesting node did not know the source at all a-priori and thus requires help to form its Trust and b) that the source would not provide a valuable assessment of itself because it is biased. Here we relax these two presumptions by noting that we may have an a-priori opinion of a source but still want help to refine it and that an honest source might indeed provide some insight into its own confidence level. We also refactor the equation slightly by using easier to understand variable names.

To understand the equation we first draw a picture of the nodes represented by it.

If we imagine many k and u nodes, the equation is,

$$T_{rs,mod}=w_{k}{\sum _{k=1}^{N_{k}}T_{rk}T_{ks} \over {\sum _{k=1}^{N_{k}}T_{rk}}}+w_{u}{\sum _{u=1}^{N_{u}}T_{us} \over N_{u}}+w_{r}T_{rs}+w_{s}T_{ss}$$

and

$$w_{k}+w_{u}+w_{r}+w_{s}=1$$

where

${\textstyle w_{k}}$ is the weight placed on the known trustworthy references

${\textstyle w_{u}}$ is the weight placed on the unknown references

${\textstyle w_{r}}$ is the weight placed on r (our view of ourself)

${\textstyle w_{s}}$ is the weight placed on s (the source’s view of itself)

${\textstyle T_{rs,mod}}$ is the modified Trust that ${\textstyle r}$ has in ${\textstyle s}$, after taking into account the other nodes’ opinions.

${\textstyle T_{rs}}$ is the a-priori Trust that ${\textstyle r}$ has in ${\textstyle s}$

${\textstyle T_{rk}}$ is the Trust that ${\textstyle r}$ has in ${\textstyle k}$

${\textstyle T_{ks}}$ is the Trust that the ${\textstyle k}$ known reference has in ${\textstyle s}$

${\textstyle T_{us}}$ is the Trust the ${\textstyle u}$ unknown reference has in ${\textstyle s}$

${\textstyle T_{ss}}$ is the Trust that ${\textstyle s}$ has in itself

${\textstyle r}$ is the node requesting information

${\textstyle s}$ is the source of answer (the respondent)

${\textstyle N_{k}}$ is the number of known references

${\textstyle N_{u}}$ is the number of unknown references

${\textstyle k}$ is a known reference (known to ${\textstyle r}$)

${\textstyle u}$ is an unknown reference

We first note that this slightly modified equation becomes equivalent to Wang and Vassileva’s Eqn. 3 when ${\textstyle w_{r}=0}$ and ${\textstyle w_{s}=0}$. We also note that the first term, in particular, has the property that when ${\textstyle T_{rk}=0}$, the corresponding ${\textstyle T_{ks}}$ simply ceases to contribute.

Indeed this equation is similar in this respect to the trust-weighted averaging scheme we saw previously. Although the paper’s approach is Bayesian (presumably because they have hard performance data on file providers), we can adopt this equation for non-Bayesian purposes.

In this scheme we are effectively creating another trust network to ask the question: how much trust do we have in Node s? This information will then be used to modify the probability that Node s responds with when asked a real question by Node r.

We are, furthermore, using nodes that are unknown to us, ones that we generally don’t use to ask questions of. This is a reasonable approach, especially for distant nodes that we may not have known nodes for.

The authors use a weighting to distinguish known and unknown references. Presumably the weighting is higher for known nodes. But it would seem that we can arbitrarily make up groups of nodes in either category to further subdivide them. We could have known nodes that have particularly good skills in rating others and group them together, or have a group of “semi-known” nodes that rank in the middle. The extension of the equation to either of these cases is trivial. To some extent we have already done this by adding the node’s ${\textstyle r}$ and ${\textstyle s}$ as contributors to the overall Trust level.

Such a scheme implies a great deal of information from various nodes. This information could be introduced via a binning technique to present trust distributions. This would clearly be valuable in and of itself but might also provide insight into groupings that could be weighted differently. A grouping of overly biased supporters, for instance, might easily be identified given a distribution of this kind.

Numerical Example

Let’s suppose we have 5 known nodes and 10 unknown nodes as follows:

${\textstyle N_{k}=5}$

${\textstyle N_{u}=10}$

The requestor node’s Trust for it’s Known nodes is high:

${\textstyle T_{rk}=[0.9,0.9,0.9,0.9,0.9]}$

But the Trust of the Known nodes for the Source node is mixed, and lower:

${\textstyle T_{ks}=[0.5,0.6,0.7,0.8,0.9]}$

The Trust of the unknown nodes for the Source node, however, is higher and more consistent:

${\textstyle T_{us}=[0.8,0.82,0.84,0.86,0.88,0.9,0.92,0.94,0.96,0.98]}$

The requestor node’s own trust in its source node is:

${\textstyle T_{rs}=0.70}$

And the source’s trust in itself is:

${\textstyle T_{ss}=1.0}$

With a weighting distribution which emphasizes the contribution of the known nodes,

${\textstyle w_{k}=0.6}$

${\textstyle w_{u}=0.2}$

${\textstyle w_{r}=0.1}$

${\textstyle w_{s}=0.1}$

we obtain, after plugging into the equation above:

${\textstyle term1={w_{k}{\sum _{k=1}^{N_{k}}T_{rk}T_{ks} \over {\sum _{k=1}^{N_{k}}T_{rk}}}}=0.6{{0.9(0.5)+0.9(0.6)+0.9(0.7)+0.9(0.8)+0.9(0.9)} \over {0.9+0.9+0.9+0.9+0.9}}=0.42}$

${\textstyle term2={w_{u}{\sum _{u=1}^{N_{u}}T_{us} \over N_{u}}}=0.2{0.8+0.82+0.84+0.86+0.88+0.90+0.92+0.94+0.96+0.98 \over 10}=0.178}$

${\textstyle term3=w_{r}T_{rs}=0.1(0.70)=0.07}$

${\textstyle term4=w_{s}T_{ss}=0.1(1.0)=0.1}$

${\textstyle T_{rs,mod}=term1+term2+term3+term4=0.768}$

It is possible, however, for us to suspect that the known nodes are biased against the source and that perhaps the unknown nodes have a more objective opinion. In this case we might change the weighting to reflect that with a consequent increase in our overall Trust:

${\textstyle w_{k}=0.2}$

${\textstyle w_{u}=0.6}$

${\textstyle w_{r}=0.1}$

${\textstyle w_{s}=0.1}$

${\textstyle T_{rs,mod}=0.844}$

This snippet reproduces these calculations.

Addendum: Mediating Tss

The analysis above does not mediate the ${\textstyle T_{ss}}$ opinion except through the weighting factor ${\textstyle w_{s}}$ applied to it. To temper this value, one likely to be larger than realistic (people often have an inflated opinion of themselves), we might try using a network similar to the one we just described above to find ${\textstyle T_{rs,mod}}$. In effect we are asking the question, how much should we trust s’s confidence in itself, ${\textstyle T_{ss}}$? We can use the same network for this without including the ${\textstyle T_{s}s}$, or simply by setting ${\textstyle w_{s}=0}$. The same equation as above then applies.

Once we have computed the answer we can multiply it by ${\textstyle T_{ss}}$ to produce an improved value of ${\textstyle T_{ss}}$, that is ${\textstyle T_{ss,mod}}$. The calculation above then proceeds with ${\textstyle T_{ss,mod}}$ to produce the desired ${\textstyle T_{rs_{m}od}}$, a trust value which will be used to find the probability of the real question at hand.

To break this down, the calculation would proceed as follows:

1. Ask the network “how much trust should we have in ${\textstyle {T_{s}s}}$, s’s confidence in itself? Set ${\textstyle w_{s}=0}$ and calculate a ${\textstyle T_{rs,mod,ss}}$ as described by the equation above.
2. Calculate ${\textstyle T_{ss,mod}=T_{rs,mod,ss}(T_{ss})}$
3. Ask the network the “real question” and calculate ${\textstyle T_{rs,mod}}$ as described by the equation above using ${\textstyle T_{ss,mod}}$ instead of ${\textstyle T_{ss}}$.
4. Use ${\textstyle T_{rs,mod}}$ as you normally would, to modify the probability that the source is giving you (using Bayes or an averaging technique).