View source for A simple averaging technique to supplement the Bayes equation

{{Main|Aggregation techniques}}

<h2>Background</h2>

As we saw [[Error bars and a problem with Bayesian modeling|previously]], the [[Bayes' theorem|Bayes equation]] can easily be misapplied to situations that are not based on rigorous [[Probability theory|probabilistic]] studies. The example given was along the lines of 100 people who are not sure whether it will be sunny or cloudy tomorrow because either they individually don’t know (P=50%) or their probabilities cancel out to become 50%. But the 101st person is certain that it will be sunny. According to Bayes, this will give you 100% certainty that tomorrow will be sunny because the 50% [[Opinion|opinion]], no matter how it is arrived at, simply has no influence.

Another simpler example is if 10 people give you a 60% chance of rain tomorrow and thereby, via the Bayes eqn, cause you to believe with near certainty that it will rain tomorrow. The problem here is that the 10 people are not conducting independent tests (or simulations) to judge the probability of rain. They are, more likely, simply reflecting the single weather report they all watched on TV.

It is clear that Bayes cannot be used in cases where the source offers no better than a handwaving estimate of probability resulting from a casual [[opinion]]. Since this is going to happen in a large number of cases, we will need a more realistic way to combine these opinions.

One way is to simply average the probabilities being given by each source. For the 10 people who watched the same weather report, the average will then be 60%, reflecting only the single source they obtained their information from.

So now we would have two methods for combining probabilities, a simple averaging technique and Bayes. It would be left to the user to choose which of these to use.

<h2>A [[Weighting factor|weighting factor]] between simple averaging and Bayes</h2>

But these two choices seem like two opposite endpoints on a continuum. On the one hand we have Bayes for rigorous probabilistic tests and on the other we have simple averaging for the most unrigorous opinions. Unlike Bayes, the averaging technique provides no reinforcement (ie the <math display="inline">P_{comb}</math> can’t be higher than the highest <math display="inline">P</math>). But it seems like a large crowd should provide some reinforcement. If 10 people say 60%, isn’t that sometimes better than a single person saying 60%? What if there were two independent weather predictions that give the chance of rain at 60% and the 10 people are divided into these two groups? Then you could apply Bayes for two sources at 60% and get a reinforced result of 69%.

A straightforward way to do this is to simply have a user-chosen weighting factor between simple averaging and Bayes:

<math display="block">P_{comb} = (1-w_{bay})P_{ave} + w_{bay}P_{bay}   
</math>

where:

<math display="inline">P_{comb}</math> is combined probability

<math display="inline">P_{ave}</math> is the simple-averaged probability

<math display="inline">P_{bay}</math> is the [[Bayes' theorem|Bayesian]] combined probability

<math display="inline">w_{bay}</math> is the weighting toward Bayes. If <math display="inline">w_{bay}</math> = 0 then only simple averaging is used. If <math display="inline">w_{bay}</math> = 1 then the algorithm uses Bayes only.

The input probabilities for <math display="inline">P_{ave}</math> and <math display="inline">P_{bay}</math> are the same. That is, they are modified using [[trust]] in exactly the same way (using [https://ceur-ws.org/Vol-1664/w9.pdf Sapienza’s equation], or using the modified forms of this equation described [[Modification_to_the_Sapienza_probability_adjustment_for_trust_to_include_random_lying,_bias,_and_biased_lying|here]]).

<h2>Combining input probabilities in simple averaging</h2> 

However, the input probabilities are not rolled up in exactly the same way to create <math display="inline">P_{comb}</math>. For Bayes, [[Node|nodes]] are combined with their parent to create <math display="inline">P_{comb}</math>. The <math display="inline">P_{comb}</math> of the parents are then combined with their parent to create a new <math display="inline">P_{comb}</math> and so on all the way to the topmost node. For simple averaging, doing this will result in double-counting nodes, so the technique is to just append probabilities as we work up and then take the average once the top-most node is reached by dividing the sum by the number of nodes.

<h2>Example</h2> 

The following example shows how this works:

<kroki lang="graphviz">
digraph G {
    fontname="Helvetica,Arial,sans-serif"
    node [fontname="Helvetica,Arial,sans-serif"]
    edge [fontname="Helvetica,Arial,sans-serif"]
    layout=dot
    0 [label="0, P=50%"]
    1 [label="1, P=50%"]
    2 [label="2, P=50%"]
    3 [label="3, P=60%"]
    4 [label="4, P=70%"]
    5 [label="5, P=80%"]
    6 [label="6, P=90%"]
    0 -> 1 [label="T=0.9",dir="both"];
    0 -> 2 [label="T=0.9",dir="both"];
    1 -> 3 [label="T=0.9",dir="both"];
    1 -> 4 [label="T=0.9",dir="both"];
    2 -> 5 [label="T=0.9",dir="both"];
    2 -> 6 [label="T=0.9",dir="both"];
    }
</kroki>
We start at the bottom and find the modified probabilities based on [[Trust|Trust]]. As noted above, this is no different than what we’ve always done:

<math display="inline">P_{mod3} = P_{nom} + T(P-P_{nom}) = 0.5 + 0.9(0.6 - 0.5) = 0.59</math>

<math display="inline">P_{mod4} = 0.5 + 0.9(0.7 - 0.5) = 0.68</math>

<math display="inline">P_{mod5} = 0.5 + 0.9(0.8 - 0.5) = 0.77</math>

<math display="inline">P_{mod6} = 0.5 + 0.9(0.9 - 0.5) = 0.86</math>

The modified probabilities for Nodes 3 and 4 are then appended to Node 1

<math display="inline">P_{mod1} = [0.50, 0.59, 0.68]</math>

The modified probabilities for Nodes 5 and 6 are appended similarly to Node 2:

<math display="inline">P_{mod2} = [0.50, 0.77, 0.86]</math>

These lists of probabilities are modified again by trust (for the 0-1 and 0-2 connection) and appended to the Node 0 probability:

<math display="inline">P_{mod01} = [0.5 + 0.9(0.5 - 0.5), 0.5 + 0.9(0.59 - 0.5), 0.5 + 0.9(0.68 - 0.5)] = [0.5, 0.581, 0.662]</math>

<math display="inline">P_{mod02} = [0.5 + 0.9(0.5 - 0.5), 0.5 + 0.9(0.77 - 0.5), 0.5 + 0.9(0.86 - 0.5)] = [0.5, 0.743, 0.824]</math>

<math display="inline">P_{mod0} = [0.5, 0.5, 0.581, 0.662, 0.5, 0.743, 0.824]</math>

The average for node 0 can now be found:

<math display="inline">P_{ave} = (0.5+0.5+0.581+0.662+0.5+0.743+0.824) / 7 = 4.31 / 7 = 0.616</math>

The Bayesian combined probability for this case is:

<math display="inline">P_{bay} = 0.964</math>

If we apply a weighting factor of, say, 20% as discussed above, we obtain:

<math display="inline">w_{bay} = 0.2</math>

<math display="inline">P_{comb} = (1-w_{bay})P_{ave} + w_{bay}P_{bay}</math>

<math display="inline">P_{comb} = (1-0.2)(0.616) + 0.2(0.964) = 0.686</math>

This [https://gitlab.syncad.com/peerverity/trust-model-playground/-/snippets/140 snippet] performs this calculation and allows you to change values and tree configuration. 

This algorithm can be [[Privacy enhancing straight average algorithm|modified to enhance the privacy]] of information transmitted up the nodes.