View source for Modification to the Sapienza probability adjustment for trust to include random lying, bias, and biased lying

{{Main|Trust}}

In the [[Internal:FromGitlab/Modification_to_the_Sapienza_probability_adjustment_for_trust_to_include_lying_and_bias|last iteration on this subject]] we modified the probability adjustment for [[trust]] to include lying and bias. We noted that the lying was random in nature, that is, the lie would be distributed evenly among all options that were not the truth. Here we modify this assumption by allowing, in addition, lying that is biased toward a particular outcome. In other words, if the source lies then it tries to lie by favoring a particular outcome, as long as that outcome is not the truth. If the outcome is the truth it falls back to random lying. Keep in mind that this is additional to the already established random lying which we will keep in the model for generality.

<h2>The Equation</h2>

The equation presented here is a little easier to follow because it accounts explicitly for all possibilities: the bias can be toward any of the options and so can the lies.

Let’s begin by defining some terms, many of which are familiar from the last iteration. Again we will assume a choice between three outcomes: Red, Blue and Green.

<math display="inline">C</math> = Number of choices (eg 3 in this case)

<math display="inline">N</math> = number of samples (this ultimately cancels out so is only useful in the derivation)

<math display="inline">P_R</math> = Probability of Red

<math display="inline">P_B</math> = Probability of Blue

<math display="inline">P_G</math> = Probability of Green

<math display="inline">P_R + P_B + P_G = 1</math> (just to be clear)

<math display="inline">T</math> = Trust, ie percent of time the reported outcome is the Truth

<math display="inline">T_r</math> = Random portion of <math display="inline">1-T</math> (same as before)

<math display="inline">T_l</math> = Lying at random portion of <math display="inline">1-T</math> (same as before)

<math display="inline">T_{lr}</math> = Lying with a bias to red (the source will lie this % of time by trying to say Red as long as Red is a lie. If Red is the truth then this portion will be random lying)

<math display="inline">T_{lb}</math> = Lying with a bias to blue

<math display="inline">T_{lg}</math> = Lying with a bias to green

<math display="inline">T_{br}</math> = Bias to red (same as before – the source will answer Red this % of time no matter what)

<math display="inline">T_{bb}</math> = Bias to blue (same as before)

<math display="inline">T_{bg}</math> = Bias to green (same as before)

<math display="inline">T + T_r + T_l + T_{lr} + T_{lb} + T_{lg} + T_{br} + T_{bb} + T_{bg} = 1</math> (just to be clear)

The equation is as follows for any particular option, eg Green:

<math display="block">P_{mod,g} = P_GT + {T_r\over C} + {T_l\over C-1}(1-P_G) + T_{lg}(1-P_G)+ T_{bg} + {P_RT_{lr}+P_BT_{lb}\over C-1}
</math>

In general, for any particular choice <math display="inline">g</math> among all the other choices <math display="inline">i</math>, we can write:

<math display="block">P_{mod,g} = P_gT + {T_r\over C} + {T_l\over C-1}(1-P_g) + T_{lg}(1-P_g)+ T_{bg} + {\Sigma P_iT_{li}\over C-1}
</math>

To be clear, the subscript <math display="inline">i</math> represents any choice except <math display="inline">g</math>, the one we’re doing the <math display="inline">P_{mod}</math> for.

<h2>Derivation</h2>

This derivation follows [[Internal:FromGitlab/Modification_to_the_Sapienza_probability_adjustment_for_trust_to_include_lying_and_bias|the last one we did]] in that we begin with a sample size of <math display="inline">N</math> and a number of choices <math display="inline">C</math> representing colors as above: Red, Blue, and Green. Instead of using a numerical example we will perform the derivation from the outset using the symbolic variable names. The following table can be created to represent the breakdown of actual and reported choices:

[[File:6e99052194ae2c35682ed2e411eee90c_image.png|image]]

We choose all the cases where Green is reported, for instance. The <math display="inline">P_{mod,g}</math> is then the addition of all the reported Green cases divided by the total number of samples:

<math display="block">P_{mod,g} = {NP_GT+{NP_GT_r\over C}+NP_GT_{bg}+{NP_BT_r\over C}+{NP_B(T_l+T_{lb})\over (C-1)}+NP_BT_{lg}+NP_BT_{bg}+{NP_RT_r\over C}+{NP_R(T_l+T_{lr})\over (C-1)}+NP_RT_{lg}+NP_RT_{bg}\over N}
</math>

The N cancels and we can collect terms to obtain:

<math display="block">P_{mod,g} = P_GT+{T_r\over C}(P_G+P_B+P_R) + {P_B(T_l+T_{lb})+P_R(T_l+T_{lr})\over C-1} + T_{lg}(P_B+P_R)+T_{bg}(P_G+P_B+P_R)
</math>

<math display="inline">P_G+P_B+P_R=1</math> and <math display="inline">P_B+P_R=1-P_G</math> so,

<math display="block">P_{mod,g} = P_GT+{T_r\over C} + {P_B(T_l+T_{lb})+P_R(T_l+T_{lr})\over C-1} + T_{lg}(1-P_G)+T_{bg}
</math>

Rearranging terms leads to:

<math display="block">P_{mod,g} = P_GT + {T_r\over C} + T_{lg}(1-P_G)+T_{bg} + {T_l(P_B+P_R)\over C-1} + {P_BT_{lb}\over C-1} + {P_RT_{lr}\over C-1}
</math>

Again noting that <math display="inline">P_B+P_R=1-P_G</math> and some further rearrangement leads to:

<math display="block">P_{mod,g} = P_GT + {T_r\over C} + {{T_l\over C-1}(1-P_G)} + T_{lg}(1-P_G) + T_{bg} + {P_BT_{lb} + P_RT_{lr}\over C-1}
</math>

This is the same as the equation presented above. We note that the first four terms are the same terms from the equation presented [[Internal:FromGitlab/Modification_to_the_Sapienza_probability_adjustment_for_trust_to_include_lying_and_bias|earlier]], the one with just random lying and bias. The next term represents the lie toward the Green outcome. The last term represents the lie toward the Blue and Red outcome that turn into random lies that favor Green as one outcome because they aren’t lies when applied to the Blue and Red outcomes.

<h2>Simple Numerical Example</h2>

To apply this equation we use a familiar example in keeping with the Red, Blue, and Green choices we’ve been using all along.

<math display="inline">C</math> = 3 = Number of choices

<math display="inline">N</math> = 120 = number of samples (this ultimately cancels out so is only useful in the derivation)

<math display="inline">P_R</math> = 60% = Probability of Red

<math display="inline">P_B</math> = 30% = Probability of Blue

<math display="inline">P_G</math> = 10% = Probability of Green

<math display="inline">T</math> = 70% = Trust, ie percent of time the reported outcome is the Truth

<math display="inline">T_r</math> = 8% = Random portion of <math display="inline">1-T</math>

<math display="inline">T_l</math> = 5% = Lying at random portion of <math display="inline">1-T</math>

<math display="inline">T_{lr}</math> = 2% = Lying with a bias to red (the source will lie this % of time by trying to say Red as long as Red is a lie. If Red is the truth then this portion will be random lying)

<math display="inline">T_{lb}</math> = 3% = Lying with a bias to blue

<math display="inline">T_{lg}</math> = 5% = Lying with a bias to green

<math display="inline">T_{br}</math> = 1% = Bias to red (the source will answer Red this % of time no matter what)

<math display="inline">T_{bb}</math> = 3% = Bias to blue

<math display="inline">T_{bg}</math> = 3% = Bias to green

<math display="inline">P_{mod,g}</math> = <tt>0.7(0.1) + (0.08/3) + (0.05/2)(1-0.1) + 0.05(1-0.1) + 0.03 + [0.3(0.03)+0.6(0.02)]/2 = 0.20467</tt>

Our initial 10% probability for the Green outcome now becomes 20.467% because of Trust, randomness, bias toward the Green outcome, and lies toward the Green outcome.

The following snippet reproduces this, or any similar, calculation:

https://gitlab.syncad.com/peerverity/trust-model-playground/-/snippets/139

<h2>The problem of user input</h2>

The equation above increases the amount of user input considerably. The untrustworthy part of Trust could previously be calculated as <math display="inline">1-T</math>, thus requiring no user input. Now the 3 choice problem above requires 8 new pieces of information from the user beyond <math display="inline">T</math> and the probability array <math display="inline">P</math>.

In practice there should be ways to reduce this. The user could be asked first if there are any biases and what they are. It is likely that the bias will only be toward a few of the choices, so the remaining ones will be zero. The next question could be whether the lying always follows the bias. If it does, no separate lying input is needed. The biased lying is then simply set equal to the bias. The next question could be how much random lying takes place, with the notion that many users will set this to zero since the biased lying is more likely. Once this is done the remaining quantity, <math display="inline">T_r</math>, can be calculated as follows:

<math display="block">T_r = (1-T) - (T_l + T_{lg} + T_{bg})
</math>