More actions
Main article: Technical overview of the ratings system
Background
In this previous post we discussed a straight average algorithm with continuous input distributions, complex trust, and intermediate results. Let’s review briefly these enhancements:
- Continuous input distributions means that the user inputs a graph or function instead of discrete individual points to represent the probability density function (Pdf). The algorithm also accepts discrete points if the user so chooses.
- Complex trust means that in addition to entering probabilities for the truth and random answers, the user enters values for random lying, lying with bias toward one of the particular outcomes, and bias toward one of the outcomes. In addition, the trust is subdivided into judgement trust (
Tj) and communication trust (Tc). - Intermediate results means in addition to the output result of each level, an intermediate result is calculated for the purpose of passing it along to the next upper level in the tree of nodes. For the straight average algorithm this means the output is simply an average of the nodes and the intermediate result is the sum of the nodes (the numerator in the average) and sample size (the denominator in the average).
Needed User Input Features
These enhancements, particularly the first two, will require additional user input in the sandbox GUI if we decide to proceed with them:
- The ability to enter a user-defined Python function. An example of such a function and its invocation looks like this:
def weibull(k, lamb, x):
w = (k/lamb)*(x/lamb)**(k-1.0)*math.exp(-(x/lamb)**k)
return w
def Pd1(x):
k = 7.0
lamb = 0.7
return weibull(k, lamb, x)
opinion1 = OpinionData(Pd1, 1)
A misc_input is also required to establish, later in the calculation code, the range of x values that will be input to the function:
misc_input = {'xmin': 0.0, 'xmax': 1.0, 'n': 10}
- Alternatively, the ability to enter a set of discrete x,y points, eg:
Pd1 = [[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], [0.0, 8.4998494312e-05, 0.005439064803658, 0.061799644413066, 0.341296334310195, 1.207904653411647, 2.822898903602754, 3.678794411714423, 1.745906232336168, 0.135698001814369, 0.00045281720613]]
The algorithm will convert all function input to points prior to doing any calculations. If it sees that points were input to start with it will skip the conversion.
- The ability to enter a complex
trust_factor, eg:
trust_factor = [ [0.8, 0.1, 0.0, 0.02, 0.03, 0.02, 0.03], #Tj, judgement trust
[0.9, 0.1, 0.0, 0.0, 0.0, 0.0, 0.0] ] #Tc, communication trust
The trust_factor is broken up into judgement trust and communication trust. Each of these in turn is comprised of the following, assuming there are two choices (rainy or sunny):
[Ttruth, Trandom, Tlierandom, Tliebias_rainy, Tliebias_sunny, Tbias_rainy, Tbias_sunny]
The trust_factor is then input to the ComponentData as follows, eg:
comp3 = ComponentData(opinion3, trust_factor, intermediate_results)
Once these inputs are established the calculation can proceed.
Q/A Session for Complex Trust
The establishment of the complex trust can be frustrating for the user since there can be many numbers and all of them must add to 1. A simple Q/A session is proposed to help users enter the values. The user begins by entering the available choices, by name, and the Q/A routine asks for the percentage in each trust category. For example if the choices are Blue, Green, and Red, the questions will be 1) Truth, 2) Random, 3) Random lying, 4) Lying with bias toward Blue, 5) Lying with bias toward Green, 6) Lying with bias toward Red, 7) Bias toward Blue, 8) Bias toward Green, 9) Bias toward Red.
The Q/A session enforces the fact that all the values given add up to 1. At each question, the user is reminded of how much probability they have remaining. If the user enters a value such that the total goes above 1, it is adjusted accordingly, the rest of the values are made 0, and the session ends. Similarly if the user enters values that add to less than 1, the last choice’s value is adjusted accordingly to make the total 1.
The attached snippet contains the code for the Q/A session. An example session is shown here:
Enter choices separated by a comma: Blue,Green,Red
Categories you will be asked about:
1 Truth
2 Random
3 Random Lying
4 Lying with bias toward Blue
5 Lying with bias toward Green
6 Lying with bias toward Red
7 Bias toward Blue
8 Bias toward Green
9 Bias toward Red
Answers must add up to 1 or system will change the last answer so it does.
The remainder tells you the max amount you have left. If you answer greater than this, the system will adjust this answer so the sum is 1 and make the rest 0
Enter the % of time (as a decimal, eg 0.35) that respondent answers as follows:
1 of 9 ==> Truth?: 0.7
Truth = 0.7 Remaining amt = 0.3
2 of 9 ==> Random?: 0.1
Random = 0.1 Remaining amt = 0.2
3 of 9 ==> Random Lying?: 0.02
Random Lying = 0.02 Remaining amt = 0.18
4 of 9 ==> Lying with bias toward Blue?: 0.03
Lying with bias toward Blue = 0.03 Remaining amt = 0.15
5 of 9 ==> Lying with bias toward Green?: 0.01
Lying with bias toward Green = 0.01 Remaining amt = 0.14
6 of 9 ==> Lying with bias toward Red?: 0.05
Lying with bias toward Red = 0.05 Remaining amt = 0.09
7 of 9 ==> Bias toward Blue?: 0.01
Bias toward Blue = 0.01 Remaining amt = 0.08
8 of 9 ==> Bias toward Green?: 0.01
Bias toward Green = 0.01 Remaining amt = 0.07
9 of 9 ==> Bias toward Red?: 0.02
Bias toward Red = 0.02 Remaining amt = 0.05
Modified last t so sum of t equals 1
Final t vector: [0.7, 0.1, 0.02, 0.03, 0.01, 0.05, 0.01, 0.01, 0.07]
Final tc, t pairs: <zip object at 0x0000022CE4A0D580>
('Truth', 0.7)
('Random', 0.1)
('Random Lying', 0.02)
('Lying with bias toward Blue', 0.03)
('Lying with bias toward Green', 0.01)
('Lying with bias toward Red', 0.05)
('Bias toward Blue', 0.01)
('Bias toward Green', 0.01)
('Bias toward Red', 0.07)