Thoughts on a privacy preserving, simulacrum resistant identity verification mechanism

Main article: Privacy, identity, and fraud in the ratings system

The digital absence of natural personhood

There are many situations in which it is desirable to limit participation (in a community, discussion, event, entitlement, vote, or anything else one can participate in) to natural persons only. Let us define “natural person” as a single living, breathing organism experiencing consciousness, sapience, and sentience.

Artificial intelligence and machine-learning technology is and has been rapidly advancing, and already prior to the recent surge in AI, social media has been overrun with bot accounts which can be directed to interact in limited ways with these platforms. Reddit upvotes, Instagram follows, re-tweets and other interactions can be more or less openly purchased from large botnets which have been built for the purpose of providing these services.

On the surface, the ability to purchase such artificial reinforcements could seem mostly innocuous. However, the ability to multiply one’s voice with one’s money, and to do so with plausible deniability, is especially dangerous. Certainly, throughout history, money has always bought influence. With money, one can buy time on television, signs in public places, and all sorts of other marketing materials. However, the arrival of social media combined with the relative ease of creating puppet accounts has, arguably for the first time, enabled this to happen in near-complete secrecy and without any controls whatsoever.

The looming threat of AI simulacra

Considering the current velocity (and acceleration) of AI technology, this power is rapidly becoming more and more alarming. Until very recently, it has been more or less limited to simple interactions such as upvoting, downvoting, and automatic reposting of content. However, this limitation is already in the past. It is already technically possible to create a “digital simulacrum” with the use of AI large-language models (LLMs). Let us define a “simulacrum” as an entity which is capable of fraudulently presenting itself as a natural person. By prompting or training an AI LLM on certain opinions and/or disinformation, it is possible to create the illusion of an entirely independent participant in any online activity, especially one limited to text-based communication. It is furthermore possible to multiply this illusion indefinitely, constrained only by resources and the limitations of the technology itself.

Additionally, we must take this “limitation” with a grain of salt. We cannot lean too heavily on the current limitations of AI with regard to generation of video or audio contents, because AI is rapidly advancing. The ability to generate credible simulacra will only get more powerful over time, as these AI imposters gain the ability to speak like us, look like us, and one day they may even walk among us undetected.

Identifying the problem to solve

There are those who advocate the end of anonymity and pseudonymity on the Internet. Many people have argued in favour of restricting Internet access and participation in social media based upon a 1-1 relationship with a government-issued identity. This would have the effect of making simulacra impossible, but it would also have other incredibly undesirable side-effects. The ability to separate one’s online identity or identities from their “real-world” identity is key to many positive aspects of modern information technology:

It helps whistleblowers and journalists to speak freely without fear of retaliation.
It allows oppressed minorities to gather and explore their identity safely.
It allows for personal experimentation and artistic expression.
In general, it provides a safe haven from oppression.

Of course, we should not ignore the weaknesses that the ease of operating simulacra entails. We should focus on the real problem, which is the ability to multiply one’s input to a platform or entitlement to resources, by fraudulently representing multiple natural persons.

A naive attestation-based system and its flaws

One potential solution is to operate on a basis of attestations. One person who is trusted by the platform to be a genuine natural person could meet another individual who also wishes to be considered a natural person, verify and attest to this fact, and then the second individual would be considered a natural person for the purposes of participation on that platform.

Unfortunately, nothing stops a person from doing this twice while presenting as two natural persons to two different people. Perhaps if it were really only one person with the ability to bestow the “natural personhood” flag, this could work, but such a system would scale very poorly and have many of the same disadvantages as relying on the government-issued identity mentioned above.

Another weakness of this system is that once the power to recognise natural personhood is no longer exclusive to parties directly related to the platform, the system is extremely weak to a sybil attack perpetrated by any of the indirectly-attested individuals. For example, if A is a trusted individual who attests the natural personhood of B, B is free to instantiate and attest the natural personhood of simulacra B[1], B[2], B[3] and so on.

Making it less naive

One way to combat this weakness, but not defeat it entirely, would be to assign each person a limited number of natural personhood “tags” to assign downstream users, perhaps offering unlimited such tags to the trusted persons, but otherwise limiting the tags given to each new attested entity.

Any initial limit above or including one is entirely ineffective for this purpose, since the above example could be modified such that for limit 1:

Trusted A attests B
B instantiates simulacra B[1] and B[2]
B attests B[1]
B directs B[1] to attest B[2]
B instantiates simulacrum B[3]
B directs B[2] to attest B[3] and so on.

Of course, a limit of zero is hardly useful either since it is effectively a centralised whitelist. An interesting case for exploration is an initial fractional limit between 0 and 1, as this could allow for somewhat useful and low-risk organic growth of the network. If a fractional limit were assigned to every newly-attested identity, no single person could independently create an attested simulacrum:

Trusted A attests B, setting its fractional tag limit to 0.5.
B instantiates simulacrum B[1].
B fractionally attests simulacrum B[1] at 0.5.
B[1] remains partially unattested and faces limits on participation.

While this goes a long way to prevent the independent schemes mentioned above, there are a few problems. Firstly, there is still the possibility for attested simulacra to be formed. Consider the following scenario:

B and C are colluding to defraud the network.
Trusted A separately attests B and C, setting both their fractional tag limits to 0.5.
B instantiates simulacrum B[1].
B fractionally attests B[1] at 0.5.
C fractionally attests B[1] at 0.5.
B[1] is an attested simulacrum.

Furthermore, if the fractional tags do not somehow regenerate, the network will fail to grow beyond a certain point, as eventually, tags will be too fractional and too sparse to result in any additional attestations. A very simple example:

Trusted A attests B, C, D, and E with limits 0.50. (Total tags: 2.0)
B and C expend their tag balance to attest F with limit 0.5 (Total tags: 1.5)
D and E expend their tag balance to attest G with limit 0.5 (Total tags: 1.0)
F and G expend their tag balance to attest H with limit 0.5 (Total tags: 0.5)
It is no longer possible to expand the network without the aid of a trusted individual.

Tag regeneration and avoiding simulacrum chaining

In certain cases it may be more desirable to arrive at a suitable “unsustainable” limit and leave well enough alone, accepting the eventual inability to expand the network without trusted attestations. For other cases this is problematic and will need a solution. One idea is to have the tag balance replenish at a certain rate, for example once per week, up to the initial amount. Then, in the scenario above, after a week had elapsed (assuming no further attestations had been made) the total tags in the system will have reached 3.5.

However, the regeneration of tags in this manner opens the system to attack again:

B and C are colluding to defraud the network.
Trusted A separately attests B and C, setting both their fractional tag limits to 0.5.
B instantiates simulacrum B[1].
B fractionally attests B[1] at 0.5.
C fractionally attests B[1] at 0.5.
B[1] is an attested simulacrum.
One week elapses. B and C again have 0.5 tags each.
C instantiates simulacrum C[1].
B and C collude to attest C[1].
After another week, total tags under the control of the B-C syndicate number 2.0 and counting.

One thought is to limit attestations not only by tags, but also by the combination of attestors, such that every subsequent attestation by an identical group of attestors applied a progressively lower coefficient to the efficacy of the tags spent in this way. We may expect this to work, and it can at least slightly impact the ability to attack the network, but a “simulacrum-chaining” attack is still possible:

B and C are colluding to defraud the network.
Trusted A separately attests B and C, setting both their fractional tag limits to 0.5.
B instantiates simulacrum B[1].
B and C each spend 0.5 tags to attest B[1].
B[1] is an attested simulacrum.
One week elapses. B and C again have 0.5 tags each.
C instantiates simulacrum C[1].
B and C attempt to attest C[1]. However, since B and C have together attested B[1], the tag cost coefficient to attest C[1] is 2.
B directs B[1] to collude with B to attest C[1]. Since the tag cost coefficient is only 1, this is possible.
C[1] is another attested simulacrum.
After another week, C[1] and B[1] can similarly be used to create attested simulacrum B[2].
This “chain” continues indefinitely.

Another thought is to have the upper tag limit decay with each attestation, perhaps in combination with the above suggestion or with some modifications. However, we may reach the conclusion that it is mathematically impossible to create an algorithm by which such a system is both resistant to simulacrum chaining and growth collapse.

Challenge-based liveness checks

Regardless of whether an indefinite simulacrum chain can be created, the existence of simulacra on the network is likely inevitable. One way to make simulacrum chaining less effective is to allow the network to challenge their authenticity and demand that they accumulate additional attestations. A specific implementation is not suggested here, but in short, the following ideas should be entertained:

The accumulated attestations of an individual are necessarily public knowledge.
A certain pattern in attestations should be clearly visible in the case of a simulacrum chain or other fraudulent attestation scheme.
It should be possible to develop heuristics which the software could apply and alert users to potential fraudulent schemes.
Both fraudulent attestations and baseless accusations will be public, visible, and should be pointed out and penalised by well-behaving software.
Additional attestations could be required to come from outside an accusation-defined syndicate, but not necessarily directly from a trusted party.

Forward-looking thoughts

Some additional discussion points:

The “tag balance” is loosely-based on the previously-discussed concept of “fractional personhood,” however they are not exactly the same and may even be combined in some way. I am still thinking about this.
- Notably, the idea of attestations based on time spent together is not covered here. These may be useful as a factor in regeneration of tags, or in the maximum tag balance assigned to an individual, or both.
For the attacks mentioned above, we assume an initial/maximum tag balance of 0.5, which only requires a syndicate with two members. Without doing the math, lower values of this constant should theoretically raise the membership requirement for a syndicate capable of a simulacrum chaining attack, at the expense of further constraining the rate of attestation into the network.
This write-up still does not lend much discussion to the specific threat of a natural person collecting multiple first-order attestations by presenting different identities in different situations. I suspect a great deal of weight will be pulled by the challenge-based liveness checks together with the immense mental load of maintaining separate, coherent, unique identities.
I would really like to spend some time on some simulations for this, as I think it’s really worth exploring these questions further.