Toggle menu
123
784
7.7K
22.9K
Information Rating System Wiki
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Tagging, discovery, and delegation scopes

From Information Rating System Wiki

Status: design proposal for discussion (September 2026). Nothing here is implemented yet except where noted as "today".

Problem

We use tags to categorize discussions and to find ones of interest. It is not clear that flat tags are the best way to answer queries like "find political discussions related to ideas like reputation-based political systems". Such a query fails with tags for a predictable reason: nobody will have applied a "reputation-based" tag consistently, and the idea appears under many names (meritocracy, reputation-weighted voting, epistocracy, ...).

The underlying issue is that tags currently do three different jobs, and each job needs something different:

Job Needs Today
Discovery ("find discussions like X") Tolerance for vague wording and paraphrase; recall matters more than precision Exact tag match, plus substring text filters
Delegation scope Stable, named, few scopes that users agree to Any flat tag, matched by exact tag id
Rating weighting Relevance agreed by the crowd Relevance threshold (at least 2 raters and an average of 80 or more)

How the system works today

  • Tags are flat. A tag is just a name plus a "Tag X is useful" proposition. There is no hierarchy, no synonyms, and no tag types or namespaces.
  • Tag relevance is crowd-rated. Each tag link on a proposition or document has its own relevancy proposition (Tag "X" is relevant to proposition "Y"). A link "meets threshold" when it has at least 2 ratings and an average of at least 0.80.
  • That threshold is not used by discovery. It drives the rating/delegation algorithms and turns tag chips green, but the tag filters in lists match any tag link, rated or not. The "tag is useful" rating is only displayed.
  • Search capabilities exist but are not wired into discussion browsing. There are vector embeddings for propositions, documents and tags, BM25 full-text search, and a hybrid content search (60% semantic + 40% keyword). The discussion list only offers tag filters and plain substring text filters.
  • Delegation is tied to exact tags. A delegation scoped to a tag only affects a proposition through that exact tag, and only when the tag link meets the relevance threshold. Adding tag hierarchy or merging duplicate tags would therefore change who votes on what; it is a governance decision, not just a UI feature.

What the research says

Flat free tagging decays unless it is governed

  • On Delicious, the tag vocabulary levelled off while the number of documents kept growing, so tags carried less and less information about content and the site got harder to navigate (Chi & Mytkowicz).
  • Stack Overflow measured the opposite trend (tag efficiency rising) because of governance: a limit on tags per question, a reputation threshold for creating tags, automatic removal of unused tags, voted synonyms that always point to the more general tag, and tag wikis.
  • Crowd consensus on tags does stabilize, but only after many people tag the same item (roughly 100 bookmarks per resource on Delicious, Golder & Huberman). Our per-link relevance ratings are valuable but will be thin for most items.

Hierarchies help browsing, but hand-built ones decay

  • Wikipedia's category graph has cycles and "topic drift" (following child categories from Computing eventually reaches Theology). Researchers who use it trim it to a small tree.
  • Tag hierarchies learned from tag co-occurrence (using how widely a tag is used as its generality) navigated better than topic models (Helic & Strohmaier).
  • Discourse practice: keep categories few and shallow; try a topic as a tag first and promote it to a category once it has traffic. Decidim 0.30 merged its scopes, areas and categories into one admin-curated hierarchical taxonomy.
  • Faceting (separate independent dimensions such as issue area, place, proposition type) is a long-standing alternative to one all-purpose hierarchy.

Semantic search and topic modelling

  • Dense (embedding) retrievers often do worse than BM25 on unfamiliar domains (BEIR). Combining BM25 and vectors with reciprocal rank fusion is hard to beat and is the usual production setup. Paraphrase-heavy queries like "reputation-based political systems" are exactly where vectors help.
  • BERTopic assigns one topic per document and tends to produce many overlapping topics on social-media text.
  • TnT-LLM (Microsoft, KDD'24): an LLM drafts a taxonomy from samples, a human reviews it briefly, then a cheap classifier labels everything. Human evaluators found it more accurate than clustering baselines.
  • Clio (Anthropic): LLM summaries per item are embedded and clustered bottom-up into a hierarchy with LLM-written titles, shown as a zoomable map.
  • Talk to the City clustered text and had an LLM name the clusters; more than half of the names had to be rewritten by hand.
  • For discovery embeddings are excellent ("find similar" needs no vocabulary agreement). For delegation they are poor: clusters shift whenever the model is re-run or content is added, and nobody can see or agree to the boundary.

How deliberation platforms organize content

  • Kialo and the MIT Deliberatorium put structure inside a debate (pro/con trees, moderated argument maps). The Deliberatorium reduced redundant ideas in 160-person groups compared with a forum, but relies on moderators.
  • Pol.is (used by vTaiwan) groups people by their agree/disagree votes and highlights statements supported across groups. Community Notes uses matrix factorization to find the main axis of disagreement and surfaces notes rated helpful by both sides ("bridging"). Known weakness: manipulation by small coordinated groups.
  • LiquidFeedback: admin-defined units, then subject areas, then issues. Delegation can be set at each level and the most specific one wins.

Delegation by topic

  • The main argument for liquid democracy is expertise in a policy area, so delegation should be scoped by policy area; a known risk is "policy incoherence" when a voter has different representatives on overlapping areas (Blum & Zuber, 2016).
  • Newer work proposes layered defaults: general default, then per-area default, then per-topic override, resolved to the most specific.
  • Google Votes: only 3.6% of votes were delegated. Its "Golden Rule": if I give you my vote, I can see what you do with it.
  • Delegation rules that only look at a voter's local neighbourhood can't guarantee better outcomes than direct voting, because power concentrates (Kahng, Mackenzie & Procaccia).
  • No paper found gives a method for designing the topic taxonomy; in practice it is curated and fairly coarse.

Key principle: machines propose, people ratify

Every approach that generates topics automatically (AI taxonomies, embedding clusters, maps) reshuffles itself on each re-run: clusters move, merge, split and get renamed. That is fine for browsing but not for delegation, where a boundary quietly changing moves someone's vote. So:

  • Automatically found topics are only proposals. The community ratifies an area before it can scope delegation, for example as a rated proposition ("Area X is a coherent delegation scope").
  • Ratified areas are versioned; later re-runs arrive as explicit merge/split/rename proposals, and existing delegations are migrated explicitly, never silently.
  • A delegator is notified when something new enters their delegated scope.

(This principle is our synthesis of the literature; no source proposes it directly.)

Proposed architecture

1. Discovery layer (free to change)

  • Hybrid search on the discussion list, using the existing BM25 + embedding search, plus a "more like this" button on every discussion.
  • Lenses: a saved query written in plain language (e.g. "reputation-based political systems"), optionally combined with tag or area filters. Users can follow, share and fork lenses (compare Are.na channels, Reddit multireddits, Bluesky custom feeds). Readers can mark whether top results are relevant, reusing our relevancy-proposition idea but only on the results people actually see.
  • Optional topic map (Clio / Nomic Atlas style) and argument-graph neighbourhoods for exploration.
  • Optional personal ranking based on what a user has rated highly (watch for filter bubbles).

2. Delegation layer (stable)

  • A small, curated, versioned hierarchy of ratified areas (e.g. Governance > Electoral systems), with most-specific-wins resolution and a general default.
  • The first set of areas can be proposed by an AI-generated taxonomy (TnT-LLM style) or by clusters in the argument graph, then ratified.
  • Existing tag delegations can carry over by mapping each tag to an area.
  • Do not bind delegation scope to anything learned by a model or driven by ratings alone: drift would silently change who holds whose vote.

3. Tags, kept but governed

  • Synonyms that point to a more general canonical tag; a description ("wiki") per tag.
  • Suggested "broader tag" links derived from tag co-occurrence; these links could themselves be rated propositions.
  • An LLM suggests tags when a discussion is created, and a person confirms them.
  • Tag filters get an option to count only tags that meet the relevance threshold.
  • Consider a reputation threshold for creating tags (possibly using the "tag is useful" rating) and pruning of unused tags.

4. The bridge between layers

Lenses and classifiers propose which area a discussion belongs to; crowd relevance ratings confirm it. Ideally that confirmation uses bridging (count relevance as agreed only when raters who normally disagree both endorse it), so one faction can't rate a tag or area onto a proposition.

More radical ideas

These are extrapolations, not proven systems.

Approach For discovery For delegation
AI-generated taxonomy (TnT-LLM, Clio) Good: quickly gives a readable hierarchy Only as a source of proposed areas that people then ratify
Saved lenses (plain-language queries) Best overall for finding things Risky as a live scope. Works as a pinned snapshot plus approval of new matches, or to suggest which area a new discussion belongs to
Embedding map (Nomic Atlas style) Great for exploring and spotting gaps Never as a scope; useful to show a delegator where their delegate's votes land
Personal relevance learned from ratings Good ranking, filter-bubble risk Can't define shared scopes; can suggest delegates who rate like you
Topics from argument links Neighbourhoods of claims that argue with each other See below
AI proxy per voter ("augmented democracy") n/a Removes topic scoping entirely, but raises serious legitimacy and power-concentration concerns

Topics from argument links

Topics could be defined by which claims argue with each other, using the supports/opposes links we already store, rather than by which claims use similar words. The strongest version is rooted delegation: "I delegate this root proposition and everything linked beneath it, to depth N."

  • The boundary follows from explicit links people created, so it is readable and auditable.
  • It only grows when someone adds a link, which the delegator can watch.
  • The delegate judges arguments that actually bear on each other, which suits a deliberation platform.
  • Caveats: no existing system does this across debates, so it is unproven; it is sparse while there are few links; one heavily linked proposition can merge two topics.

A newer idea is browsing by opinion rather than subject (from Pol.is / Community Notes): "discussions where groups disagree" or "claims people across groups support". It uses rating data we already have and is something no tag system can offer.

Suggested order of work

  1. Add hybrid search and "more like this" to the discussion list (cheapest, highest value; the search already exists).
  2. Add a "relevance threshold met" option to tag filters.
  3. Lenses: saved, followable queries (mostly UI plus one table).
  4. Areas for delegation: a design decision to discuss before building, since it changes how delegation works.
  5. Tag governance and AI tag suggestions.

Open questions

  • How many tags exist today and how consistently are they used? (Determines how urgent tag cleanup is.)
  • Should areas replace tag-scoped delegation, or sit alongside it with tags mapped to areas?
  • Is rooted (argument-subtree) delegation worth prototyping as an experimental scope type?
  • Who ratifies areas, and with what threshold?

References

Tagging and folksonomies
Hierarchies and navigation
Search and topic modelling
Deliberation platforms
Liquid democracy and delegation