Signaling Credibility at Scale

A Computational Framework for the 2024 U.S. Election

Eva-Maria Vogel, Christian Pipal, Morgan Wack, Frank Esser | University of Zurich

COMPTEXT · 2026

Today

Why PSMIs matter Political influencers as credibility-without-institutions
What we looked for Five dimensions of credibility signaling across party lines
How we did it LLM classification of 51k posts + GEE and mixed-effects models
Results Democrats signal benevolence and ordinariness; Republicans consistency
Your input The speaker–author mismatch problem

The question

News influencers have become central political information sources, especially for younger audiences.

  • 21% of U.S. adults get news from social media influencers
  • 37% among 18–29-year-olds (Gottfried et al., 2024)

Unlike journalists or politicians, they have no institutional anchor for credibility.

How do partisan news influencers signal credibility — and does it drive audience engagement? Do Democrats and Republicans use different strategies?

Five dimensions of political influencer credibility

Dimension Source What it signals
Expertise SCT Knowledge and interpretive authority
Benevolence SCT (goodwill) Care for the audience; concern for well-being
Consistency SCT + P-PA Principled advocacy despite opposition or cost
Ordinariness P-PA Closeness to ordinary people; rejection of elitism
Immediacy P-PA Unfiltered, spontaneous thoughts and emotions

Expertise and Benevolence = vertical credibility (SCT). Consistency, Ordinariness, Immediacy = Perceived Political Authenticity (P-PA).

Expertise

Does the post invoke expert knowledge?

@libsoftiktokofficial: “As a physician and a mom, I know that minors can’t receive Tylenol without parental consent. But with Amendment 4, they could receive an abortion without parental consent.”

Code: YES. The influencer invokes her own professional credential (“As a physician”) to authorize the claim. First-person credential directly relevant to the topic.

Hyp. Prediction Logic (MFT)
H1a Democrats signal more expertise Higher institutional trust in science among Dem audiences

Benevolence

Is the influencer personally expressing care for others?

@marwilliamson: “I care about animals; I care about people even more. Government should not have told that man what to do with his squirrel, and it sure as hell should not be telling women what to do with our bodies.”

Code: YES. “I care about…” is an explicit first-person care signal directed at a broader constituency.

Hyp. Prediction Logic (MFT)
H1b Democrats signal more benevolence Care/fairness foundations align with Dem moral frame

Consistency

Is the influencer claiming persistence despite cost or opposition?

@TomFitton: “They Want Me Censored for Telling the Truth on Election Integrity!”

Code: YES. The influencer names censorship as a personal cost (“They Want Me Censored”) and frames continued speech as defiance. Concrete threat + first-person persistence.

Hyp. Prediction Logic (MFT)
H1c Republicans signal more consistency Loyalty/in-group commitment; binding foundation

Ordinariness

Does the influencer present as a regular, non-elite person?

@misha: “Come take a stroll with me tomorrow, Oct. 12 at 9 AM sharp to knock on doors for democracy: 1524 Garrison Rd., Charlotte, NC”

Code: YES. The influencer describes a ground-level participatory activity alongside the audience — door-knocking, giving a street address.

RQ Prediction Logic
RQ1a Ordinariness: direction? Populist right vs. creator-economy left — competing logics

Immediacy

Would a professional communications team have approved this as-is?

@terrencekwilliams: “OMG! Im on edge! … I cannot believe today is election day. I am so exhausted, tired. I have not been to sleep. I’ve been up all night. I have been excited. I have been nervous. I have been hopeful. I’ve been doubtful… My emotions have been all over the place.”

Code: YES. Unfiltered emotional exposure in real time. A professional communications team would not have approved this.

RQ Prediction Logic
RQ1b Immediacy: direction? Authenticity vs. authority norms on the right

Methods

Data pipeline

Raw collection
91,133 posts
140 accounts · 3 platforms
Sep 1 – Nov 4, 2024
→
After filtering
51,009 posts
Remove retweets & replies
→
Primary analysis
50,873 posts
135 partisan accounts
57 Democrat · 78 Republican
Top 50 most-followed U.S.-based accounts per platform mentioning Trump or Harris.
Manually coded: news influencers · politicians · celebrities · journalists & media personalities.

Gemini 3.1 Flash Lite classification pipeline

Model: Gemini 3.1 Flash Lite  ·  Strategy: Few-shot  ·  Variables: stacked by group

Group Variables (all binary) N posts
Political screen political, formal_politics, lifestyle_politics 51,009
Partisan alignment pro/anti-Trump, pro/anti-Harris 51,009
Credibility signals expertise, benevolence, consistency, ordinariness, immediacy 51,009

Temperature = 0 throughout.

Inside a single classification call

Codebook "expertise": { "key_question": "Would a viewer think this platform provides expert knowledge?", "definition": "Claim own credentials OR deploy a domain expert as resource..." }
Few-shot examples {"post_text": "I've been studying immigration policy at UW for 17 years. Here's what the peer-reviewed evidence shows.", "labels": {"expertise":1, ...}}
Task instruction Classify each of the following posts using the codebook above. (Each post shows the author's username first.) {POSTS}
Structured output [{"id": "post_001", "expertise": 0|1, "benevolence": 0|1, "consistency": 0|1, "ordinariness": 0|1, "immediacy": 0|1 }, ...]

Classifier validation

Variable n F1 Note
Political 289 .985
Formal politics 277 .960
Expertise 289 .862
Benevolence 280 .822
Consistency 282 .895
Ordinariness 278 .746 Visual-dependent cues
Immediacy 259 .687 Visual-dependent cues
you cannot see
the tears
through the text

Ordinariness and Immediacy rely heavily on visual and audio signals: home setting, casual clothes, voice breaking. These are systematically underdetected by text-only classifiers.

Measuring partisan leaning: behavioral scoring

Rather than assigning party labels manually, we derive partisanship from what accounts actually post.

\[\text{Partisan score}_j = \frac{\text{pro-Trump + anti-Harris posts}_j}{\text{all partisan-coded posts}_j}\]

  • Score < 0.50 → Democrat-leaning  (n = 57)
  • Score > 0.50 → Republican-leaning  (n = 78)

Validating the behavioral scores

Calling commercial LLMs at scale

  • Cost
    Iterating on codebooks or models adds up fast
  • Replicability
    Model updates and stochasticity mean results may not reproduce
  • Opacity & dependency
    Training data undisclosed; pipeline runs at a for-profit company's discretion

Can a fine-tuned ML model replace the LLM?

Political & Partisan: n = 500 · 2,000 · 5,476 Credibility: n = 500 · 2,000 · 10,000 · 45,991

Results

Partisan differences in credibility signaling

Benevolence and Ordinariness are significantly higher among Democrats (OR = 2.8 and 2.3)

Consistency is the only signal Republicans use more (OR = 0.47, roughly 2× higher for Republicans)

Partisan differences in credibility signaling

Expertise and Immediacy show no significant partisan difference

Platform–party collinearity (Cramér's V = 0.58): TikTok skews Democrat, X skews Republican. Platform dummies included as covariates to separate partisan from platform effects.

Credibility signals and audience engagement

Thank you

Eva-Maria Vogel, Christian Pipal, Morgan Wack, Frank Esser

University of Zurich

Contact: eva-maria.vogel@uzh.ch

Open question

Who is actually speaking in the post?

Credibility signals are self-referential by design — “I am an expert”, “I care about you”, “I’ve always stood by this.” The signal only makes sense if the influencer is the one producing it.

In practice, social media posts frequently feature voices other than the account holder: interview guests, news clips, quoted text, reaction content.

What we do to reduce the problem:

  • Exclude retweets and replies — the most error-prone post types
  • Pass the author’s username to the model so it can infer whether the speaker is someone else

What remains unresolved:

The model still makes errors when the influencer is reacting to or quoting another person without explicit attribution. This is the largest remaining source of miscoding — and it is concentrated precisely in credibility variables, where first-person framing is definitional.

Discussion: How should computational studies handle the speaker–author mismatch in social media data? Is post-level exclusion sufficient, or do we need speaker-attribution at the sentence level?

Appendix

Statistical models: partisan differences (RQ1)

Generalized Estimating Equations (logit link)

  • Posts nested within accounts · exchangeable working correlation
  • Predictor: Democrat vs. Republican (behavioral score)
  • Covariate: platform (Instagram reference) · sandwich standard errors
  • Five separate models, one per credibility variable

Statistical models: engagement effects (RQ2)

Linear mixed-effects model

  • Outcome: log(likes + 1)
  • Five credibility signals + five party × signal interactions
  • Platform dummies absorb the party–platform correlation (Cramér’s V = 0.58)
  • Random intercepts for accounts
  • Party enters only through interactions (not as main effect)

Robustness: news influencers only (n = 77)

Exploratory: credibility signaling by account type