Archived link

5,000 AI-Controlled Fake X Accounts Linked to China Disinformation Campaign

Researchers have uncovered a network of at least 5,000 fake X (formerly Twitter) accounts that appear to be controlled by AI in a disinformation campaign linked to China – and the activity appears to be heating up as the U.S. election approaches.

The X disinformation network, dubbed “Green Cicada” by researchers, “primarily engages with divisive U.S. political issues and may plausibly be staged to interfere in the upcoming presidential election.”

The network has also amplified divisive political issues in other democracies, including Australia, western Europe, India, Japan and other democratic countries.

The finding is the latest example of attempted interference in the U.S. presidential election, which just this month has seen reports of increasing activity by Iran.

[…]

The researchers, from CyberCX. […] said the network is unlikely to be very effective in its current state, but they added that it “is plausible that the network operators are preparing to increase activities in the lead up to the U.S. presidential election.”

Most accounts on the network are currently dormant, but activity increased sharply in July. The network has been rectifying operational errors over time – including reducing malformed outputs – which could make its activities more effective and harder to detect over time.

The network uses a Chinese-language LLM system and links to an AI researcher affiliated with Tsinghua University and Zhipu AI, a prominent Chinese AI company. So far the actors haven’t had specific political leanings, but instead have focused on amplification of divisive content, “consistent with China’s information operation playbook,” the researchers said.

[…]

The researchers said [that] “our findings also indicate** key gaps in X’s willingness and ability to detect inauthentic content. **While we have observed X taking sporadic action against Green Cicada Network accounts during our period of monitoring, we have observed a failure to take systemic action against overtly linked accounts.”

“We note that X has reversed initiatives put in place by Twitter to combat inauthentic activity, including efforts to detect, label and/or ban inauthentic accounts.”

The researchers said the network is a sign of things to come, with generative AI able to produce “a significant scale of malicious output with limited human oversight, at low cost and with low barriers to entry. It is possible that the system underpinning the network is operated by high-end consumer-grade hardware and is developed by just one individual.

“We assess that a more mature, future version of the system underlying the Green Cicada Network would be extremely difficult for parties other than X to detect.”

  • thingsiplay@beehaw.org
    link
    fedilink
    arrow-up
    7
    ·
    3 months ago

    X (formerly Twitter)

    I love how everyone always need to explain what X actually stands for. If you have to explain it with the old name, why not just use the old name “Twitter” instead “X (formerly Twitter)”?

    And somehow every controversy is linked to the US, even if the controversy is about another country in the world. :D

    • Sauerkraut@discuss.tchncs.de
      link
      fedilink
      arrow-up
      9
      ·
      edit-2
      3 months ago

      Musk himself keeps calling it Twitter. He did it multiple times in his interview with Trump.

      I lowkey love that Musk refuses to change it back because it highlights the fact that the richest man in the world is a huge idiot and that capitalism is not a meritocracy.

  • Baggins@piefed.social
    link
    fedilink
    English
    arrow-up
    6
    ·
    3 months ago

    Government should not be using it whatsoever and any employees using it face disciplinary action.

    • Sauerkraut@discuss.tchncs.de
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      3 months ago

      Disagree with your second point. Punitive measures are better served as a temporary stopgap. So a better approach would be: the government should either nationalize twitter or force twitter to be open to federalization (like lemmy) and then build their own publically owned instance.

    • 0x815@feddit.orgOP
      link
      fedilink
      English
      arrow-up
      8
      ·
      3 months ago

      @technocrit@[email protected]

      Ok… But what about the zios doing the same thing in support of genocide?

      ‘Whataboutism, the rhetorical practice of responding to an accusation or difficult question by making a counteraccusation, by asking a different but related question, or by raising a different issue altogether. Whataboutism often serves to reduce the perceived plausibility or seriousness of the original accusation or question by suggesting that the person advancing it is hypocritical or that the responder’s misbehavior is not unique or unprecedented. Acts of whataboutism typically begin with rhetorical questions of the form “What about…?”’

      Source