After 17 years as publisher of science fiction and fantasy short stories at Clarkesworld, The electronic and print magazine he founded in 2006Neil Clark has become adept at weeding out the occasional plagiarism from the hundreds of submissions he receives each month.

The problem worsened during the pandemic lockdown phase, when confined-at-home workers were looking for ways to boost income. But it exploded in December, after San Francisco-based OpenAI publicly released ChatGPT, a particularly sophisticated machine for generating prose.

Of the 1,200 applications he received this month, Clark believes, 500 were automated. He’s stopped accepting submissions until at least the end of the month, but he’s confident that if he hadn’t, the synthetic pile would have reached parity with legitimate submissions by then — “and more than likely passed,” he says.

“We saw no reason to think it was slowing down,” says Clark, 56. As he did with plagiarism, he rejected machine-generated content and permanently banned its providers.

A veteran of the software industry, Clarke has developed a general grammar that enables him to automatically recognize written prose. He doesn’t share them, fearing simple tweaks could allow users to circumvent his standards. “It will still be bad stories and it will still be rejected, but it will be ten times harder for us.”

Rarely have regulators begun to consider the legal implications of software that can scour the web for raw material, including the potential for widespread copyright infringement and forgery in prose and art.

The advent of authenticity “will make generative AI attractive to malicious use where the truth is less important than the message it conveys, such as online misinformation and harassment campaigns.” Alex Engler of the Brookings Institution wrote.

The onslaught of machine-written material has become Topic A in the periodical industry, because ChatGPT and similar programs known as chatbots have demonstrated their ability to produce prose that mimics human writing to an astonishing degree.

“Everyone talks about this, and a lot of people focus on what this will mean for science publishing,” says H. Holden Thorpe, editor-in-chief of the Science Journals Group. “This whole madness has taken over the world, and everyone is in a frenzy.”

Science has imposed the strictest policy on content generated by AI tools like chatbots between tech publishers: It’s forbidden, period.

Banning posts includes “text generated from artificial intelligence, machine learning, or similar algorithm tools” as well as “shapes, images, or accompanying graphics.” Artificial intelligence software cannot be listed as an author. Science warns that it will treat a violation of this policy as “scientific misconduct.”

Thorp further explained in an editorial that using programs such as ChatGPT would violate a basic science principle that authors must certify that their submitted work is “original”. This is “enough to indicate that text written by ChatGPT is unacceptable,” he wrote: “It is, after all, stolen from ChatGPT.”

This goes far beyond the rules set by nature, which are in the top of the most prestigious scientific journals along with science. Nature magazines specify that Language generation programs cannot be supported As an author on a paper, as has been experienced in some published papers: “Any attribution of authorship carries with it responsibility for the work,” Nature explains, “and AI tools cannot bear such responsibility.”

But Nature allows researchers to use such tools in preparing their papers, as long as they “document such use in the methods sections or acknowledgments of appreciation” or elsewhere in the text.

Science has chosen to take a firmer stance, Thorpe told me, to avoid repeating the difficulties that arose from the advent of Photoshop, which enables image manipulation, in the 1990s.

“In the beginning, people did a lot of things with Photoshop on their photos that we now think were unacceptable,” he says. “When it first came out, we didn’t have a lot of rules about that, but now people like to go back to the old papers and say, ‘Look what they did with Photoshop,’ we don’t want a repeat of that.”

Science journals will wait for the scientific community to unite around standards for acceptable use of AI software before revisiting their rules, Thorpe says: “We started with a very strict set of rules. It’s a lot easier to loosen up your guidelines later than it is to tighten them.”

However, underlying concerns about ChatGPT and its AI cousins ​​are what may be exaggerated impressions of how adept they are at replicating human thought.

What has been continually overlooked, in part because of the power of noise, is how poorly the intellectual processes that humans carry out naturally, and in general, almost flawlessly, are handled.

Naive media stories about “Shine” ChatGPT Focusing on these successes “without looking seriously at the range of mistakes,” he noted Computer scientists Gary Marcus and Ernest Davis.

Helpfully, they’ve compiled Database of hundreds of occasional soaring flops by ChatGPT, Microsoft’s Bing, and other language-generating software: for example, their inability to do simple arithmetic, count to five, understand the order of events in a narrative or understand that a person can have two parents—not to mention their tendency to Fill in the gaps in their knowledge with fabrications.

AI advocates maintain that these shortcomings will eventually be fixed, but when or even if this will happen is uncertain, since human thought processes themselves are not well understood.

The glorification of ChatGPT’s supposed skills ignores that the program and other chatbots essentially scour the Internet for found content — blogs, social media, old digitized books, personal discourses by the ignorant, learned discoveries by the wise — and use algorithms to correlate what they find. together. In a way that mimics the feeling, but it isn’t.

Human-generated content is their raw material, human designers “train” them on where to find content to respond to a query, and human-written algorithms are their set of instructions.

The result is like a magician’s trick. Chatbots appear human because each line of output is a reflection of human input.

The amazement that humans experience at the apparent sensibility of chatbots is not a new phenomenon. It evokes what Joseph Weisenbaum, the inventor of Elisa, a 1960s-era natural language program that could replicate a therapist’s responses to a “patient’s” complaints, noticed in the emotional reactions of users interacting with the program.

What I didn’t realize wrote later“Can very brief exposure to a relatively simple computer program induce powerful delusional thinking in completely normal people.”

It’s true that ChatGPT is good enough at imitation to fool even educated professionals at first glance. According to a team of researchers at the Universities of Chicago and Northwestern, four professional medical reviewers managed just that Correctly picking up 68% of summaries generated by ChatGPT of scientific papers published from a stack of 25 real abstracts generated by chat. (They also incorrectly identified 14% of the original abstracts as machine-typed.)

Nature notes that “ChatGPT can write great student essayssummarize research papers, answer questions well enough to pass medical exams and generate useful computer code.”

But these are all fairly general writing classes, and they’re usually imbued with rote language. “Artificial intelligence is not creativity, but iterative,” says John Scalzi, the author of dozens of sci-fi books and short stories who objected to his publisher, Tor Books,’s use of machine-generated art for the cover of a fellow author’s latest novel.

(Tor said published statement that he was unaware that the cover art “may have been AI-generated”, but said that due to production schedules he had no choice but to move forward with the cover once he knew the source. The publisher said it “defended the creators of SFF [that is, science fiction and fantasy] Since our inception, we will continue to do so. “)

On the potential for machine-generated content to continue to be improved, “I think it’s bound to get to a ‘good enough’ level,” says Scalzi. “I don’t think AI is It would create a still artwork akin to “Blood Streak” or “How Stella Got Her Groove Back”.

At the moment, undisclosed or poorly disclosed use of chatbots was treated as an occupational crime comparable to plagiarism. Two rectors at Vanderbilt University’s Peabody College of Education and Human Development have been suspended after the school released an email in response to the February 13 shooting at Michigan State University calling on the Vanderbilt community to “work together” in Creating a safe and inclusive environment on campus.

The email carried a notice in small print that it was a “refactor” from ChatGPT. The Michigan shooter killed three students and injured five.

To be fair, Vanderbilt’s ChatGPT email wasn’t all that different from what humans on the university staff might have presented themselves, an example of condolence “thoughts and prayers” in the wake of a public tragedy, which seems empty and robotic even when produced by beings of flesh and blood.

Chatbot products have yet to break through to elite levels of creative writing or art. Clark says the motive for submitting machine-written stories to his magazine does not appear to be aspirations for creative achievement but the pursuit of a quick buck.

“A lot is coming from “Side hustle” sites on the Internet.” “They are misled by people who promote these money-making schemes.”

Others fear that the ease of creating real-looking but machine-generated prose and images will make the technology a tool for making mistakes, much as cryptocurrencies have found their most reliable use cases in scam and ransomware attacks.

This is because these programs are not ethical entities but rather tools that respond to the motives of their users and depend on the source of the material towards which they are directed. Microsoft had to Tweak the new ChatGPT-powered Bing search enginefor example, when she responded strangely and annoyinglyand even insulting to the requests of some users.

ChatGPT’s production is theoretically constrained by ethical “firewalls” that its developers devised, but are easy to avoid – “nothing more than lipstick on an immoral pig,” as Marcus notes.

He adds, “We now have the most widely used chatbots in the world, … glorified by the media, yet with moral protection barriers that just kind of work.” “There is little, if any, government regulation in place to do much about it. The possibilities are endless now for advertising, phishing farms, and rings of fake websites that are undermining online trust. It’s a disaster in the making.”