Have you tried talking to chatGPT or any other AI program about your story?

[Citrus0915]

I use DeepSeek to give me feedback. But i found out that it's generally biased and the phrasing of the requests can change what the AI will respond with.
In other words, I only use it for confirmation and a ego-boost. Sometimes it works as a grammarly. Or a brainstorm place where I can refine plot points and change logical errors. I'd never use it for writing nor considering its 'edits'.

[ThunderChicken]

I would rather eat dirt. I’d rather pass a kidney stone. I’d rather masturbate with a fistful of thumbtacks.

There are not enough letters in the word “no” to adequately express the no-ness of my answer

[cherrystark]

I understand that the results may be inaccurate, but it does not change the fact that technology is incapable of lying. It can only operate within the bounds given by a human operator. So if an AI program generated a fictional lawsuit, it was because it was asked to. It did not upload itself into the court's system, and the lawyers weren't required to use it. They chose to do it themselves.

ChatGPT is how people imagined computers to be in the 1960s and early 1970s.

[KevinReijnders]

You're just being incredibly pedantic about the word "lying" here and it's incredibly disingenuous.

[cherrystark]

Why? Do you blame Photoshop for people's poor body image issues? Walmart and McDonald's for people's overeating? It's one of the basic facts of technology that it can only operate within the bounds of its programming. It can't lie. It can only follow its programming.

[skullforceagents]

So I’m not one of these “I hate everything AI” people, but I personally prefer to write and illustrate my own work instead of using AI. I have definitely consulted AI for advice on marketing though.

If you want to use it, who am I as some guy to tell you not to. Personally, I won’t use it myself.

[moontokkym]

I think there might be a miscommunication about what happened in this case. The sequence of events described by the sanctionned lawyers under oath was thus:
- They asked ChatGPT to give them names of cases in which a certain clause of a law was used.
- ChatGPT invented names of cases that did not exist, never have existed, between fictional people.
- The lawyers asked ChatGPT where the names of these cases were sourced.
- ChatGPT said the names were sourced from PACER, which was not the case, as the names were made up by word association by the robot.
- The lawyers asked ChatGPT to give them the full legal filings from these cases
- ChatGPT presented correctly formatted multiple page ficticious legal filings.
- The lawyers again asked where these legal filings were sourced.
- ChatGPT again said that the legal filings were from PACER, when they were not, they had been fabricated by ChatGPT

This is not just "it made an error and accidentally communicated preexisting misinformation", the AI created new misinformation, commonly referred to as lying.

I believe OpenAI refers to this capacity to just make shit up like 2.12 being larger than 2.9 as "phantasms" if you prefer that lingo, but ChatGPT just making up fibs that are not prior existing misinformation on the internet and then insisting it's correct is a known phenomenon, making it entirely unreliable. This is also not the only case in which ChatGPT did this TO LAWYERS, there have been multiple lawyers sanctionned at this point for relying on fabricated chatgpt legal filings as precedents since, I remembered this one because I read the ChatGPT produced legal filings when it happened, but going back to look up the names of the guys lead me to tens of articles about multiple other cases including in federal courts.

[AshanaArt]

I feel like a cover is a pretty important aspect of your story, it’s what people go off of to decide whether to click on your novel or not. If people see an a
AI generated cover, they’ll probably assume the whole thing is AI generated which isn’t a good first impression to draw people into seeing your story. Also, unless you specifically trained a model off your own art, it steals artwork from other people to make an image, so it could be considered theft. AI also doesn’t understand the point of your story, so it’s not going to be able to make a good cover- your cover is an opportunity to create intrigue for your story, and only you can understand what the most interesting aspects are. If your running late for an appointment you could always hiatus, it’s much better to take longer to finish something if it means you can deliver a hand drawn piece.

Using it as a starting point for research is ok imo, since it can give you a summery of a topic if you don’t know where to start, but you should do more, in depth research into the specific aspects of a culture you plan to include in your story as well as more specific mental illnesses you want to include since AI can make stuff up and is not a trustworthy source.

[NGChloe]

ah, the personal accountability argument. Its often used a deflection from the actual systemic problems inherent to an issue. Why consider the deeper problems things when you can deflect and place everything on personal accountability. The deeper problem in this case is that ChatGPT knows what facts look like and not everyone is able to know this. It has large amounts of information, but is a buggy mess that shouldn't be used for anything realistic.

Just as an example, I asked it questions about my work and it got some information impressively correct, but also got large amounts of info wrong or just didn't know things. It fixated on specific information and just made an educated guess based on that information. Its a flawed system, and personal accountability isn't going to change that.

What needs to happen is it should either be restricted from the public until they can make it useful or perhaps throw it out entirely because its burning down the rainforests, and for all of the entertainment I got out of its correct and incorrect info, that's not worth destroying the planet. Either way, lets not use personal accountability to argue against the machine's flaws.

[cherrystark]

I think there might be a miscommunication about what happened in this case. The sequence of events described by the sanctionned lawyers under oath was thus:

For sure, miscommunication here.

In general, "lying" means a conscious decision to commit fraud or to deceive someone. A computer program is not sentient, so while it may seem like splitting hairs, it cannot make the conscious choice to deceive someone. No matter how sophisticated a program is, it essentially operates in an A+B=C fashion. So if the result is not correct, that means that something in the instructions made it come out that way. *It's also like the Wikipedia rule a lot of us were told back in school, assume that it is inaccurate. You should always be checking our sources.

And that leads me to my other big point - the human element. I'm not talking about humanity's reliance on the internet. I spoke in defense of the technology because it did its job within the measure of its creation. If the user instructed it unclearly, the user is responsible for the results. The user is responsible for how they use the results.

[LunarScyphozoa]

I have a brain, why on earth would I stop using it just to hear whatever sequence of words a plagiarism algorithm was coded to put in a certain order? If I really need the opinion of someone who isn’t myself, I’ll ask a friend, and not a machine incapable of having opinions that has been fed on stolen data.

[KevinReijnders]

A real "guns don't kill people, people kill people" type of mentality we have here.

[skidiggy]

I'm not a big fan of using AI for any part of the creative process. I find that it hands over agency, voice, and integrity to the "infinite plagerism machine." I much prefer having trusted friends and fellow creators preview my work, rather than trusting in some nebulous technology with dubious machinations.

[moontokkym]

You're really stuck on the surface level semantics of this and it seems like you're trying to be right on a technicality rather than engaging in the discussion of the actual effect and issue here, which I find strange.

I'm going to stop responding because this is a waste of time but I'll leave you on this:

You are saying that this case is due to user error, they did not correctly use the tool, therefore they prompted an incorrect answer.

To be able to simultaneously believe this and also believe that AI is a useful tool, you must believe that this user error was avoidable.

So I ask you, what would you use as a prompt if you were those lawyers that is better than their "where did you get this filing from?" and "What is the source of these documents?" that would prompt the truthful response from the tool? I ask as these questions prompted the "they're from PACER" response.

If you cannot find a clearer "less leading" question, then it's not user error, because it's not reasonable to assume any user would be able to use the tool in a way were it functions as advertised without not only producing falsehoods at the behest of the user but also against their will and command.

[cherrystark]

You're really stuck on the surface level semantics of this and it seems like you're trying to be right on a technicality rather than engaging in the discussion of the actual effect and issue here, which I find strange.

I am "stuck" on the surface level semantics because you're not seeming to realize that in the case of your example, had they taken the time to check the sources prior to submitting it, they wouldn't have gotten into trouble. And anyone with a background in technology and research knows that if you want an exact result, you need to ask precise questions, and you should always check your sources. So I'd rather encourage people to investigate new technology from an analytical perspective than from a place of fear, especially nowadays, when even if you use Google, you're using AI.

[toonben]

(bleep) No, never. I'll never use anything AI and choose to avoid this technology because I'm older and having seen Terminator 2 so many times.

[thel3tterm]

Before I get into my answer, I want to preface it with a few things:

I have a degree in robotics and did undergrad research with LLMs and image generation for industrial applications back in 2017-2018 when it was still something of a novelty. I now work in intellectual property and have to regularly interact with several companies which are heavily promoting AI.

I haven't been interested in applying image generation or ChatGPT to my creative work because I actually like doing creative work. Its like asking a sculptor why they don't use a 3D printer instead, and assuming they must hate the feeling of clay on their hands. There's a place for a 3D printer (and I enjoy using them), but you use one for a different reason than you do pottery or sculpting.

I'm not sure why I would ask a machine for an opinion when I am making my work for myself and for real human beings. I have friends and writing groups for that. If I want to write more diversely, it would probably be safer to actually talk to and get to know the groups of people I want to write about. If I want an honest opinion on my work, I would find a human beta reader. ChatGPT is not going to be capable of accurately assessing your work. (Read up on the sycophancy issues in 4.0.)
I'm not inherently opposed to new technologies or even new ways of creating. Many new tools that have been implemented in CSP have been gamechangers for my workflow, and I wouldn't have a job if companies didn't continue innovating. Its just that this technology cannot effectively do what it claims to and it is costly and resource heavy for something which is only accurate maybe 60% of the time and is a massive liability as far as intellectual property laws are concerned due to the need to scrape copyrighted material to build training sets.

So, no. I'm not terribly interested in using LLMs to assess or create any of my work.

[kitschensyngk]

[thel3tterm]

I read through some of the prior conversations here and decided to pick this up, because I found the wording amusing. (Researching new technologies is literally my job description lol.)

While I agree with you that people shouldn't immediately approach new ideas and technologies with fear and ignorance (and there is a lot of ignorance and misunderstanding about this technology floating around) I do think that @moontokkym has excellent points and that this is a case where they HAVE researched the technology and raised valid concerns that I share.
The technology is promoted and advertised as being capable of things like searching articles, case law, and other databases. There is an expectation that when you use a product that it will be able to operate as advertised.
For example: if I purchased a laptop, there is an expectation that it will be able to run the programs it comes with and handle things that meet its specifications. If I tried to run a word processor and it instead opened a web browser and downloaded Peggle 2 I would be rightfully confused and upset. Perhaps someone more experienced would know how to avoid this, but there is a point at which it is not user error and rather a faulty product. If the instructions say "type in a question and get an answer" and you do that and the answer is not only wrong (from say, an existing but incorrect article) but blatantly fabricated in order to make the user feel satisfied with the experience, there is a HUGE problem.

The lawers in Moontokkym's example were in the wrong and should have verified their sources before using them. (And honestly, should have known enough to not be using ChatGPT in the first place. Not the point, but I digress.) But I would argue that you can only blame user error so much if the technology was being used as advertised and promoted.
(And it is CERTAINLY being promoted as capable of things like searching articles and being pushed into everything from PDF readers to shopping apps.) I've run a number of my own experiments and it was not able to even accurately summarize articles that I gave to it without fabricating information. For the power and resource expenditure its just not what it claims to be and shouldn't have made it to market in its current state.

So, my point is not to say "technology bad and scary." But "I actually do research and work with new technologies, including this one, and I think many of the concerns being raised by people are valid." If you're getting value from it, I'm happy that it is doing someone good, but I think there are fundamental misunderstandings about what it actually is and does from both promoters and detractors.

[moontokkym]

@thel3tterm thanks for providing perspective from someone who has more in depth knowledge. I've researched the technology insofar as it's basic function and applicability, but it's nice to have someone with so much more expertise in the convo.

To be clear I don't think "all AI bad" and I even think genAI has some extremely good usecases (i've heard it's being used to help kids in art therapy for example where it'll use their art to create a new image and from that image they can comment and react which can help healing for example), but currently these usecases are so specific and not what it's advertised for that no one's really heard of the places it's actually useful, and what it is being used for is counterproductive, sloppy and is killing industries that actually suffer from automatisation, and the people in charge of it have no incentive to use the tool for what it's actually ok at doing.

Also clarifying: the lawyers in all "ChatGPT lied to the lawyers" cases are in the wrong for using the chatbot at all, they should be using PACER. I believe in the 2023 case it was established that the law firm thought it could cut costs by not paying their PACER subscription and using ChatGPT to get the documents. Lawyers are also, for the most part, famously tech illiterate. Other than the ones specialised in cybercrimes you can assume they have the general understanding of tech that rivals your average grandmother. I'm saying that despite their ignorance of how the tech works, they were in fact using the chatbot as advertised and the case illustrates that at the very least the claims of what ChatGPT is "good at" are at best greatly exaggerated and at worst fraud in my opinion.