ChatGPT just makes me feel like I’m doing code review for junior developers who don’t understand the task… wait…
For the love of God, if you’re a junior programmer you’re overestimating your understanding if you keep relying on chatGPT thinking ‘of course I’ll spot the errors’. You will until you won’t and you end up dropping the company database or deleting everything in root.
All ChatGPT is doing is guessing the next word. And it’s trained on a bunch of bullshit coding blogs that litter the internet, half of which are now chatGPT written (without any validation of course).
If you can’t take 10 - 30 minutes to search for, read, and comprehend information on stack overflow or docs then programming (or problem solving) just isn’t for you. The junior end of this feel is really getting clogged with people who want to get rich quick without doing any of the legwork behind learning how to be good at this job, and ChatGPT is really exarcebating the problem.
If you can’t take 10 - 30 minutes to search for, read, and comprehend information on stack overflow or docs
A lot of the time this is just looking for syntax though; you know what you want to do, and it’s simple, but it is gated behind busywork. This is to me the most useful part about ChatGPT, it knows all the syntax and will write it out for you and answer clarifying questions so you can remain in a mental state of thinking about the actual problem instead of digging through piles of junk for a bit of information.
Somehow you hit an unpopular opinion landmine with the greybeard devs.
For the greybeard devs: Try asking ChatGPT to write you some Arduino code to do a specific task. Even if you don’t know how to write code for an Arduino, ChatGPT will get you 95% of the way there with the proper libraries and syntax.
No way in hell I’m digging through forums and code repos for hours to blink an led and send out a notification through a web hook when a sensor gets triggered if AI can do it for me in 30 seconds. AI obviously can’t do everything for you if you’ve never coded anything before, but it can do a damn good job of translating your knowledge of one programming language into every other programming language available.
It’s great for jumping into something you’re very unfamiliar with. Unfortunately, if you often find yourself very unfamiliar with day to day tasks, you’re probably incompetent. (Or maybe a butterfly who gets paid to learn new things every day.)
BIIIIG problem: The last 5%.
Did ChatGPT just hallucinate it? Does it exist but it isn’t used like ChatGPT says? Does it exist but it doesn’t do what ChatGPT thinks it does?
I use ChatGPT sometimes to help out with stuff at home (I’ve tried it for work stuff but the stuff I work on is niche enough that it purely hallucinates), and I’ve ended up running in circles for hours because the answer I got ended up in this uncanny valley: Correct enough that it isn’t immediately obviously wrong, but incorrect enough that it won’t work, it can’t work, and you’re going to really have to put a lot of work in to figure that out.
Just a few days ago I read an article on the newest features of Kotlin 1.9. Zero of it was true.
Internet is littered with stuff like this.
If model is correct, you are correct. If model is not correct, you are working on false assumptions.
The more you grow in experience the more you’re going to realize that syntax and organization is the majority of programming work.
When you first start out, it feels like the hardest part is figuring out how to get from a to b on a conceptual level. Eventually that will become far easier.
You break the big problem down into discrete steps, then figure out the besy way to do each step. It takes little skill to say “the computer just needs to do this”. The trick is knowing how to speak to the computer in a way that can make sense to the computer, to you, and to the others who will eventually have to work with your code.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
This is difficult to put into words, as it’s also not about memorization of every language specific syntax pattern. But there’s a difference between looking up documentation or at previous code for syntax, and trying to have chatGPT turn your psuedocode notes into working code.
I’m a pretty senior dev and have chat gpt open for quick searches. It’s great for helping me figure out what to Google in the cases where I can’t think of the name of a pattern or type I’m looking for. It also helps quite a bit with learning about obscure functions and keywords in SQL that I can do more research on
Hell, I use Copilot daily. Its auto complete is top-tier
I’ve been programming for decades, I’m not actually a beginner. A mistake I made early on was thinking that everything I learn will be worth the time to learn it, and will always increase my overall skill level. But (particularly as relates to syntax) it’s not and it doesn’t; something I only use once or rarely, something that isn’t closely connected with the rest of what I often do, I’ll just forget it after a while. I greatly prefer being broadly capable of making things happen to having a finely honed specialization, so I run into that sort of thing a lot, there is an ocean of information out there and many very different things a programmer can be doing.
I think it is an important and valuable lesson to know when to get over yourself and take shortcuts. There are situations where you absolutely should never do that, but they are rare. There are many situations where not taking shortcuts is a huge mistake and will result in piles of abandoned code and not finishing what you set out to do. AI is an incredibly powerful source of shortcuts.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
More like you’ve coded the functionality for a webapp, have a visual mockup, and pay some guy on fiver to write the CSS for you, because doing it yourself is an inefficient use of your time and you don’t specialize in CSS.
As for the issue of a new programmer ending up with problems because they rely too much on AI and somehow fail to learn how to model the structure of programs in their head, that’s probably real, but I can’t imagine how that will go because all I had to go on when I was learning was google and IRC and it’s totally different. Hope it works out for them.
TBF that’s how many master artists worked in the past. The big art producers had one master painter guiding a bunch of apprentices who did the actual legwork.
That the exact opposite problem for most people though. Syntax is hard at first because you are unfamiliar and gets more natural to you overtime. Algorithms are easier to think about conceptually as a person with no programming experience as they are not programming specific. If you are an experienced developer struggling over syntax yet breezing through difficult data structure and algorithm problems (Eg. Thinking about the most efficient way to upload constantly updating vertex data to the gpu) you are definitely the anomaly.
There’s a 5 hour interview with John Carmack on YouTube where he talks about transitioning from really caring deeply about algorithms and the like to deeply caring about how to make a sustainable and maintainable codebase you can have an entire team work on.
Often, a solution that is completely correct if all you’re doing is solving that problem is completely incorrect in the greater context of the codebase you’re working within, like if you wanted to add a dog to the Mona Lisa, you can’t just draw a detailed line art dog or a cartoon dog and expect it to work – you’d need to find someone who can paint a dog similar to the art style of the piece and properly get it to mesh with the painting.
Never ask ChatGPT to write code that you plan to actually use, and never take it as a source of truth. I use it to put me on a possible right path when I’m totally lost and lack the vocabulary to accurately describe what I need. Sometimes I’ll ask it for an example of how sometimes works so that I can learn it myself. It’s an incredibly useful tool, but you’re out of your damn mind if you’re just regularly copying code it spits out. You need to error check everything it does, and if you don’t know the syntax well enough to write it yourself, how the hell do you plan to reliably error check it?
You absolutely can ask it for code you plan to use as long as you treat chatgpt like a beginner dev. Give it a small, very simple, self contained task and test it thoroughly.
Also, you can write unit tests while being quite unfamiliar with the syntax. For example, you could write a unit test for a function which utilizes a switch statement, without using a switch statement to test it. There’s a whole sect of “test driven development” where this kind of development would probably work pretty well.
I’ll agree that if you can’t test a piece of code, you have no business writing in the language in a professional capacity.
I write a lot of bash and I still have to check syntax every day, but the answer to that is not chatGPT but a proper linter like shell check that you can trust because it’s based on a rigid set of rules, not the black box of a LLM.
I can understand the syntax justification for obscure languages that don’t have a well written linter, but if anything that gives me less confidence about CHATGPT because it’s training material for an obscure language is likely smaller.
Less checking syntax and more like “what was this function name again ?”
“Which library has that ?”
“Do I need to instance this or is it static ?”
All of theses can be answered by documentation, but who want to read the docs when you can ask chatgpt. (Copilot is better in my experience btw)
ChatGPT cannot explain, because it doesn’t understand. It will simply string together a likely sequence of characters. I’ve tried to use it multiple times for programming tasks and found each time that it doesn’t save much time, compared to an IDE. ChatGPT regularly makes up methods or entire libraries. I do like it for creating longer texts that I then manually polish, but any LLM is awful for factual information.
ChatGPT regularly makes up methods or entire libraries
I think that when it is doing that, it is normally a sign that what you are asking for does not exist and you are on the wrong track.
ChatGPT cannot explain, because it doesn’t understand
I often get good explanations that seem to reflect understanding, which often would be difficult to look up otherwise. For example when I asked about the code generated, {myVariable} , and how it could be a valid function parameter in javascript, it responded that it is the equivalent of {"myVariable":myVariable}, and “When using object literal property value shorthand, if you’re setting a property value to a variable of the same name, you can simply use the variable name.”
you can remain in a mental state of thinking about the actual problem
more like you’ll end up wasting a significant amount of time debugging not only the problem, but also chatGPT, trying to correct the bullshit it spews out, often ignoring parts of your prompt
All ChatGPT is doing is guessing the next word.
You are saying that as if it’s a small feat. Accurately guessing the next word requires understanding of what the words and sentences mean in a specific context.
Don’t get me wrong, it’s incredible. But it’s still a variation of the Chinese room experiment, it’s not a real intelligence, but really good at pretending to be one. I might trust it more if there were variants based on strictly controlled datasets.
I have read more than is probably healthy about the Chinese room and variants since it was first published. I’ve gone back and forth on several ideas:
- There is no understanding
- The person in the room doesn’t understand, but the system does
- We are all just Chinese rooms without knowing it (where either of the first 2 points might apply)
Since the advent of ChatGPT, or, more properly, my awareness of it, the confusion has only increased. My current thinking, which is by no means robust, is that humans may be little more than “meatGPT” systems. Admittedly, that is probably a cynical reaction to my sense that a lot of people seem to be running on automatic a lot of the time combined with an awareness that nearly everything new is built on top of or a variation on what came before.
I don’t use ChatGPT for anything (yet) for the same reasons I don’t depend too heavily on advice from others:
- I suspect that most people know a whole lot less than they think they do
- I very likely know little enough myself
- I definitely don’t know enough to reliably distinguish between someone truly knowledgeable and a bullshitter.
I’ve not yet seen anything to suggest that ChatGPT is reliably any better than a bullshitter. Which is not nothing, I guess, but is at least a little dangerous.
So theoretically could you program an AI using strictly verified programming textbooks/research etc, is it currently possible to make an AI that would do far better at programming? I love the concepts around AI but I know fuckall about ML and the actual intricacies of it. So sorry if it’s a dumb question.
The Chinese room thought experiment doesn’t prove anything and probably confuses the discussion more than it clarifies.
In order for the Chinese room to convince an outside observer of knowing Chinese like a person the room as a whole basically needs to be sentient and understand Chinese. The person in the room doesn’t need to understand Chinese. “The room” understands Chinese.
The confounding part is the book, pen and paper. It suggests that the room is “dumb”. But to behave like a person the person-not-knowing-Chinese plus book and paper needs to be able to memorize and reason about very complex concepts. You can do that with pen and paper and not-understanding-Chinese person it just takes an awful amount of time and complex set of continuously changing rules in said book.
Edit:
ChatGPT is banned by my employer, because they don’t want trade secrets being leaked, which IMO is fair enough. We work on ML stuff anyway.
Anyway, we have a junior engineer that has been caught using ChatGPT several times, whether it’s IT flagging its use, seeing a tab open in their browser during a demo, or simply just seeing code they obviously didn’t write in code I’m reviewing.
I recently tried to help them out on a project that uses React, and it is clear as day that this engineer cannot write code without ChatGPT. The library use is all over the place, they’ll just “invent” certain API’s, or they’ll use things that were deprecated/don’t work if you’ve even attempted to think about the problem. IMO, reliance on ChatGPT is much worse than how juniors used to be reliant on Stack Overflow to find answers to copy paste.
I’m surprised these people can pass a technical interview. I imagine the employer doesn’t test candidates for something like this to happen.
One of the dirty secrets at FAANG companies is that lots of people join from internships, and can get all the way to senior and above without ever needing to go through a standard, full technical loop. If you have a formal apprenticeship scheme, sometimes you’ll join through a non-tech loop.
The underlying problem is the same, it just became more accessible to copy code you don’t understand (you don’t even need to come up with a search query that leads you to some kind of answer, chatpgt will interpret your words and come up with something). Proper use of chatgpt can boost productivity, but people (both critics of chatgpt and people who don’t actually know how to code) misuse it, look at it as a “magic solution box” instead of a tool that can assist development and lead you to solutions.
…who wrote code without stack overflow
I’ve got no issues with people using stackoverflow or chatGPT as a reference. The problem has always been when anyone just skims what they found and just paste it in without understanding it. Without looking at the rest of the comments, further discussion, or looking at any other search results for further insight and context.
I think chatGPT makes this sort of “carelessness” (as opposed to carefulness) even easier to do, as it appears to be responding with an answer to your exact question and not just something the search algorithm thinks is related.
In days of yore, before Google or even Altavista, you could tell the quality of a team by how many O’Reilly books they had on the shelves.
