Comment by sudneo@lemm.ee on Watching the Generative AI Hype Bubble Deflate

Even if they plateaued in place where they are right now it would lead to major shakeups in humanity’s current workflow

Like which one? Because it’s now 2 years we have chatGPT and already quite a lot of (good?) models. Which shakeup do you think is happening or going to happen?

Computer programming has radically changed. Huge help having llm auto complete and chat built in. IDEs like Cursor and Windsurf.

I’ve been a developer for 35 years. This is shaking it up as much as the internet did.

@remindme@mstdn.social 1 year. Let me know about the seachange of new 10x transform based programmers that have automated me out of a job.

@horse_battery_staple Ok, I will remind you on Friday Dec 26, 2025 at 7:49 AM PST.

I quit my previous job in part because I couldn’t deal with the influx of terrible, unreliable, dangerous, bloated, nonsensical, not even working code that was suddenly pushed into one of the projects I was working on. That project is now completely dead, they froze it on some arbitrary version.
When junior dev makes a mistake, you can explain it to them and they will not make it again. When they use llm to make a mistake, there is nothing to explain to anyone.
I compare this shake more to an earthquake than to anything positive you can associate with shaking.

More business for me. As a DevOps guy, my job is to create automation to flag “ terrible, unreliable, dangerous, bloated, nonsensical, not even working code”

And so, the problem wasn’t the ai/llm, it was the person who said “looks good” without even looking at the generated code, and then the person who read that pull request and said, again without reading the code, “lgtm”.

If you have good policies then it doesn’t matter how many bad practice’s are used, it still won’t be merged.

The only overhead is that you have to read all the requests but if it’s an internal project then telling everyone to read and understand their code shouldn’t be the issue.

The problem here is that a lot of the time looking for hidden problem is harder than writing good code from scratch. And you will always be at a danger that llm snuck some sneaky undefined behaviour past you. There is a whole plethora of standards, conventions, and good practices that help humans to avoid it, which llm can ignore at any random point.
So you’re either not spending enough time on review or missing whole lot of bullshit. In my experience, in my field, right now, this review time is more time consuming and more painful than avoiding it in the first place.
Don’t underestimate how degrading and energy sucking it is for a professional to spend most of the working time sitting through autogenerated garbage, and how inefficient it is.

This is a problem with your team/project. It’s not a problem with the technology.

A technology that makes people put bad code is a problematic technology. If your team/project managed to overcome it’s problems so far doesn’t mean it is good or overall helpful. Peoole not seeing the problem is actually the worst part.

Sir, I use it to assist me in programming. I don’t use it to write entire files or functions. It’s a pattern recognizer.

Your team had people who didn’t review code. That’s a problem.

I hardly see it changed to be honest. I work in the field too and I can imagine LLMs being good at producing decent boilerplate straight out of documentation, but nothing more complex than that.

I often use LLMs to work on my personal projects and - for example - often Claude or ChatGPT 4o spit out programs that don’t compile, use inexistent functions, are bloated etc. Possibly for languages with more training (like Python) they do better, but I can’t see it as a “radical change” and more like a well configured snippet plugin and auto complete feature.

LLMs can’t count, can’t analyze novel problems (by definition) and provide innovative solutions…why would they radically change programming?

I hardly see it changed to be honest. I work in the field too and I can imagine LLMs being good at producing decent boilerplate straight out of documentation, but nothing more complex than that.

I think one of the top lists on advent of code this year is a cheater that fully automated the solutions using LLMs. Not sure which LLM though, I use LLMs quite a bit and ChatGPT 4o frequently tells me nonsense like “perhaps subtracting by zero is affecting your results” (issues I thought were already gone in GPT 4, but I guess not, Sonnet 3.5 does a bit better in this regard).

Maybe some postmortem analysis will be interesting. The AoC is also a context in which the domain is self-contained and there is probably a ton of training material on similar problems and tasks. I can imagine LLM might do decently there.

Also there is no big consequence if they don’t and it’s probably possible to bruteforce (which is how many programming tasks have been solved).

I think you’re spot on with LLMs being mostly trained on these kinds of tasks. Can’t say I’m an expert in how to build a training set, but I imagine it’s quite easy to do with these kinds of problems because it’s easy to classify a solution as correct or incorrect. This is in contrast to larger problems which are less guided by algorithmic efficiency and more by sound design/architecture.

Still, I think it’s quite impressive. You don’t have to go very far back in time to have top of the line LLMs unable to solve these kinds of problems.

Also there is no big consequence if they don’t and it’s probably possible to bruteforce (which is how many programming tasks have been solved).

Usually with AoC part 1 is brute-forceable, but part 2 is not. Very often part 1 is to find the 100th number, and part 2 is to find the 1 000 000 000 000th number or something. Last year, out of curiosity, I had a brute-force solution for one problem that successfully completed on ~90% of the input. Solution was multi-threaded and running on a 16 core CPU for about 20 days before I gave up. But the LLMs this year (not sure if this was a problem last year) are in the top list of fastest users to solve the problems.

Just to precise, when I said bruteforce I didn’t imagine a bruteforce of the calculation, but a brute force of the code. LLMs don’t really calculate either way, but what I mean is more: generate code -> try to run and see if tests work -> if it doesn’t ask again/refine/etc. So essentially you are just asking code until what it spits out is correct (verifiable with tests you are given).

But yeah, few years ago this was not possible and I guess it was not due to the training data. Now the problem is that there is not much data left for training, and someone (Bloomberg?) reported that training chatGPT 5 will cost billions of dollars, and it looks like we might be near the peak of what this technology could offer (without any major problem being solved by it to offset the economical and environmental cost).

Just from today https://www.techspot.com/news/106068-openai-struggles-chatgpt-5-delays-rising-costs.html

You’re missing it. Use Cursor or Windsurf. The autocomplete will help in so many tedious situations. It’s game changing.

ChatGPT 4o isn’t even the most advanced model, yet I have seen it do things you say it can’t. Maybe work on your prompting.

That is my experience, it’s generally quite decent for small and simple stuff (as I said, distillation of documentation). I use it for rust, where I am sure the training material was much smaller than other languages. It’s not a matter a prompting though, it’s not my prompt that makes it hallucinate functions that don’t exist in libraries or make it write code that doesn’t compile, it’s a feature of the technology itself.

GPTs are statistical text generators after all, they don’t “understand” the problem.

It’s also pretty young, human toddlers hallucinate and make things up. Adults too. Even experts are known to fall prey to bias and misconception.

I don’t think we know nearly enough about the actual architecture of human intelligence to start asserting an understanding of “understanding”. I think it’s a bit foolish to claim with certainty that LLMs in a MoE framework with self-review fundamentally can’t get there. Unless you can show me, materially, how human “understanding” functions, we’re just speculating on an immature technology.

As much as I agree with you, humans can learn a bunch of stuff without first learning the content of the whole internet and without the computing power of a datacenter or consuming the energy of Belgium. Humans learn to count at an early age too, for example.

I would say that the burden of proof is therefore reversed. Unless you demonstrate that this technology doesn’t have the natural and inherent limits that statistical text generators (or pixel) have, we can assume that our mind works differently.

Also you say immature technology but this technology is not fundamentally (I.e. in terms of principle) different from what Weizenabum’s ELIZA in the '60s. We might have refined model and thrown a ton of data and computing power at it, but we are still talking of programs that use similar principles.

So yeah, we don’t understand human intelligence but we can appreciate certain features that absolutely lack on GPTs, like a concept of truth that for humans is natural.

Exactly this. Things have already changed and are changing as more and more people learn how and where to use these technologies. I have seen even teachers use this stuff who have limited grasp of technology in general.

My kid’s teachers had what I thought was a fantastic approach - have the kids write an outline. Use an LLM to generate an essay from that outline, then critique the essay

I don’t know anything about the online news business but it certainly appears to have changed. Most of it is dreck, either way, and those organizations are not a positive contributor to society, but they are there, it is a business, and it has changed society

I don’t see the change. Sure, there are spam websites with AI content that were not there before, but is this news business at all? All major publishers and newspapers don’t (seem to) use AI as far as I can tell.

Also I would argue this is no much of a change except maybe in simplicity to generate fluff. All of this existed already for 20 years now, and it’s a byproduct of the online advertisement business (that for sure was a major change in society!). AI pieces are just yet another way to generate content in the hope of getting views.

Review of legal documents.

Oh boy…what can possibly go wrong for documents where small minutiae like wording can make a huge difference.

Creating legal documents, no. Reviewing legal documents for errors and inaccuracies totally.

No, not that either. Unless you consider “use LLM to summarize the changes/errors/inaccuracies, then have a human read the whole thing again” an improvement over “just have a human read the whole thing”.

Because LLM will do all these things:

point you toward issues
point you toward non-issues
not point you toward issues
change stuff even when “instructed” not to

If there is one thing you don’t want to throw an LLM at without full, unbiased review, it’s documents where the wording is legally binding. And if you have to do a full, unbiased review to begin with, where you can’t even trust your tool to have highlighted all the important parts, you may as well not bother with the tool.

I really can’t see this being done by any sane person. Why would you have a generator of text reviewing stuff (besides grammar)? Do you have any reference of some companies doing this, perhaps?

Its complex pattern matching and looking up existing case law online. This work has been outsourced to contracting companies for at least 7 years that I’m aware of. If it is something that can be documented in a run book for non professionals to do for twenty cents on the dollar then there is no reason it can’t be done by a script for .002.

Aside from a handful of business that tried to do that and failed miserably, some of them failing in actual court, you mean?

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Watching the Generative AI Hype Bubble Deflate(ash.harvard.edu)

Technology

!technology@lemmy.world

Our Rules

Approved Bots

Community stats

Community moderators