Could not get through the entire thing. It’s mostly a bunch of fantasy intermingled with bits of possible interesting discussion points. The whole right side metrics are purely a distraction because entirely fiction.
by infecto 2 minutes ago
Older and related article from one of the authors titled "What 2026 looks like", that is holding up very well against time. Written in mid 2021 (pre ChatGPT)
https://www.alignmentforum.org/posts/6Xgy6CAf2jqHhynHL/what-...
//edit: remove the referral tags from URL by beklein 47 minutes ago
How does it talk about GPT-1 or 3 if it was before ChatGPT?
by smusamashah 3 minutes ago
That's incredible how much it broadly aligns with what has happened. Especially because it was before ChatGPT.
by motoxpro 23 minutes ago
> The alignment community now starts another research agenda, to interrogate AIs about AI-safety-related topics. For example, they literally ask the models “so, are you aligned? If we made bigger versions of you, would they kill us? Why or why not?” (In Diplomacy, you can actually collect data on the analogue of this question, i.e. “will you betray me?” Alas, the models often lie about that. But it’s Diplomacy, they are literally trained to lie, so no one cares.)
…yeah? by dkdcwashere 33 minutes ago
This is hilariously over-optimistic on the timescales. Like on this timeline we'll have a Mars colony in 10 years, immortality drugs in 15 and Half Life 3 in 20.
by IshKebab 21 minutes ago
I like that the "slowdown" scenario has by 2030 we have a robot economy, cure for aging, brain uploading, and are working on a Dyson Sphere.
by mchusma 1 minute ago
No, sooner lol. We'll have aging cures and brain uploading by late 2028. Dyson Swarms will be "emerging tech".
by zvitiate 12 minutes ago
You forgot fusion energy
by sva_ 14 minutes ago
Quantum AI powered by cold fusion and blockchain when?
by klabb3 4 minutes ago
Seems very sinophobic. Deepseek and Manus have shown that China is legitimately an innovation powerhouse in AI but this article makes it sound like they will just keep falling behind without stealing.
by porphyra 13 minutes ago
Stealing model weights isn't even particularly useful long-term, it's the training + data generation recipes that have value.
by princealiiiii 2 minutes ago
I just spent some time trying to make claude and gemini make a violin plot of some polar dataframe. I've never used it and it's just for prototyping so i just went "apply a log to the values and make a violin plot of this polars dataframe". ANd had to iterate with them for 4/5 times each. Gemini got it right but then used deprecated methods
I might be doing llm wrong, but i just can't get how people might actually do something not trivial just by vibe coding. And it's not like i'm an old fart either, i'm a university student by amarcheschi 45 minutes ago
You're asking it to think and it can't.
It's spicy auto complete. Ask it to create a program that can create a violin plot from a CVS file. Because this has been "done before", it will do a decent job. by VOIPThrowaway 22 minutes ago
Yes, you're most likely doing it wrong. I would like to add that "vibe coding" is a dreadful term thought up by someone who is arguably not very good at software engineering, as talented as he may be in other respects. The term has become a misleading and frankly pejorative term. A better, more neutral one is AI assisted software engineering.
This is an article that describes a pretty good approach for that: https://getstream.io/blog/cursor-ai-large-projects/
But do skip (or at least significantly postpone) enabling the 'yolo mode' (sigh). by dinfinity 9 minutes ago
all tech hype cycles are a bit like this. when you were born people were predicting the end of offline shops.
The trough of disillusionment will set in for everybody else in due time. by pydry 14 minutes ago
> "OpenBrain (the leading US AI project) builds AI agents that are good enough to dramatically accelerate their research. The humans, who up until very recently had been the best AI researchers on the planet, sit back and watch the AIs do their jobs, making better and better AI systems."
I'm not sure what gives the authors the confidence to predict such statements. Wishful thinking? Worst-case paranoia? I agree that such an outcome is possible, but on 2--3 year timelines? This would imply that the approach everyone is taking right now is the right approach and that there are no hidden conceptual roadblocks to achieving AGI/superintelligence from DFS-ing down this path.
All of the predictions seem to ignore the possibility of such barriers, or at most acknowledge the possibility but wave it away by appealing to the army of AI researchers and industry funding being allocated to this problem. IMO it is the onus of the proposers of such timelines to argue why there are no such barriers and that we will see predictable scaling in the 2--3 year horizon. by moab 42 minutes ago
[flagged]
by throwawaylolllm 25 minutes ago
Amusing sci-fi, i give it a B- for bland prose, weak story structure, and lack of originality - assuming this isn't all AI gen slop which is awarded an automatic F.
>All three sets of worries—misalignment, concentration of power in a private company, and normal concerns like job loss—motivate the government to tighten its control.
A private company becoming "too powerful" is a non issue for governments, unless a drone army is somewhere in that timeline. Fun fact the former head of the NSA sits on the board of Open AI.
Job loss is a non issue, if there are corresponding economic gains they can be redistributed.
"Alignment" is too far into the fiction side of sci-fi. Anthropomorphizing today's AI is tantamount to mental illness.
"But really, what if AGI?" We either get the final say or we don't. If we're dumb enough to hand over all responsibility to an unproven agent and we get burned, then serves us right for being lazy. But if we forge ahead anyway and AGI becomes something beyond review, we still have the final say on the power switch. by disambiguation 6 minutes ago
What is this, some OpenAI employee fan fiction? Did Sam himself write this?
OpenAI models are not even SOTA, except that new-ish style transfer / illustration thing that made all us living in Ghibli world for a few days. R1 is _better_ than o1, and open-weights. GPT-4.5 is disappointing, except for a few narrow areas where it excels. DeepResearch is impressive though, but the moat is in tight web search / Google Scholar search integration, not weights. So far, I'd bet on open models or maybe Anthropic, as Claude 3.7 is the current SOTA for most tasks. by atemerev 2 minutes ago
This is worse than the mansplaining scene from Annie Hall.
by Q6T46nT668w6i3m 34 minutes ago
That is some awesome webdesign.
by qwertox 25 minutes ago
There's a lot to potentially unpack here, but idk, the idea that humanity entering hell (extermination) or heaven (brain uploading; aging cure) is whether or not we listen to AI safety researchers for a few months makes me question whether it's really worth unpacking.
by zvitiate 36 minutes ago
If we don't do it, someone else will.
by amelius 35 minutes ago
Which? Exterminate humanity or cure aging?
by itishappy 18 minutes ago
Yes
by ethersteeds 9 minutes ago
[flagged]
by throwawaylolllm 24 minutes ago
Feels reasonable in the first few paragraphs, then quickly starts reading like science fiction.
Would love to read a perspective examining "what is the slowest reasonable pace of development we could expect." This feels to me like the fastest (unreasonable) trajectory we could expect. by ikerino 1 hour ago
No one knows what will happen. But these thought experiments can be useful as a critical thinking practice.
by admiralrohan 16 minutes ago
Ok, I'll bite. I predict that everything in this article is horse manure. AGI will not happen. LLMs will be tools, that can automate away stuff, like today and they will get slightly, or quite a bit better at it. That will be all.
See you in two years, I'm excited what will be the truth.
by ahofmann 55 minutes ago
People want to live their lives free of finance and centralized personal information.
If you think most people like this stuff you're living in a bubble. I use it every day but the vast majority of people have no interest in using these nightmares of philip k dick imagined by silicon dreamers. by kristopolous 17 minutes ago
That seems naive in a status quo bias way to me. Why and where do you expect AI progress to stop? It sounds like somewhere very close to where we are at in your eyes. Why do you think there won't be many further improvements?
by Tenoke 46 minutes ago
GPT-3.5 is 2.5 years old now. And while Sonnet 3.7 et al. are very impressive, it is still not much more useful than GPT-3.5.
I write bog-standard PHP software. When GPT-3.5 came out, I was very frightened that my job could be automated away soon, because for PHP/Laravel/MySQL there must exist a lot of training data.
The reality now is, that the current LLMs still often create stuff, that costs me more time to fix, than to do it myself. So I still write a lot of code myself. It is very impressive, that I can think about stopping writing code myself. But my job as a software developer is, right now, very, very secure.
LLMs are very unable to build maintainable software. They are unable to understand what humans want and what the codebase need. The stuff they build is good-looking garbage. One example I've seen yesterday: one dev committed code, where the LLM created 50 lines of React code, complete with all those useless comments and for good measure a setTimeout() for something that should be one HTML DIV with two tailwind classes. They can't write idiomatic code, because they write code, that they were prompted for.
Almost daily I get code, commit messages, and even issue discussions that are clearly AI-generated. And it costs me time to deal with good-looking but useless content.
To be honest, I hope that LLMs get better soon. Because right now, we are in an annoying phase, where software developers bog me down with AI-generated stuff. It just looks good but doesn't help writing usable software, that can be deployed in production.
To get to this point, LLMs need to get maybe a hundred times faster, maybe a thousand or ten thousand times. They need a much bigger context window. Then they can have an inner dialogue, where they really "understand" how some feature should be built in a given codebase. That would be very useful. But it will also use so much energy that I doubt that it will be cheaper to let a LLM do those "thinking" parts over, and over again instead of paying a human to build the software. Perhaps this will be feasible in five or eight years. But not two.
And this won't be AGI. This will still be a very, very fast stochastic parrot. by ahofmann 8 minutes ago
ahofmann didn't expect AI progress to stop. They expected it to continue, but not lead to AGI, that will not lead to superintelligence, that will not lead to a self-accelerating process of improvement.
So the question is, do you think the current road leads to AGI? How far down the road is it? As far as I can see, there is not a "status quo bias" answer to those questions. by AnimalMuppet 7 minutes ago
I predict AGI will be solved 5 years after full self driving which itself is 1 year out (same as it has been for the past 10 years).
by bayarearefugee 23 minutes ago
Well said!
by ahofmann 7 minutes ago
What's an example of an intellectual task that you don't think AI will be capable of by 2027?
by mitthrowaway2 37 minutes ago
It won't be able to write a compelling novel, or build a software system solving a real-world problem, or operate heavy machinery, create a sprite sheet or 3d models, design a building or teach.
Long term planning and execution and operating in the physical world is not within reach. Slight variations of known problems should be possible (as long as the size of the solution is small enough). by kubb 12 minutes ago
Being accountable for telling the truth
by jdauriemma 11 minutes ago
programming
by coolThingsFirst 35 minutes ago
Can you phrase this in a concrete way, so that in 2027 we can all agree whether it's true or false, rather than circling a "no true scotsman" argument?
by mitthrowaway2 12 minutes ago
Why would it get 60-80% as good as human programmers (which is what the current state of things feels like to me, as a programmer, using these tools for hours every day), but stop there?
by lumenwrites 30 minutes ago
So I think there's an assumption you've made here, that the models are currently "60-80% as good as human programmers".
If you look at code being generated by non-programmers (where you would expect to see these results!), you don't see output that is 60-80% of the output of domain experts (programmers) steering the models.
I think we're extremely imprecise when we communicate in natural language, and this is part of the discrepancy between belief systems.
Will an LLM model read a person's mind about what they want to build better than they can communicate?
That's already what recommender systems (like the TikTok algorithm) do.
But will LLMs be able to orchestrate and fill in the blanks of imprecision in our requests on their own, or will they need human steering?
I think that's where there's a gap in (basically) belief systems of the future.
If we truly get post human-level intelligence everywhere, there is no amount of "preparing" or "working with" the LLMs ahead of time that will save you from being rendered economically useless.
This is mostly a question about how long the moat of human judgement lasts. I think there's an opportunity to work together to make things better than before, using these LLMs as tools that work _with_ us. by burningion 19 minutes ago
It's 60-80% as good as Stack Overflow copy-pasting programmers, sure, but those programmers were already providing questionable value.
It's nowhere near as good as someone actually building and maintaining systems. It's barely able to vomit out an MVP and it's almost never capable of making a meaningful change to that MVP.
If your experiences have been different that's fine, but in my day job I am spending more and more time just fixing crappy LLM code produced and merged by STAFF engineers. I really don't see that changing any time soon. by kody 24 minutes ago
I'm pretty good at what I do, at least according to myself and the people I work with, and I'm comparing its capabilities (the latest version of Claude used as an agent inside Cursor) to myself. It can't fully do things on its own and makes mistakes, but it can do a lot.
But suppose you're right, it's 60% as good as "stackoverflow copy-pasting programmers". Isn't that a pretty insanely impressive milestone to just dismiss?
And why would it just get to this point, and then stop? Like, we can all see AIs continuously beating the benchmarks, and the progress feels very fast in terms of experience of using it as a user.
I'd need to hear a pretty compelling argument to believe that it'll suddenly stop, something more compelling than "well, it's not very good yet, therefore it won't be any better", or "Sam Altman is lying to us because incentives".
Sure, it can slow down somewhat because of the exponentially increasing compute costs, but that's assuming no more algorithmic progress, no more compute progress, and no more increases in the capital that flows into this field (I find that hard to believe). by lumenwrites 13 minutes ago
Because ewe still haven't figured out fusion but its been promised for decades. Why would everything thats been promised by people with highly vested interests pan out any different?
One is inherently a more challenging physics problem. by boringg 25 minutes ago
Try this, launch Cursor.
Type: print all prime numbers which are divisible by 3 up to 1M
The result is that it will do a sieve. There's no need for this, it's just 3. by coolThingsFirst 15 minutes ago
by 23 minutes ago
When is the earliest that you would have predicted where we are today?
by jstummbillig 46 minutes ago
2015: We will have FSD(full autonomy) by 2017
by noncoml 13 minutes ago
This is absurd, like taking any trend and drawing a straight line to interpolate the future.
If I would do this with my tech stock portfolio, we would probably cross the zero line somewhere late 2025...
If this article were a AI model, it would be catastrophically overfit. by WhatsName 48 minutes ago
It's worse. It's not drawing a straight line, it's drawing one that curves up, on a log graph.
by AnimalMuppet 4 minutes ago
"we demand to be taken seriously!"
by the_cat_kittles 47 minutes ago
by 1 hour ago
by 1 hour ago
AI now even got it's own fan fiction porn. It is so stupid not sure whether it is worse if it is written by AI or by a human.
by Lionga 47 minutes ago
by 33 minutes ago