Elon Musk filed a lawsuit in San Franciscoā€™s Superior Court accusing OpenAI and its CEO, Sam Altman, of betraying the startupā€™s initial commitment to openness, the betterment of society, and lack of profit as a motive. Among other things, Muskā€™s 35-page complaint argues that OpenAI has violated its original deal to share its GPT large language models with Microsoft, which stated that the software giant would lose access to new LLMs once OpenAI had achieved AGI. According to the complaint, OpenAI reached that epoch-shifting moment a year ago with GPT-4, its most powerful model to date.

Muskā€”who cofounded OpenAI but left in 2018ā€”is at least as entitled as anyone to come up with his own definition of AGI. His complaint describes it as ā€œa general purpose artificial intelligence systemā€”a machine having intelligence for a wide variety of tasks like a human.ā€ That does sound like GPT-4 as I, a mere layperson, experience it in ChatGPT Plus.

But Muskā€™s declaration that the AGI era is already upon us is hardly the consensus among AI scientists. Even those who think itā€™s not far off predict arrival dates that are least a few years away. And GPT-4 falls well short of meeting OpenAIā€™s own explanation of the term: ā€œA highly autonomous system that outperforms humans at most economically valuable work.ā€

Consider the evidence:

GPT-4 isnā€™t remotely autonomous; indeed, it does its best work when humans provide plenty of hand-holding in the form of detailed prompts. The world is still in the process of figuring out what tasks GPT-4 can do, and we frequently overrate its competence. Thatā€™s not even getting into the fact that OpenAIā€™s reference to ā€œmost economically valuable workā€ suggests that true AGI may involve not just software but also sophisticated robotics that donā€™t exist yet. To guess when OpenAIā€”or a rival such as Google, Anthropic, Meta, Mistral, or Perplexityā€”might reach AGI, as OpenAI defines it, is to expect that itā€™ll be an obvious moment in time. But OpenAIā€™s definition, like all the others, is squishy and difficult to put to a conclusive test. To riff on Supreme Court Justice Potter Stewartā€™s famous comment about pornography, maybe weā€™ll know it when we see it. At the moment, however, Iā€™m convinced that obsessing over AGIā€™s existence or nonexistence is counterproductive.

The whole notion of AGI is predicated on the assumption that AI started out dumber than a human but could someday match or exceed our level of thinking. Already, though, generative AI is different than human intelligenceā€”far closer to omniscient than any individual flesh-and-blood thinker, yet also preternaturally gullible and prone to blurring fact and fiction in ways that donā€™t map to common human frailties. Thatā€™s because itā€™s a predictive engine, trained to string together words without truly understanding them. If its present trajectory of simulated brilliance mixed with boneheadedness continues, it might wander off in a direction far afield from most definitions of AGI.

Even if the world lands on a new, more inclusive definition of AGI, it may be hard to prove whether a particular LLM has attained it. Muskā€™s lawsuit cites proof points of GPT-4ā€™s reasoning power, such as its scoring in the 90th percentile on the Uniform Bar Exam for lawyers and the 99th percentile on the GRE Verbal Assessment. That it can do so is astounding. But acing tests is not synonymous with performing useful work. And even if it were, who gets to decide how many tests an LLM must pass before itā€™s achieved AGI rather than just bobbled somewhere in its vicinity?

For decades, the Turing Testā€”which a computer would pass by fooling a human into thinking that it, too, was humanā€”was computer scienceā€™s beloved thought experiment for determining when AI had gotten real. Strangely enough, itā€™s useless as a tool for assessing todayā€™s LLM-based chatbots. But not because they know too little to fake humanity convincingly, or canā€™t express it glibly enoughā€”but because they betray their artificiality by being so good at churning out endless wordage on more topics than any human knows. AGI could end up in a similar predicament: a benchmark, devised by humans, thatā€™s rendered obsolete by the technology it was meant to measure.

DID YOU HEAR THE ONE ABOUT THE ā€œMAC CAR?ā€ Last week, Appleā€™s long, expensive quest to build an autonomous EV entered its rearview-mirror phaseā€”a sad fate my colleague Jared Newman blamed on the companyā€™s sometimes counterproductive pursuit of perfection. Wondering what an Apple car would be like has been an obsession for techies since 2012, when news broke that Steve Jobs had toyed with getting into the automobile business even before there was an iPhone. Or maybe it started in 2008, when reports of a meeting between Steve Jobs and Volkswagenā€™s CEO led to wild speculation about an ā€œiCar.ā€

Or how about 1998? According to Snopes, thatā€™s when a joke involving cars designed by software companies began spreading like crabgrass across the internet, eventually evolving into an urban legend involving a Bill Gates keynote and a General Motors press release. Along with a Microsoft car that crashed twice a day and occasionally needed its engine replaced for no apparent reason, it mentioned a ā€œMac carā€ that ā€œwas powered by the sun, was reliable, five times as fast, twice as easy to driveā€”but would only run on 5% of the roads.ā€

  • General_Effort@lemmy.world
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    1
    Ā·
    8 months ago

    The idea is that a ā€œrealā€ AI should be able to detect implied inconsistencies in the training data and point them out?

    • GiveMemes@jlai.lu
      link
      fedilink
      English
      arrow-up
      1
      Ā·
      8 months ago

      Indeed. It should be able to reason. Like a human. Not a hard idea to grasp tbh.

        • GiveMemes@jlai.lu
          link
          fedilink
          English
          arrow-up
          1
          Ā·
          edit-2
          8 months ago

          https://www.azquotes.com/picture-quotes/quote-i-shall-not-today-attempt-further-to-define-the-kinds-of-material-but-i-know-it-when-potter-stewart-143-23-28.jpg

          I guess I should elaborate a bit. This is from a famous SC court case concerning ā€˜obscenityā€™ itā€™s almost impossible to provide any kind of definition concerning reason or thinking because itā€™s on the very edge of what we can ever really ā€˜knowā€™. At the same time I know that if we train something on both the questions and the answers and make it really efficient at giving the right answers, itā€™s obviously not thinking, just indexing information. A great example is how AI canā€™t create new information without a seed of absolute randomness. Humans donā€™t have a random bone in their body.

          A fun (though outdated) video series about the edge of the knowable:

          https://youtube.com/playlist?list=PL3096540179B12F8D&si=SC-F1_T9WFSn-5XN

          • General_Effort@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            Ā·
            8 months ago

            You know, Alan Turing describes almost the same problem in 1950, though he talks about defining ā€œthinkingā€. He was famously good at reasoning and proposed a solution.

            • GiveMemes@jlai.lu
              link
              fedilink
              English
              arrow-up
              1
              Ā·
              edit-2
              8 months ago

              Thatā€™s kinda the whole point of my comment is that things like Turingā€™s method completely fall apart under heavy scrutiny. Further, the Turing Test specifically tells you nothing about whether or not something IS thinking, just that it MAY be. Big difference.

              I see you didnā€™t engage with the rest of my comment tho. Would you like to?

              Just wanted to add this as it and stuff like it comes up pretty quickly when you research the turing test:

              "On the other hand, there are several criticisms and limitations of the Turing Test as a measure of machine intelligence. Some of the main issues include:

              The test focuses solely on the ability to mimic human-like behavior and communication, rather than on the underlying intelligence or consciousness of the machine.

              The test is heavily dependent on the human evaluatorā€™s subjective judgment, and may be influenced by factors such as the machineā€™s appearance or the humanā€™s own biases.

              The test does not take into account the possibility that a machine could be intelligent in ways that are fundamentally different from human intelligence.

              The test does not consider the possibility of a machine deceiving the human evaluator, by providing pre-programmed or rehearsed responses rather than truly understanding the meaning of the questions."

              LLMs would fall into the last, as they train on the ā€œanswersā€ so to speak and just match them to the ā€œquestionā€.

              • General_Effort@lemmy.world
                link
                fedilink
                English
                arrow-up
                1
                Ā·
                8 months ago

                I see you didnā€™t engage with the rest of my comment tho. Would you like to?

                I am not sure if I should. The topic is veering into the spiritual. To me, this is merely a matter of intellectual curiosity. But for many people it is a very emotional subject. I do not wish to cause emotional distress.

                • GiveMemes@jlai.lu
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  arrow-down
                  1
                  Ā·
                  edit-2
                  8 months ago

                  We both know this has nothing to do with ā€˜emotional distressā€™ and everything to do with your overly large ego being bruised by the fact that youā€™re wrong. Itā€™s classic fallacious behavior to argue as you have and then not engage with the opposition. The only ā€œemotionally distressedā€ one here is you, and itā€™s honestly really sad considering itā€™s an anonymous forum and nobody even knows that itā€™s you being stupid behind the screen. :/

                  • General_Effort@lemmy.world
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    Ā·
                    8 months ago

                    Huh. An emotional subject, indeed. I didnā€™t think merely pointing it out would be enough to trigger you. Sorry for causing you distress. Iā€™m just not good at picking up emotional cues.