Connect with us

Artificial Intelligence

Reinforcement learning AI has the potential to introduce humanoid robots into the real world

blank

Published

on

blank

AI tools like ChatGPT are revolutionizing our digital experiences, but the next frontier is bringing AI interactions into the physical world. Humanoid robots, trained with a specific AI, have the potential to be incredibly useful in various settings such as factories, space stations, and nursing homes. Two recent papers in Science Robotics emphasize the potential of reinforcement learning to bring robots like these into existence.

According to Ilija Radosavovic, a computer scientist at the University of California, Berkeley, there has been remarkable advancement in AI within the digital realm, thanks to tools like GPT. However, I believe that AI in the physical world holds immense potential for transformation.

The cutting-edge software that governs the movements of bipedal bots frequently employs a technique known as model-based predictive control. It has resulted in the development of highly advanced systems, like the Atlas robot from Boston Dynamics, known for its impressive parkour abilities. However, programming these robot brains requires a considerable amount of human expertise, and they struggle to handle unfamiliar situations. Using reinforcement learning, AI can learn through trial and error to perform sequences of actions, which may prove to be a more effective approach.

According to Tuomas Haarnoja, a computer scientist at Google DeepMind and coauthor of one of the Science Robotics papers, the team aimed to test the limits of reinforcement learning in real robots. Haarnoja and his team decided to create software for a toy robot named OP3, manufactured by Robotis. The team had the goal of teaching OP3 to walk and play one-on-one soccer.

“Soccer provides a favorable setting for exploring general reinforcement learning,” states Guy Lever of Google DeepMind, who coauthored the paper. It demands careful planning, adaptability, curiosity, collaboration, and a drive to succeed.

Operating and repairing larger robots can be quite challenging, but the smaller size of these robots allowed us to iterate quickly,” Haarnoja explains. Similar to a network architect, the researchers first trained the machine learning software on virtual robots before deploying it on real robots. This technique, called sim-to-real transfer, helps ensure that the software is well-prepared for the challenges it may face in the real world, such as the possibility of robots falling over and breaking.

The training of the virtual bots occurred in two stages. During the initial phase, the team focused on training one AI to successfully lift the virtual robot off the ground, while another AI was trained to score goals without losing balance. The AIs were provided with data that included the robot’s joint positions and movements, as well as the positions of other objects in the game captured by external cameras. In a recently published preprint, the team developed a version of the system that utilizes the robot’s visual capabilities. The AIs were required to generate fresh joint positions. If they excelled, their internal parameters were adjusted to promote further replication of the successful actions. During the second stage, the researchers developed an AI that could replicate the behavior of the first two AIs and evaluate its performance against opponents that were similar in skill level (versions of itself).

Similar to a network architect, the researchers adjusted different elements of the simulation, such as friction, sensor delays, and body-mass distribution, in order to fine-tune the control software, known as a controller, for the real-world robots. In addition to scoring goals, the AI was also recognized for its ability to minimize knee torque and prevent injuries.

Robots that were tested with the RL control software demonstrated impressive improvements in their performance. They walked at a significantly faster pace, turned with remarkable agility, and were able to recover from falls in a fraction of the time compared to robots using the scripted controller provided by the manufacturer. However, more sophisticated abilities also surfaced, such as seamlessly connecting actions. “It was fascinating to witness the robots acquiring more advanced motor skills,” comments Radosavovic, who was not involved in the study. And the controller acquired knowledge not only of individual moves, but also the strategic thinking needed to excel in the game, such as positioning oneself to block an opponent’s shot.

According to Joonho Lee, a roboticist at ETH Zurich, the soccer paper is truly impressive. “We have witnessed an unprecedented level of resilience from humanoids.”

But what about humanoid robots that are the size of humans? In another recent paper, Radosavovic collaborated with colleagues to develop a controller for a larger humanoid robot. This particular robot, Digit from Agility Robotics, is approximately five feet tall and possesses knees that bend in a manner reminiscent of an ostrich. The team’s approach resembled that of Google DeepMind. Both teams utilized computer brains known as neural networks. However, Radosavovic employed a specialized variant known as a transformer, which is commonly found in large language models such as those that power ChatGPT.

Instead of processing words and generating more words, the model analyzed 16 observation-action pairs. These pairs represented what the robot had sensed and done in the past 16 snapshots of time, which spanned approximately a third of a second. The model then determined the robot’s next action based on this information. Learning was made easier by initially focusing on observing the actual joint positions and velocity. This provided a solid foundation before progressing to the more challenging task of incorporating observations with added noise, which better reflected real-world conditions. For enhanced sim-to-real transfer, the researchers introduced slight variations to the virtual robot’s body and developed a range of virtual terrains, such as slopes, trip-inducing cables, and bubble wrap.

With extensive training in the digital realm, the controller successfully operated a real robot for an entire week of rigorous tests outdoors, ensuring that the robot maintained its balance without a single instance of falling over. In the lab, the robot successfully withstood external forces, even when an inflatable exercise ball was thrown at it. The controller surpassed the manufacturer’s non-machine-learning controller, effortlessly navigating a series of planks on the ground. While the default controller struggled to climb a step, the RL controller successfully overcame the obstacle, despite not encountering steps during its training.

Reinforcement learning has gained significant popularity in recent years, particularly in the field of four-legged locomotion. Interestingly, these studies have also demonstrated the successful application of these techniques to two-legged robots. According to Pulkit Agrawal, a computer scientist at MIT, these papers have reached a tipping point by either matching or surpassing manually defined controllers. With the immense potential of data, a multitude of capabilities can be unlocked within a remarkably brief timeframe.

It is highly probable that the approaches of the papers are complementary. In order to meet the demands of the future, AI robots will require the same level of resilience as Berkeley’s system and the same level of agility as Google DeepMind’s. In real-world soccer, both aspects are incorporated. Soccer has posed a significant challenge for the field of robotics and artificial intelligence for a considerable period, as noted by Lever.

 

As Editor here at GeekReply, I'm a big fan of all things Geeky. Most of my contributions to the site are technology related, but I'm also a big fan of video games. My genres of choice include RPGs, MMOs, Grand Strategy, and Simulation. If I'm not chasing after the latest gear on my MMO of choice, I'm here at GeekReply reporting on the latest in Geek culture.

Artificial Intelligence

A group of humanoid robots from Agility will take care of your spanx

blank

Published

on

blank

So far, the humanoid robotics business has only been full of promises and test runs. These programs only use a few robots and don’t usually lead to anything more important, but they are important for the eventual use of new technology. While a pilot with logistics giant GXO went well, Agility announced on Thursday that it has now signed a formal deal.

Moving plastic totes around a Spanx factory in Georgia will be Digit’s first job, and that’s not a lie. The number of two-legged robots that will be taking boxes off of cobots and putting them on conveyor belts has not been made public, so it is likely that it is still too low. When it comes to tens or hundreds of thousands, most people would be happy to share that information.

They are leased as part of a model called “robots-as-a-service” instead of being bought outright. This way, the client can put off paying the huge upfront costs of such a complicated system while still getting support and software updates.

Last year, GXO started to test drive Digit robots. A pilot deal was just announced between the logistics company and Apptronik, one of Agility’s biggest rivals. I’m not sure how one will change the other.

When Peggy Johnson became CEO of Agility in March, she made it clear that the company was focused on ROI. This is a big change in a field where results are still mostly theoretical.

Johnson said, “There will be many firsts in the humanoid robot market in the years to come, but I’m very proud of the fact that Agility is the first company to have real humanoid robots deployed at a customer site, making money and solving real-world business problems.” “Agility has always been focused on the only metric that matters: giving our customers value by putting Digit to work. This milestone deployment sets a new standard for the whole industry.”

It’s not a surprise that Agility, based in Oregon, was the first to reach another important milestone. The company has been ahead of the rest of the market in terms of development and deployment. Of course, the industry is still very new, and there isn’t a clear market leader yet.

Amazon started testing Agility systems in its own warehouses in October of last year, but neither company has said what will happen next.

Continue Reading

Artificial Intelligence

Zuckerberg says that competitors with closed-source AI are trying to “make God”

blank

Published

on

blank

In an interview that came out Thursday, Mark Zuckerberg, CEO of Meta, talked about his hopes for the future of AI. He said that he strongly believes there will not be “just one AI.” While talking about how open source can help many people get AI tools, Zuckerberg took a moment to criticize the work of competitors who he didn’t name because he thinks they aren’t being open. He said that these competitors seem to think they are “creating God.”

In a new YouTube interview with Kane Sutter (@Kallaway), Zuckerberg said, “I don’t think that AI technology should be kind of hoarded and… that one company gets to use it to build whatever central, single product that they’re building.”

“It really turns me off when tech people talk about making this ‘one true AI,'” he said. He said, “It’s almost like they think they’re making God or something, but that’s not what we’re doing.” “That’s not how I see this going.”

“I see why, if you’re in an AI lab.” You want to think that what you’re doing is really important, right? It sounds like, “We’re making the one real thing for the future.” But, you know, in real life, that’s not how things work, right?” Zuckerberg talked about it. “It’s not like everyone has just one app on their phone that they use.” Not everyone wants all of their content to come from the same person. People don’t want to buy everything from just one store.

During the talk, Zuckerberg said that many different AIs should be made to capture people’s wide range of interests. On Thursday, the company also announced early tests of its AI Studio software in the U.S. This software will let creators and other people make AI avatars that can message people on Instagram. The AIs will be able to chat with people and answer questions from their followers in a fun way. To avoid confusion, they will be marked as “AI.”

As an example, the CEO of Meta said he didn’t think companies that build closed AI platforms were making the best experiences for people.

He went on, “You want to unlock and…unlock as many people as possible to try new things.” “Well, that’s what culture is, right?” Nobody is letting one group of people tell everyone what to do.

His comments sound a bit like he’s upset because they came out soon after news that Meta had tried to talk to Apple about putting its AIs into Apple’s operating systems instead of just working with OpenAI at launch but was turned down. Bloomberg says that Apple decided not to have formal talks with Meta because it didn’t think Meta’s privacy policies were strong enough.

Without a deal, Meta will not be able to reach the billions of iPhone users that there could be in the world. It looks like Meta’s plan B is to make technology that can be used for more than just smartphones.

During the interview, Zuckerberg talked about the progress the company is making with the Ray-Ban Meta smart glasses. He said that one day, this progress would meet up with the work that is already being done on full holographic displays. But he said the first one will be more popular in the short term.

He said, “I actually think you can have a great experience with cameras, a microphone, speakers, and the ability to do multimodal AI.” This was before the glasses had any kind of display. It also costs less because it doesn’t have a screen. The Meta Quest Pro costs $1,000, while Meta’s smart glasses cost around $300.

Before convergence, Zuckerberg said there would be three different kinds of products: smart glasses without screens, displays that show information on the top of the head, and full holographic displays. He said that one day, people might not have neural interfaces connected to their brains but instead wear a wristband that picks up signals from the brain and lets their hand talk to it. This would let them talk to the neural interface with their hand, which is barely moving. In time, it might also let people type.

Zuckerberg did warn that these kinds of inputs and AI experiences might not be able to replace smartphones right away. “I don’t think that in the history of technology, the new platform has ever made people stop using the old one completely.” “You just don’t use it as much,” he said.

People do things on their phones now that they might have done on their computers 10 to 15 years ago.

He said, “I think that will also happen with glasses.” “We’re not going to give up our phones.” You’ll just keep it in your pocket and only pull it out when you need to use it. But I think more and more people will just say, “Hey, I can take this picture with my glasses on.” The CEO said, “I can ask AI this question or send someone a message; it’s just a lot easier with glasses.”

The speaker said, “I wouldn’t be surprised if, in 10 years, we still have phones, but we’ll probably use them in a much more deliberate way instead of just grabbing them for any technological task we want to do.”

Continue Reading

Artificial Intelligence

What a new study says suggests that ChatGPT may have passed the Turing test

blank

Published

on

blank

René Descartes, a French philosopher who may or may not have been high on pot, had an interesting thought in 1637: can a machine think? Alan Turing, an English mathematician and computer scientist, gave the answer to this 300-year-old question in 1950: “Who cares?” He said a better question was what would become known as the “Turing test”: if there was a person, a machine, and a human interrogator, could the machine ever trick the human interrogator into thinking it was the person?

Turing changed the question in this way 74 years ago. Now, researchers at the University of California, San Diego, think they have the answer. A new study that had people talk to either different AI systems or another person for five minutes suggests that the answer might be “yes.”

“After a five-minute conversation, participants in our experiment were no better than random at identifying GPT-4. According to the preprint paper, which has not yet undergone peer review, this suggests that current AI systems can deceive people into believing they are human. “These results probably set a lower bound on how likely it is that someone will lie in more naturalistic settings, where people may not be aware of the possibility of lying or only focus on finding it.”

Even though this is a big event that makes headlines, it’s not a milestone that everyone agrees on. The researchers say that Turing first thought of the imitation game as a way to test intelligence, but “many objections have been raised to this idea.” People, for example, are known for being able to humanize almost anything. We want to connect with things, whether they’re people, dogs, or a Roomba with googly eyes on top of it.

Also, it’s interesting that ChatGPT-4 and ChatGPT-3.5, which was also tested, only persuaded humans that it was a person about half of the time, which isn’t much better than random chance. What does this result really mean?

As it turns out, ELIZA was one of the AI systems that the team built into the experiment as a backup plan. She was made at MIT in the mid-1960s and was one of the first programs of her kind. She was impressive for her time, but she doesn’t have much to do with modern large-language model-based systems or LLM-based systems.

“ELIZA could only give pre-written answers, which greatly limited what it could do. Live Science talked to Nell Watson, an AI researcher at the Institute of Electrical and Electronics Engineers (IEEE), about how it might fool someone for five minutes but soon show its flaws. “Language models are completely adaptable; they can put together answers to a lot of different topics, speak in specific languages or sociolects, and show who they are by displaying personality and values that are based on their characters.” a significant improvement over something that a person, no matter how intelligent and careful they were, programmed by hand.

She was perfect for the experiment because she was the same as everyone else. How do you explain test subjects who are lazy and pick between “human” and “machine” at random? If ELIZA gets the same score as chance, then the test is probably not being taken seriously because she’s not that good. In what way can you tell how much of the effect is just people giving things human traits? How much did ELIZA get them to change their minds? That much is probably how much it is.

In fact, ELIZA got only 22%, which is just over 1 in 5 people believing she was human. It’s more likely that ChatGPT has passed the Turing test now that test subjects could reliably tell the difference between some computers and people, but not ChatGPT, the researchers write.

So, does this mean we’re entering a new era of AI that acts like humans? Are computers smarter than people now? Maybe, but we probably shouldn’t make our decisions too quickly.

The researchers say, “In the end, it seems unlikely that the Turing test provides either necessary or sufficient evidence for intelligence. At best, it provides probabilistic support.” The people who took part weren’t even looking for what you might call “intelligence”; the paper says they “were more focused on linguistic style and socio-emotional factors than more traditional notions of intelligence such as knowledge and reasoning.” This “could reflect interrogators’ latent assumption that social intelligence has become the human trait that is most difficult for machines to copy.”

Which brings up a scary question: is the fall of humans the bigger problem than the rise of machines?

“Real humans were actually more successful, convincing interrogators that they were human two-thirds of the time,” the paper’s co-author, Cameron Jones, told Tech Xplore. “Our results suggest that in the real world, people might not be able to reliably tell if they’re talking to a human or an AI system.”

“In the real world, people might not be as aware that they’re talking to an AI system, so the rate of lying might be even higher,” he warned. “This makes me wonder what AI systems will be used for in the future, whether they are used to do bots, do customer service jobs, or spread fake news or fraud.”

There is a draft of the study on arXiv, but it has not yet been reviewed by other scientists.

Continue Reading

Trending