Artificial Intelligence
Reinforcement learning AI has the potential to introduce humanoid robots into the real world
AI tools like ChatGPT are revolutionizing our digital experiences, but the next frontier is bringing AI interactions into the physical world. Humanoid robots, trained with a specific AI, have the potential to be incredibly useful in various settings such as factories, space stations, and nursing homes. Two recent papers in Science Robotics emphasize the potential of reinforcement learning to bring robots like these into existence.
According to Ilija Radosavovic, a computer scientist at the University of California, Berkeley, there has been remarkable advancement in AI within the digital realm, thanks to tools like GPT. However, I believe that AI in the physical world holds immense potential for transformation.
The cutting-edge software that governs the movements of bipedal bots frequently employs a technique known as model-based predictive control. It has resulted in the development of highly advanced systems, like the Atlas robot from Boston Dynamics, known for its impressive parkour abilities. However, programming these robot brains requires a considerable amount of human expertise, and they struggle to handle unfamiliar situations. Using reinforcement learning, AI can learn through trial and error to perform sequences of actions, which may prove to be a more effective approach.
According to Tuomas Haarnoja, a computer scientist at Google DeepMind and coauthor of one of the Science Robotics papers, the team aimed to test the limits of reinforcement learning in real robots. Haarnoja and his team decided to create software for a toy robot named OP3, manufactured by Robotis. The team had the goal of teaching OP3 to walk and play one-on-one soccer.
“Soccer provides a favorable setting for exploring general reinforcement learning,” states Guy Lever of Google DeepMind, who coauthored the paper. It demands careful planning, adaptability, curiosity, collaboration, and a drive to succeed.
Operating and repairing larger robots can be quite challenging, but the smaller size of these robots allowed us to iterate quickly,” Haarnoja explains. Similar to a network architect, the researchers first trained the machine learning software on virtual robots before deploying it on real robots. This technique, called sim-to-real transfer, helps ensure that the software is well-prepared for the challenges it may face in the real world, such as the possibility of robots falling over and breaking.
The training of the virtual bots occurred in two stages. During the initial phase, the team focused on training one AI to successfully lift the virtual robot off the ground, while another AI was trained to score goals without losing balance. The AIs were provided with data that included the robot’s joint positions and movements, as well as the positions of other objects in the game captured by external cameras. In a recently published preprint, the team developed a version of the system that utilizes the robot’s visual capabilities. The AIs were required to generate fresh joint positions. If they excelled, their internal parameters were adjusted to promote further replication of the successful actions. During the second stage, the researchers developed an AI that could replicate the behavior of the first two AIs and evaluate its performance against opponents that were similar in skill level (versions of itself).
Similar to a network architect, the researchers adjusted different elements of the simulation, such as friction, sensor delays, and body-mass distribution, in order to fine-tune the control software, known as a controller, for the real-world robots. In addition to scoring goals, the AI was also recognized for its ability to minimize knee torque and prevent injuries.
Robots that were tested with the RL control software demonstrated impressive improvements in their performance. They walked at a significantly faster pace, turned with remarkable agility, and were able to recover from falls in a fraction of the time compared to robots using the scripted controller provided by the manufacturer. However, more sophisticated abilities also surfaced, such as seamlessly connecting actions. “It was fascinating to witness the robots acquiring more advanced motor skills,” comments Radosavovic, who was not involved in the study. And the controller acquired knowledge not only of individual moves, but also the strategic thinking needed to excel in the game, such as positioning oneself to block an opponent’s shot.
According to Joonho Lee, a roboticist at ETH Zurich, the soccer paper is truly impressive. “We have witnessed an unprecedented level of resilience from humanoids.”
But what about humanoid robots that are the size of humans? In another recent paper, Radosavovic collaborated with colleagues to develop a controller for a larger humanoid robot. This particular robot, Digit from Agility Robotics, is approximately five feet tall and possesses knees that bend in a manner reminiscent of an ostrich. The team’s approach resembled that of Google DeepMind. Both teams utilized computer brains known as neural networks. However, Radosavovic employed a specialized variant known as a transformer, which is commonly found in large language models such as those that power ChatGPT.
Instead of processing words and generating more words, the model analyzed 16 observation-action pairs. These pairs represented what the robot had sensed and done in the past 16 snapshots of time, which spanned approximately a third of a second. The model then determined the robot’s next action based on this information. Learning was made easier by initially focusing on observing the actual joint positions and velocity. This provided a solid foundation before progressing to the more challenging task of incorporating observations with added noise, which better reflected real-world conditions. For enhanced sim-to-real transfer, the researchers introduced slight variations to the virtual robot’s body and developed a range of virtual terrains, such as slopes, trip-inducing cables, and bubble wrap.
With extensive training in the digital realm, the controller successfully operated a real robot for an entire week of rigorous tests outdoors, ensuring that the robot maintained its balance without a single instance of falling over. In the lab, the robot successfully withstood external forces, even when an inflatable exercise ball was thrown at it. The controller surpassed the manufacturer’s non-machine-learning controller, effortlessly navigating a series of planks on the ground. While the default controller struggled to climb a step, the RL controller successfully overcame the obstacle, despite not encountering steps during its training.
Reinforcement learning has gained significant popularity in recent years, particularly in the field of four-legged locomotion. Interestingly, these studies have also demonstrated the successful application of these techniques to two-legged robots. According to Pulkit Agrawal, a computer scientist at MIT, these papers have reached a tipping point by either matching or surpassing manually defined controllers. With the immense potential of data, a multitude of capabilities can be unlocked within a remarkably brief timeframe.
It is highly probable that the approaches of the papers are complementary. In order to meet the demands of the future, AI robots will require the same level of resilience as Berkeley’s system and the same level of agility as Google DeepMind’s. In real-world soccer, both aspects are incorporated. Soccer has posed a significant challenge for the field of robotics and artificial intelligence for a considerable period, as noted by Lever.
René Descartes, a French philosopher who may or may not have been high on pot, had an interesting thought in 1637: can a machine think? Alan Turing, an English mathematician and computer scientist, gave the answer to this 300-year-old question in 1950: “Who cares?” He said a better question was what would become known as the “Turing test”: if there was a person, a machine, and a human interrogator, could the machine ever trick the human interrogator into thinking it was the person?
Turing changed the question in this way 74 years ago. Now, researchers at the University of California, San Diego, think they have the answer. A new study that had people talk to either different AI systems or another person for five minutes suggests that the answer might be “yes.”
“After a five-minute conversation, participants in our experiment were no better than random at identifying GPT-4. According to the preprint paper, which has not yet undergone peer review, this suggests that current AI systems can deceive people into believing they are human. “These results probably set a lower bound on how likely it is that someone will lie in more naturalistic settings, where people may not be aware of the possibility of lying or only focus on finding it.”
Even though this is a big event that makes headlines, it’s not a milestone that everyone agrees on. The researchers say that Turing first thought of the imitation game as a way to test intelligence, but “many objections have been raised to this idea.” People, for example, are known for being able to humanize almost anything. We want to connect with things, whether they’re people, dogs, or a Roomba with googly eyes on top of it.
Also, it’s interesting that ChatGPT-4 and ChatGPT-3.5, which was also tested, only persuaded humans that it was a person about half of the time, which isn’t much better than random chance. What does this result really mean?
As it turns out, ELIZA was one of the AI systems that the team built into the experiment as a backup plan. She was made at MIT in the mid-1960s and was one of the first programs of her kind. She was impressive for her time, but she doesn’t have much to do with modern large-language model-based systems or LLM-based systems.
“ELIZA could only give pre-written answers, which greatly limited what it could do. Live Science talked to Nell Watson, an AI researcher at the Institute of Electrical and Electronics Engineers (IEEE), about how it might fool someone for five minutes but soon show its flaws. “Language models are completely adaptable; they can put together answers to a lot of different topics, speak in specific languages or sociolects, and show who they are by displaying personality and values that are based on their characters.” a significant improvement over something that a person, no matter how intelligent and careful they were, programmed by hand.
She was perfect for the experiment because she was the same as everyone else. How do you explain test subjects who are lazy and pick between “human” and “machine” at random? If ELIZA gets the same score as chance, then the test is probably not being taken seriously because she’s not that good. In what way can you tell how much of the effect is just people giving things human traits? How much did ELIZA get them to change their minds? That much is probably how much it is.
In fact, ELIZA got only 22%, which is just over 1 in 5 people believing she was human. It’s more likely that ChatGPT has passed the Turing test now that test subjects could reliably tell the difference between some computers and people, but not ChatGPT, the researchers write.
So, does this mean we’re entering a new era of AI that acts like humans? Are computers smarter than people now? Maybe, but we probably shouldn’t make our decisions too quickly.
The researchers say, “In the end, it seems unlikely that the Turing test provides either necessary or sufficient evidence for intelligence. At best, it provides probabilistic support.” The people who took part weren’t even looking for what you might call “intelligence”; the paper says they “were more focused on linguistic style and socio-emotional factors than more traditional notions of intelligence such as knowledge and reasoning.” This “could reflect interrogators’ latent assumption that social intelligence has become the human trait that is most difficult for machines to copy.”
Which brings up a scary question: is the fall of humans the bigger problem than the rise of machines?
“Real humans were actually more successful, convincing interrogators that they were human two-thirds of the time,” the paper’s co-author, Cameron Jones, told Tech Xplore. “Our results suggest that in the real world, people might not be able to reliably tell if they’re talking to a human or an AI system.”
“In the real world, people might not be as aware that they’re talking to an AI system, so the rate of lying might be even higher,” he warned. “This makes me wonder what AI systems will be used for in the future, whether they are used to do bots, do customer service jobs, or spread fake news or fraud.”
There is a draft of the study on arXiv, but it has not yet been reviewed by other scientists.
Meta finally put out its long-awaited API for Threads today, so developers can start making games and apps that use it. Third-party developers will be able to create new experiences around
Mark Zuckerberg also posted about the launch of the API, saying, “The Threads API is now widely available and will be coming to more of you soon.”
Engineer for Threads Jesse Chen wrote in a blog post that developers can now use the new API to publish posts, get their own content, and set up reply management tools. In other words, developers can let users hide or show replies or reply to certain ones.
It will also have analytics that let developers see things like the number of views, likes, replies, reposts, and quotes at the media and account level, the company said.
Adam Mosseri, the CEO of Instagram, first talked about the company’s work on the Threads API in October 2023. The API was first released in a closed beta with partners like Techmeme, Sprinklr, Sprout Social, Social News Desk, Hootsuite, and a few other developers. Chen said at that time that Meta planned to let many developers use the API in June. As promised, the company kept its word.
Along with the launch of the new API, the company also put out an open-source reference app on GitHub so developers can play with it.
In 2023, it was hard for third-party developers who made tools for social networks because social networks like Twitter (now X) and Reddit limited or shut down API access at different levels. This is because decentralized social networks like Mastodon and Bluesky are more open to developers. With more than 150 million users, Meta’s Threads is the most popular new social network. Since Threads now works with the fediverse and has an API, third-party developers can make some great social media experiences.
Artificial Intelligence
Apple has officially announced its intention to collaborate with Google’s Gemini platform in the future
After delivering a keynote presentation at WWDC 2024, which unveiled Apple Intelligence and announced a collaboration with OpenAI to integrate ChatGPT into Siri, Senior Vice President Craig Federighi confirmed the intention to collaborate with more third-party models. The initial instance provided by the executive was one of the companies that Apple was considering for a potential partnership.
“In the future, we are excited about the prospect of integrating with other models, such as Google Gemini,” Federighi expressed during a post-keynote discussion. He promptly stated that the company currently has no announcements to make, but that is the overall direction they are heading in.
OpenAI’s ChatGPT is set to become the first external model to be integrated at a later date this year. Apple announces that users will have the ability to access the system without the requirement of creating an account or paying for premium services. Regarding the integration of that platform with the updated iOS 18 version of Siri, Federighi confirmed that the voice assistant will notify users before utilizing its own internal models.
“Now you can accomplish this task directly using Siri, without the need for any additional tools,” stated the Apple executive. “Siri, it is crucial to ascertain whether you will inquire before proceeding to ChatGPT.” Subsequently, you can engage in a dialogue with ChatGPT. Subsequently, if there is any pertinent data mentioned in your inquiry that you wish to provide to ChatGPT, we will inquire, ‘Would you like to transmit this photograph?’ From a privacy standpoint, you always maintain control and have complete visibility.
- Gadgets9 years ago
Why the Nexus 7 is still a good tablet in 2015
- Mobile Devices9 years ago
Samsung Galaxy Note 4 vs Galaxy Note 5: is there room for improvement?
- Editorials9 years ago
Samsung Galaxy Note 4 – How bad updates prevent people from enjoying their phones
- Mobile Devices9 years ago
Nexus 5 2015 and Android M born to be together
- Gaming9 years ago
New Teaser For Five Nights At Freddy’s 4
- Mobile Devices9 years ago
Google not releasing Android M to Nexus 7
- Gadgets9 years ago
Moto G Android 5.0.2 Lollipop still has a memory leak bug
- Mobile Devices9 years ago
Nexus 7 2015: Huawei and Google changing the game