Building an Ecosystem Designed to Build a Human-Level AI
The biggest leap forward in history may be within reach. The last mile will be a team effort.
Around 2009, when I was at D-Wave, I met and pitched investment in the company to Peter Thiel. At some point he asked me a question along the lines of “tell me something that’s true that very few people agree with you on.” Apparently he asks this of everyone.
I remember this because the answer I gave had nothing to do with D-Wave or quantum computation. My answer was that human-level AI will be built by 2030. I’d just spent an hour describing the most advanced quantum computation project on Earth. You’d think I would be able to answer with something related to that… but no. I’d become obsessed with what I had started thinking about as The Big Problem — how to understand our own minds well enough to build one.
Over the 14 years since then my colleagues and I have been obsessively grinding away at The Big Problem, first at D-Wave, then at Kindred, and now at Sanctuary. Every year we have methodically been taking two steps forward for every one step back. While as I age I get less sure of basically everything, I still (somewhat hesitantly) think my exuberance in 2009 about when we solve The Big Problem was justified. I still think we will get there by 2030, even after working on the front lines of this problem for more than a decade and understanding in a visceral way how hard it is. Although I don’t know if today I could give Thiel the same answer as I did back then. A lot more people agree with me now than back in 2009.
It Takes a Village
I was born and spent my first few years in Tanzania with the Swahili-speaking Masai. My first spoken word was apparently dawa, which means medicine. Perhaps related, I also almost got eaten by army ants at least once. Good times.
I remember some Swahili but not much. One of the phrases I remember is Asiye funzwa na mamaye hufunzwa na ulimwengu, which roughly means “it takes a village to raise a child”. I don’t know if this is actually true for people. But it sure is true for technology. Behind the computers, networks, and phones we take for granted are thousands of people, all contributing to the creation and maintenance of these things. We have a tendency to idolize individuals, like Steve Jobs and Elon Musk, and there is no doubt those two are unique. But the reality is that it really does take a village to create advanced technologies.
When I was at Kindred, one of my investors was Pierre Lamond. Pierre is legendary in the venture capital community. In 1967 he became CEO of National Semiconductor, and then a partner at Sequoia from 1981 through 2009. He would tell fascinating stories of the early days of Sequoia’s investment strategies in Apple and Atari. Back then most of the parts of the systems Apple and Atari were selling had few suppliers. Things like memory, motherboards, processors, keyboards, and screens were all new and both companies struggled to consistently find good parts and suppliers for their products. So what Sequoia did is fund the entire home computing ecosystem. They backed dozens of companies that were building the unsexy parts that Apple and Atari needed to thrive.
This strategy was amazingly effective. A flywheel began to turn where Apple and Atari got top notch parts, making their products great, which meant more people bought them, which caused capital to flow into the entire Sequoia ecosystem. This changed how computers got adopted into our homes. Make no mistake, this could have gone very differently. If Sequoia had not pumped capital into all those companies you probably couldn’t name, Apple and Atari — and by extension the entire computing industry — would have had unreliable and expensive products very few people would have put up with.
Pierre’s stories about the early days of the computing industry resonated with me. We faced the same kinds of challenges at D-Wave. The parts required to build a quantum computer either didn’t exist, or they were not good enough. I had spent a good part of my life there investing in things like dilution refrigerators and superconducting chip fabrication. Looking back I wish I’d met Pierre ten years earlier. As an aside, Pierre was chairman of Cypress Semiconductor for many years, the place where we built D-Wave’s fabrication capability — but we invested there years after he’d already left.
When we started Sanctuary back in 2018, the Sequoia strategy was very much on my mind. We had set ourselves the mission to be the first organization in the world to create human-level AI. We committed to doing so by thinking of that AI as the control system for a humanoid robot. This vision of what we wanted to build entangled two different grand challenges. Not only did we need to build a new kind of robot that could move with the grace and fluidity of a person, but we also needed to build a software control system for it that had most of the properties of the human mind.
So how to proceed? As soon as the ink was dry on the incorporation papers, we started trying to figure out what the ecosystem for building a human-level intelligence in a humanoid robot should look like. What was our ‘Sequoia strategy’? What parts does something like this need to have? Who are the organizations that we should bet on to provide them? What should we commit to building ourselves? When things got confusing, we tried to build analogies between what we wanted to build and the very early days of the computing industry.
The Start of an Ecosystem for Building Human-Level AI
Complex technologies are built by coalitions of organizations working together. When new categories arise, such as home computers, quantum computers, or human-level AI, the ecosystem around them needs to be actively built by strategic investment. People who see something new and important coming have to capitalize and support organizations that could build currently non-economic parts of the new thing. Market forces don’t work in this case because the product doesn’t exist yet, and therefore demand for its parts doesn’t exist yet, and therefore the suppliers of those parts don’t have an economic incentive to build them.
We’ve drawn on our experiences and the lessons learned from the early days of the semiconductor industry to start building an ecosystem to support delivering against Sanctuary’s mission. Here we’d like to introduce some of this ecosystem. Each of these companies are working on critical parts of one or both of the two grand challenges we must overcome to reach mission success.
To start, some context on our approach to building human-level AI. We have committed to a set of principles that include thinking of building human-level AI as being the same as building a control system for a humanoid robot. In the 1980s and 90s, brilliant work was done on what robotic control systems of this type would need to look like. When we started seriously thinking about taking on this challenge, one of the first things we did was go back and read what people like John McCarthy, Nils Nillson, Marvin Minsky, Rodney Brooks, and Alan Newell were thinking needed doing. It’s unfortunate that the newer generation of AI researchers is largely unaware of this work. Like neural nets themselves, ideas can be good ideas but just too early.
One of the key features that makes controlling robots in the real world distinct from many other uses of AI is that, at least if you believe the above cast of characters, there is a requirement for these control systems to perceive, act on, and think about the world symbolically. That is, whatever it is that their minds are doing, they are doing it with symbols. The human mind uses a specific set of these symbols, and those are the ones robots with human-like minds should use.
Because of this one of our first choices for an ecosystem partner was Cycorp. Cycorp is the longest running AI organization in the world. Their Cyc technology traces its roots back to 1984. Cycorp’s leader, Doug Lenat, is a legend in the field of artificial intelligence. He has steadfastly championed symbolic reasoning and logic as a path to human-level AI. You can learn more about him and Cycorp on Lex Fridman’s podcast. The Cycorp team under Doug’s leadership have built what we believe is one of the most important parts of this puzzle — the part that uses logic and reasoning on symbols that a mind uses to understand the world.
Cycorp’s people have thoughtfully engineered tens of millions of logical rules intended to model what we think of as common sense. Cyc reasons using common sense both deeply and quickly, using over a thousand specialized reasoners which attack each problem as a team.
What is common sense, exactly? What we mean by this needs to be written down somewhere. This is excruciatingly difficult, and I think, outside of the scope of statistical approaches such as Large Language Models (LLMs). I don’t think scaling LLMs will produce common sense.
One of the things Doug said that stuck with me is that Cyc encodes all the things that never get written down that we all take for granted. We are betting that Cyc and LLMs working together will be much, much greater than the sum of their parts. And since Cyc operates by logical pro- and con- argumentation, unlike LLMs, it can fully explain every step in its reasoning. This is important for any AI to be trustworthy. This is very important when AI controls machines. Doing this right is very important to us.
Sanctuary and Cycorp have worked closely together for several years. We have deeply integrated Cyc into Sanctuary’s technology stack. Our business relationship includes an equity position and an exclusive license to Cyc for humanoids.
We have seen remarkable success and validation of our intuition on Cyc’s central role in delivering human-like AI. Here’s Doug on the partnership: “Cycorp’s mission is to deliver on the founding promise of the field of AI — the creation of technologies that model and understand how the world works, as deeply and broadly as people do, enabling them to responsibly, sanely and ethically solve problems like we do. I am thrilled to partner with Sanctuary to push towards mission success, and together build the most important technology in history.” Amen to that!
Apptronik is one of the few organizations in the world that could realistically produce robots that can move with the function, mobility, grace, and fluidity of the human body. When we were looking for suppliers of physical robots and their parts, we looked closely at all the usual suspects. Apptronik was superior to all of them in ambition, technological capability, and alignment with our mission. Our first joint project back in 2019 had this over the top description of what success would look like: “A robot like nobody has ever seen. Organic curves, delicate generative structure, clean and deliberate lines, delicate yet nimble motions. A living triumph of modern artwork and precision motion control.” You can see some of the progress they are making on building embodiments for us.
We have now worked together with Apptronik for more than three years, and have invested in their success and mission, with an equity position and a board seat. From Jeff Cardenas, CEO: “Apptronik is a world leader in building advanced bipedal walking robotics systems designed to work with and alongside people. Sanctuary is a key investor and supporter of our company from the early days. We are very excited to have the opportunity to work with their team and CEO, Geordie Rose, who has been providing us with his expertise as a member of our board of directors. Apptronik’s mission is to build the world’s best humanoid robots, and working together with Sanctuary, we have dramatically accelerated progress towards both our and their mission success.”
Robots require models of the world personal to their minds. These models are what they use to think about what to do, before they do it. While we use many different technologies to build world models that don’t require our support and investment, such as Blender, DeepMind’s Mujoco physics engine, and Epic’s Unreal Engine 5, some of our requirements do require investment. One of the organizations working on aspects of building world models in our ecosystem is Common Sense Machines, or CSM. CSM is building state of the art systems for producing 3D assets for world models. We have invested in CSM over others in the space as we are very aligned with us on what is required to build human-level AI and how to get there.
Max Kleiman-Weiner, co-founder and CEO: “Learning generative world models is a critical component in successfully creating artificial general intelligence. It’s comparable to how a child learns about the world through experience. We give AI that ability. The way our technology is being used by Sanctuary is aligned with the original and greatest goal of the field of artificial intelligence — building machines that think like we do. We are very excited about being part of what Sanctuary is building.”
One of the key elements of our approach is learning from demonstration, where the demonstration examples are provided by a human in the loop control paradigm called analogous teleoperation. This control style outfits a person in a thing we call a pilot rig, which transmits the sense data from the robot to the person, and converts that person’s actions into actions the robot performs.
Contoro is an Austin-based robotics company that manufactures high precision robotic systems originally designed for injury rehabilitation. We saw the possibility of using them instead as part of our pilot rigs. Youngmok Yun, CEO, Contoro: “We believe our technology will unlock the unlimited potential of teleoperation – the ability to remotely control robots from anywhere in the world. Contoro and Sanctuary AI have the same desire to solve labor problems by combining robotics and human-like intelligence.”
Another important part of the teleoperations system are haptic gloves that both transmit the sense of touch to the pilot and allow the pilot to, with great precision, manipulate the world. The hands on Sanctuary robots are the most advanced robotic hands ever built, in the sense of mimicking the human hand. We use HaptX gloves for this purpose, and have built a very healthy partnership with HaptX for this use case. Jake Rubin, founder, chairman and CEO, HaptX: “The gloves we’ve created at HaptX feature hundreds of microfluidic actuators, so when pilots interact with virtual objects, they feel completely real. Our partnership with Sanctuary will help push the boundaries of teleoperation capabilities for a true-to-life embodiment experience.”
Related, touch sensors on the hands of the Sanctuary robots ideally would transmit the same kind of information to the AI mind that our own fingers do to our own minds. We acquired Tangible Research to help us implement the best possible haptic technology into our systems. You may have seen Tangible technology already, as there is a widely shared video of Jeff Bezos piloting a system Tangible developed. Jeremy Fishel, founder and CTO: “We believe one of the reasons robotic dexterity has been so far behind human ability is because the sense of touch is quite difficult to implement. Integrating our tactile sensors into the phenomenal hand hardware Sanctuary has created, will help the robotic hand capabilities increase exponentially and become more human-like.”
Sanctuary robots can see, hear, feel, and to a certain extent, understand the world. But this is not enough. They also need to be able to use their hands just like we do.
Work is delivered by hands. We use them all the time for nearly everything we do. More than 98% of all work requires fine manipulation. This is a key reason that the labor market is so much bigger than the current robotics market. Robots without human-like hands simply can’t do much. In some real sense, from the perspective of doing real work, the rest of the robot is only there to get the hands to the work that needs doing.
The human hand is a marvel. Robot hands that mimic human hands are very hard both to build and to control. To help with this, we acquired the entire intellectual property portfolio of Giant AI. The team at Giant pioneered a set of techniques for using vision to help robots grasp and manipulate objects the way we do. The acquisition of these assets strengthened our already strong and rapidly growing intellectual property position. Being first mover in this new category gives us the strategic advantage of owning many foundational technologies that are necessary for anyone to make this type of technology work in commercial settings.
While building humanoid robots is more straightforward than building their minds, there are still areas that require long-term investment. One of these is in actuator technology. We have developed a relationship with Exonetik to investigate the use of a new approach to building actuators that might shine in the humanoid context. Pascal Larose, co-founder and CEO: “Exonetik is developing a novel type of actuator for Sanctuary’s use case that enables breakthroughs in safe, efficient, powerful, and graceful human-like movement. The use of our actuators in Sanctuary general-purpose robots will be huge for both organizations, and for all Sanctuary’s customers.”
At the intersection of the mind and the body lies the need for powerful, reliable networks. A fleet of robots will have many aspects of their minds in other places, running on computer networks that could be thousands of miles away. To ensure we have the best possible solutions to networking issues, Bell and Verizon have partnered with us.
John Watson, Group President, Business Markets, Customer Experience and AI, and Sanctuary board member: “Bell Ventures was founded to drive innovation and support Canadian entrepreneurs. Investing in Sanctuary AI clearly aligns with those objectives. We envision a future where Sanctuary technology can be used to help address labour shortages. The Bell 5G network will not only power the Sanctuary AI labor as a service model across Canada, but will also provide edge computing capabilities and security to other Canadian organizations utilizing their technology.”
Michelle McCarthy, Managing Director, Verizon Ventures: “At Verizon Ventures, we strive to be at the intersection of connectivity and technology. Our mission is to support startups that have developed the most cutting-edge solutions with the power to shape the future leveraging advanced network capabilities such as Verizon’s 5G Ultra Wideband network and Mobile Edge Compute. Seeing the Sanctuary AI technology deployed in a commercial setting was a major milestone not only for the company but for this emerging industry.”
Being able to build a small number of concept robots is necessary as a first step toward mission success. But ultimate success requires building complex machines at scale. Humanoid robots are roughly the same complexity to build as cars. Magna is one of the few companies in the world with the experience of producing these kinds of systems at scale, and has invested in Sanctuary. Josh Berg, Managing Director, Magna Technology Investments: “As a leading mobility technology company, Magna has extensive experience in developing and manufacturing complex systems. Factory of the future concepts such as humanoid robots certainly falls into this category and we see great potential for this technology in automotive manufacturing. Our investment in Sanctuary was a strategic move to help facilitate advancements in this area. We are excited to explore the future of this technology together.”
As the Sanctuary fleet grows, one use case is to set the goals of these robots to do work. Nils John Nilsson, one of the AI pioneers that has profoundly influenced our approach, put it this way:
“I claim that achieving real human-level artificial intelligence would necessarily imply that most of the tasks that humans perform for pay could be automated… Machines exhibiting true human-level intelligence should be able to do many of the things humans are able to do. Among these activities are the tasks or “jobs” at which people are employed. I suggest we replace the Turing test by something I will call the “employment test.” To pass the employment test, AI programs must be able to perform the jobs ordinarily performed by humans. Progress toward human-level AI could then be measured by the fraction of these jobs that can be acceptably performed by machines.”
Workday Ventures, the strategic capital arm of the enterprise cloud applications provider Workday that uses AI and ML to elevate human capabilities to make finance and HR more intelligent, joined our investor group. Barbry McGann, managing director and senior vice president, Workday Ventures: “At Workday, we are shaping the new world of work, by helping customers adapt and thrive in a changing world. More than 50% of the Fortune 500 rely on Workday to intelligently manage their talent and their financial processes. We are very excited to work with Sanctuary to help address the labor and skills challenges facing many organizations today.”
Early use cases for humanoid robots that require general intelligence will be important to get right. One of the organizations partnering to investigate a diverse set of use cases is Canadian Tire Corporation (CTC). We recently announced our first deployment at a Mark’s retail store in Langley, BC, Canada. The week-long pilot at the CTC-owned store successfully tested the general-purpose robot in a ‘real-life’ store environment with 110 retail-related tasks completed correctly, including front and back-of-store activities. Cari Covent, VP, Data, Analytics and AI, Canadian Tire Corporation: “Working with Sanctuary AI has enabled Canadian Tire Corporation to further explore cutting-edge innovations and accelerate operational efficiency. With the Mark’s pilot, we were able to focus human resources on higher-value and more meaningful work, like customer service and engagement. By making strategic investments and working with partners like Sanctuary AI, we are furthering our customer understanding, customer experience, and operational efficiencies to drive our Better Connected strategy.”
The Road Ahead
Investing in and cultivating an ecosystem isn’t in itself enough to ensure delivering on Sanctuary’s mission. Understanding our own minds well enough to build one is one of the most difficult challenges people have ever taken on. But we think strategic investment in the right kind of ecosystem is necessary for success.
To date we haven’t talked much publicly about what we are doing, and how we are doing it. This isn’t because we wanted to keep anything secret. We just wanted to focus on building the thing.
Now that we have gotten the technology to a point where many of the key challenges are understood, we are going to spend more time talking about what we are doing and where technologies related to human-level AI really are. We will provide regular updates on progress towards our objective that will be accessible to everyone. This technology will affect every person on the planet. It’s important for people to understand what it is and isn’t, when it’s likely to arrive, and what that will look like.
To stay up to date with the latest developments, milestones, and accomplishments, you can subscribe to Sanctuary on Substack and follow us on Instagram, LinkedIn, Twitter, and YouTube.
Geordie Rose
Co-Founder and former CEO
Sanctuary AI
Sanctuary is on a mission to create the world’s first human-like intelligence in general-purpose robots that will help us work more safely, efficiently, and sustainably, helping to address the labor challenges facing many organizations today. Sanctuary's growing list of customers and investors represents a wide variety of industries across Canada, the U.S., and other countries around the world.
Join us and work with world leaders in the fields of artificial intelligence, cognition, and robotics—all working together toward a common goal. Visit our careers page to learn more.