But can we still rely on Turings thoughts today? Is it still a blueprint for achieving artificial intelligence in machines? Does it mean that we just need to implement some deep learning into a robot eh voila... AI will emerge? If there is a lesson that we can learn from Turings analysis of artificial intelligence I believe it is the fact that he concludes that AI is not some ordinary function that we can simply implement in a machine. Instead he lists a set of explicit requirements:
Learning
Turing clearly concludes that it is an iterative process of improvement, yet learning is involved. This is not to say that we just need to implement Deep-Learning methods and we are done. Learning will definitely be data driven, as the system needs to sample the environment and will have to learn from this data. However, in an integrated AI-approach it must make more use of this information. Learned results must be stored and organized in a way that lets them be reused in later events – remember the process is iterative, so there is potentially the full life span of the system available for learning – and the learned results must be integrated to create Meta-Knowledge that will allow the systems require much less samples from the environment to come to conclusions in later stages of the process. In the best-case future learning will not be bound to look at millions of data points (which is an extremely expensive process) to learn the statistical correlations and hidden dynamics for each new case, instead a combination with more classical reasoning approaches will result in single shot (or with only a few samples) learning.
Physical interaction
He draws the conclusion that it needs to be a process of improvement from interaction with the real world. A simulation apparently is not enough. He specifically points out the need for interaction and refers to the very physical part of it. Not information exchange or some passive form of interplay of the system and the world rather he explicitly speaks of the physical interaction manifested in the ability to move around and to manipulate the environment.
From these explicit requirements we can or must derive some implicit requirements:
Complex body
Therefor a physical body is mandatory and this physical body needs to be of a minimum of structural complexity. As a robot will hardly be able to navigate a real-world environment or manipulate objects in the environment if it does not have legs or arms/hands to accomplish this.
Island skills
And finally, this requirement also asks for the ability to be able to master some of the ‘island talents’ that were already discussed. E.g. such a system must be able to perceive the environment and extract features of the environment with high precision and speed, to name just two of those island talents that computers are extremely good at today.
Reasoning and planning
Moreover, it must be able to make sense of these features and objects in the sense that it must have a model of the environment and the objects within it that relates the features and objects to each other in order to be able to reason and plan.
Control
And finally, it needs to possess of a control regime that allows it to move and manipulate in a meaningful and goal directed way in order to use the movements and manipulations for learning.
One can argue if or if not, this is a recipe to achieve AI in machines and one can also argue if the physical part (the body as well as the real world) is really so important. My personal opinion here is an absolute yes! Physics is important simply because the fact that it withdraws itself from perfect modelling and surprises us with effects for which a priori solutions cannot be pre-compiled instead it requires to develop mechanisms and concepts to handle these not to be modelled effects in an efficient way. Once again to handle a problem is not the same as to solve it and I believe that here lies the clue for future research on AI and robotics and why integrative AI is an important next step and is in contrast to the contemporary application-oriented AI or Functional-AI. The ‘Turing Option’ will open up a new dimension to these machines. The physical world...
Functional-AI: the era of island talents
Finding the right path in an era of increasing resources
Today we can still relax and watch with amusement the helpless steps of some of the robotic systems at the DARPA challenge (https://www.darpa.mil/program/darpa-robotics-challenge) to coordinate their own two feet just to walk up a stairway and we can smile at the helpless looking efforts trying to use a key to open a door... However, we should not be laughing too loud.
What are the requirements for modern robots that would increase their performance to a level where they would actually be of any usefulness to humanity; reliability; resilience; traceability of actions; fault tolerance; learning from a few examples instead of millions of data points; collaborating with people in a team and proactively solving problems themselves are just a few and they seem far away.
However, today, we can already produce structurally (kinematically) very complex robots (e.g. humanoids), which can be produced by lightweight construction methods, new and intelligent materials and especially by generative production techniques. These robots have a strong disposition for interaction with the real world (which is the prerequisite not only for learning motoric skills), which in many areas comes very close to human abilities, and can be operated effectively at the same time, i.e. they are efficient in terms of their energy requirements and the ratio of size/mass and the number of active/passive degrees of freedom.
The importance of a physical body
But one should ask the question of what are the fundamental conceptual aspects of why a body is needed. Why is embodiment so important and how does this relate to the concept of integrated AI. The idea of embodiment is actually around for a long time, see [6] as a landmark paper featuring this concept. The original idea of the concept of embodiment was to provide a new approach to robot control that diverged significantly from the so far dominant Sense-Plan-Act (SPA) concept. In order to get around the classical SPA flow it was mandatory to consider the structural and morphological features of the system in question. One has to admit that this approach yielded some impressive results given the limited computing power and software (control) concepts involved. E.g. Wall-following suddenly became a piece of a few dozen lines of software running on an 8-Bit Micro-Controller. The reason this was possible was that instead of the classical SPA approach no modeling at all was involved and hence no sophisticated algorithms were needed. However, generality of the approach of course was lost as it was a piece of software that would implement Wall-Following on this one particular machineFootnote 8 and no other. I know because I spent a good time of my career building such systems thinking these would conquer new worlds... [7]. Instead of complicated mathematical models, e.g. mathematical representations of the environment and the robot, this approach used the given morphology of the robot as the model of the environment itself. This was done with respect to nature that was cited as an architect that designed systems (in this case biological ones) according to the needs of a given environmental niche and yet the system itself (given all its kinematic structures and possibilities) was the best available model of the environment.
It became obvious very quickly that strictly following the embodied approach would not push us beyond the border and yet hybrid architectures have come up that tried to combine the best of both worlds, fast none model based reactive layers with rather slow but model based higher level planning layers (see [8] for a summary) and in fact today most robots doing useful things in real world environments would employ a hybrid architecture in one form or the other.
What we can learn from some 30+ years of research on embodiment in Robotics and AI is twofold: On the one hand we understood that exploiting the features of the physical structure of the system we are trying to control makes a lot of sense and helps to achieve more robust performance on the other hand without the higher-level planning and reasoning layers these systems do not cross the threshold of significance for any useful application.
However, I think that we have not exploited the idea of embodiment deeply enoughFootnote 9 before it became unpopular, or to put it in different words, before other developments became more promising and therefor more popular. This is to say that the increase in computing power was very fast over the last 30+ years actually so fast that you just had to wait a little while until a very complex algorithm would became possible to be executed on a computer chip on your robot. As a consequence of this development it simply did not make a lot of sense to dig deeper into embodiment and to come up with systems that would employ what I would call ‘kinematic intelligence’, referring to features built into the mechanical structure of the system that enable, facilitate or just simplify certain ‘intelligent’ function (a good example are passive walkers [9]). Instead the more power full computer chips allowed to use very powerful algorithms that accounted for the very low kinematic intelligence of the systems by ‘modelling the pitfalls of the hardware away’, in other words very complicated control laws could be used that were able to deal with low intelligent hardware concepts instead of putting more effort into the design of the systems hardware or body (and using the extra algorithmic power for other things...). I was again among those who took the bait when at the end of the 1990’s a colleague and I were trying to make a robot autonomously navigate in sewage pipes.Footnote 10 It turned out to be a real challenge to design a system that could just physically travel down a concrete pipe [10]. We had many concepts in mind that would be able to deal with the challenging environment, however in the end we decided to screw a laptop and a few sensors to a modified radio-controlled toy truck and instead used the power of the laptop to implement a neural network that learned [11] to classify the structure of the pipes and types of junctions that the system encountered. Because it was extremely difficult to physically back track to last known positions in the pipe network – it would have required to actually use the much more sophisticated system designs that we had already thrown overboard – we developed a navigation system based on a distribution of so-called belief states an approach that was later developed by others [12] to be called probabilistic navigation. I cannot say I regret the path I followed at that time but I sometimes ask myself what if we would have used the much more complex designs we had figured out and used the extra computing power that we got for free to use it for higher aspects of cognition? Already at that time we dreamed of systems that would just stay down in the pipe systems forever (their live time) and continuously learn about the dynamics and changes in the environment and the system itself to become what we would call today a long-term autonomous system or a live long learning system or as Nils Nilsson termed it in the 1970’s a ‘never to turn off’ system.
My corollary on the importance of a body for integrative AI would be the following:
Complexity of the systems has to grow beyond a certain threshold in order for Integrative AI approaches to be reasonable or in other words for the ‘Turing Option’ to become available. Once the complexity of our systems does cross this threshold we will be able to observe these developments:
-
a)
Methods of integrative AI will be developed on a conceptual and framework level
-
b)
As a result, the level of intelligence in these systems will grow fast and
-
c)
complexity will come down again.
In fact, with the increasing complexity of kinematic chains (e.g. in manipulators or the legs and arms of humanoid robots) a solution using classical differential equations is no longer efficient or even becomes impossible when it comes to parallel kinematics or closed kinematic chains [13]. Only recently (deep) learning methods have been used to derive efficient models for control [14] and it seems to be a very reasonable assumption that these methods will be the tool of choice to cope with the dynamics of complex kinematic systems interacting with unpredictable environments,Footnote 11 especially if model-based approaches are combined with data driven learning methods, yet the need for integration is already visible even if today only in partial areas and not so much yet on a system-environment long term interaction level.
As a side note we should recognize that improvements in natural language processing nearly stayed a flat line in the chart of historical developments up until Neural Network based approaches and especially Deep-Learning methods entered the scene, when the performance curve sky rocketed [15].
Therefore, the day we will be able to see a humanoid robot that integrates several AI-Technologies to run thru the forest, open a door with a key, or stitching a wound of a soldier while talking to him or her in a calm decent voice using the right words to psychologically calm the person down, while in the background it is planning the fastest path to the hospital given the current weather forecast and available transportation options, is most likely not too far away.
Towards integrative-AI
The greatest needs for research are effective approaches to the organization of the different processes that must be used to effectively operate e.g. robots as described above (http://www.willowgarage.com/blog/2012/04/16/open-source-robotics-foundation). If one looks at the list of required characteristics of these systems - in particular to be able to cooperate with humans in a team - a system is indeed described or required that can be described as AI-complete, in the sense that it actually requires and has to integrate all aspects of AI and that cannot be reduced to a simpler (less complex) solution. The methods range from the use of machine learning methods to control highly complex kinematics, the use of deep neural networks in sensor-based perception, the planning of complex action sequences, the reasoning from facts (those given to the system and those generated by the system itself) and finally the recognition of intentions of the human partner and the action of the robot adapted to a complex context.
Learning from millions of data points cannot be the right way to learn. A retired colleague of mine from the University of Bremen [16], who studied the frogs brain for decades, keeps nagging me by saying: “How is it possible that my frog can solve the problem to catch a fly with a brain of 7 grams of weight requiring a few Watt of power and your robot needs to look at millions of flies just to learn what a fly looks like – left alone to mange to catch it – and requires kilowatts of power...’”.
Apart from being embarrassed I am trying to tell him that we have missed out to study how to organize and structure the things that we have once learned. Instead we focused a lot in the past decades to the process of learning itself and we apparently made some very good progress but we made less progress on studies on how to structure, organize and eventually network the things we have learned. Biological systems must have found ways to learn things much quicker and with less effort from what we are currently doing. There are many ways of learning and one aspect of learning is what could be called learning over generations. This concept refers to the fact that by generations of evolution some of the things that have been learned by earlier generations of learners gets built into the hardware of the next generation of learners. This occurs as a co-development process in biological systems: on the structural (mechanical) level the frog evolved a longer tongue but at the same time also evolved a brain region (algorithmic level) to control the tongue. It can also be observed on the neuro functional level where e.g. the part of the brain that was developed to control the tongue was linked to the input from the visual part of the brain of the frog to form a more complex ensemble that solves the fly catching problem in coming generations even without any thinking. So, what was once a very costly process for many generations of frogs has been preserved and transformed into a system of lesser complexity.Footnote 12 One could say that the investment (to spent so much effort to learn fly catching) finally paid off for the species. We have not come to this level of Design principals in AI-Research or in AI-enabled-Robotics yet. But I think this is where we should be heading for and I think this is what the Turing Option meant at its core.
While in the last decades we have made considerable progress in the area of the different sub-disciplines of AI, the Turing Option (robotics) forces us to study the integration of these sub-disciplines into one system. On the background of the story of the frog it is important to note that this cannot and should not be considered a ‘trivial’ (software) engineering problem. Instead it is a problem that challenges us to act more economically on our resources and to find ways to melt down things that have once been learned with great effort into simpler, less complex structural elements of our systems.Footnote 13
Here is also a reason why the body is so indispensable for generating multi-purpose-AI systems. The physical body quasi serves as a long-time storage medium for things that have once been acquired (learned) by the system on a purely algorithmic level. While the algorithmic level is where we are very flexible and fast we can evolve new concepts but when it comes for these concepts to be efficient tools they need to be implemented in a less computation demanding way. Somehow, we are re-approaching the original idea of embodiment by seeing the body as the best model of the environment. But while the last time we stopped at building single examples or proof of concept that in fact the body can be a good environmental model we should this time go for a deeper approach and study ways how we can systematically take advantage of this concept by building systems that improve over generations and with every generation they outsource some of the costly acquired knowledge into the structural or functional design of the next generation of systems. Of course, this requires first that we have a notion of generations of systems (robots) we should in fact develop ‘Generation Thinking’ when it comes to AI-system design. Interestingly enough we do have notions of generations of smart phones or cars, but we do not have a notion of generations of AI systems at least not in a systematic way.
The challenge and scientific question are how to efficiently integrate the different complex levels; from control to reasoning, planning, and interaction. The term efficient here does not mean deterministic but refers to the above-mentioned ability to handle such complex systems. The important difference is that we need very complex machines (robots) to study or create artificial intelligent systems, which can only develop intelligence step by step from the learning interaction with a natural environment, and which, however, due to their structural complexity and the inherent complexity of the natural environment, force us to use non-deterministic methods to control these complexities.
Thus, we are required to develop organizational principles or integration structures that make these systems immanent non-determinism manageable to the extent that the resulting systems remain efficient machines, i.e. that they accomplish their tasks in reasonable time and with reasonable resources.
It should be pointed out that integrative AI is not Strong-AI as one may speculate. In fact, it is not even something that will take us beyond of the set P of problems efficiently solvable in polynomial time.
I would like to support my argument by a simple set theoretic argumentation. If we construct a set of problems that collects all the AI algorithms, then we can describe the set of Problems solvable by AI-algorithms combined by the functional composition of at least one element of this set. Functional AI could then be described as the set of problems solvable by one specific AI algorithm or a single element of the above collection of all AI-Algorithms, e.g. NLP or Human Face recognition. We can safely conclude that: FunctionalAI ⊆ P.
In contrast to this class of problems we can quantify Integrative AI as the set of problems that requires to apply at least two AI-algorithms. So, we can describe the set of problems that are solved thru methods of Integrative AI as the cross product of the set of all AI-Algorithms. However, IntegrativeAI ⊆ P still holds and therefore we must assume that we will have to be able to solve at least one problem outside of P to achieve Strong AI. Because we must assume that StrongAI ⊂ NP, we can conclude that IntegrativeAI ≠ StrongAI.
This argument implies that integrative AI can be defined as the set of combinations of one or more AI algorithms, note that this definition does not say anything about how these algorithms are to be functionally composed. However, this definition also leads to the result that integrative AI is not something that will solve problems beyond P and it is no way to achieve strong AI or AI superiority.