Intel Loihi Robotics Lit Review

A summary of ]different published robotics applications of Intel's neuromorphic control board, Loihi

All of these sources were found in the review paper: M. Davies et al., "Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook," in Proceedings of the IEEE, vol. 109, no. 5, pp. 911-934, May 2021

G. Tang, N. Kumar and K. P. Michmizos, "Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware", arXiv:2003.01157, Mar. 2020

• Combined a spiking neural network (SNN) with a deep neural network (DNN) and trained them in conjunction on mapless navigation

• The spiking actor network (SAN), a type of SNN, was deployed on the Loihi

• Loihi communicates with the on-board computer of the TurtleBot2, which runs ROS to actually process controlling and sensing

Figure 3 from Tang et al. 2020

T. DeWolf, P. Jaworski and C. Eliasmith, "Nengo and low-power AI hardware for robust embedded neurorobotics", arXiv:2007.10227, Jul. 2020

Rover Problem

• Built their controller without using neural networks (using traditional tools), then converted it into an SNN to put it onto the Loihi

  • Visual input processor developed as a DNN converted to an SNN

  • Control signals created by a NEF (neural engineering framework). User must first find conventional circuit that solves their problem, then convert to NEF

• Once vision and control sub-networks converted, connected together on Loihi

• Use Python interfaces to communicate with the board (NengoInterfaces)

Arm Control Problem

• Augments an existing PD controller to assist a robot arm (Kinova Jaco) in reaching while holding an unexpected weight

• Loihi adds context sensitive I term to controller to account for varying parameters. This is done seperately to regular PD controller (operational space controller; OSC)

Fig 2.1 of DeWolf et al. 2020

• PD OSC runs in Python, then uses NengoInterfaces to send joint data to Loihi (or other places; can run on regular CPU or GPU)

• NEF manages disseminating sensory signals to neurons

I. Polykretis, G. Tang and K. P. Michmizos, "An astrocyte-modulated neuromorphic central pattern generator for hexapod robot locomotion on Intel’s Loihi", Proc. Int. Conf. Neuromorphic Syst., pp. 1-9, Jul. 2020

• Controlled a fully simulated hexapod with a CPG network on the Loihi

• Created a new bursting neuron compartment model for Loihi, then used bursting activity as a pacemaker for CPG

Fig 1 of Polykretis et al. 2020

Fig 2 of Polykretis et al. 2020

• Actual communication to the simulation is done through ROS

  • Have a "faster than real time" communication speed between ROS and Loihi, since everything is simulated

P. Balachandar and K. P. Michmizos, "A spiking neural network emulating the structure of the oculomotor system requires no learning to control a biomimetic robotic head", arXiv:2002.07534, 2020

• Developed an oculomotor controller and put it into a robot head

  • Head made with Dynamixel servos

• Controller was developed based on knowledge of existing biological structure of eye

• Included some training component just to see if the system would improve with it; network development strategy did not require any training to implement

• Did not discuss how the Loihi controller interfaces with the servos, but included C++ code in the appendix (which I couldn't immediately find)

Fig 1 of Balachandar et al 2020