> For the complete documentation index, see [llms.txt](https://wvu-neuromint.gitbook.io/neuromint-resources/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://wvu-neuromint.gitbook.io/neuromint-resources/literature/intel-loihi-robotics-lit-review.md).

# Intel Loihi Robotics Lit Review

A summary of ]different published robotics applications of Intel's neuromorphic control board, Loihi

{% hint style="info" %}
All of these sources were found in the review paper:\
[M. Davies *et al*., "Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook," in *Proceedings of the IEEE*, vol. 109, no. 5, pp. 911-934, May 2021](https://ieeexplore.ieee.org/document/9395703)
{% endhint %}

### [G. Tang, N. Kumar and K. P. Michmizos, "Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware", *arXiv:2003.01157*, Mar. 2020](https://arxiv.org/abs/2003.01157v2)

* Combined a spiking neural network (SNN) with a deep neural network (DNN) and trained them in conjunction on mapless navigation
* The spiking actor network (SAN), a type of SNN, was deployed on the Loihi
* Loihi communicates with the on-board computer of the [TurtleBot2](https://www.turtlebot.com/turtlebot2/), which runs ROS to actually process controlling and sensing

![Figure 3 from Tang et al. 2020](https://2585980925-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MbT0JvqdvF6kx3gpkbo%2F-MhyDHztnzkG5tXjgo2k%2F-MhyIJ0rrMbvDwbY3rSE%2FTang%20et%20al%202020%20Fig%203.PNG?alt=media\&token=14bde7cc-6b50-467a-b223-33ed6d111045)

### [T. DeWolf, P. Jaworski and C. Eliasmith, "Nengo and low-power AI hardware for robust embedded neurorobotics", *arXiv:2007.10227*, Jul. 2020](https://arxiv.org/abs/2007.10227v2)

#### Rover Problem

* Built their controller without using neural networks (using traditional tools), then converted it into an SNN to put it onto the Loihi
  * Visual input processor developed as a DNN converted to an SNN 
  * Control signals created by a NEF (neural engineering framework). User must first find conventional circuit that solves their problem, then convert to NEF
* Once vision and control sub-networks converted, connected together on Loihi
* Use Python interfaces to communicate with the board (NengoInterfaces)

#### Arm Control Problem

* Augments an existing PD controller to assist a robot arm (Kinova Jaco) in reaching while holding an unexpected weight
* Loihi adds context sensitive I term to controller to account for varying parameters. This is done seperately to regular PD controller (operational space controller; OSC)

![Fig 2.1 of DeWolf et al. 2020](https://2585980925-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MbT0JvqdvF6kx3gpkbo%2F-MhyDHztnzkG5tXjgo2k%2F-MhyN8RC1Om8nhqZig08%2FDeWolf%20et%20al%202020%20Fig%202.PNG?alt=media\&token=0339ebb5-440c-45aa-9452-dc2470ef1cdc)

* PD OSC runs in Python, then uses NengoInterfaces to send joint data to Loihi (or other places; can run on regular CPU or GPU)
* NEF manages disseminating sensory signals to neurons 

### [I. Polykretis, G. Tang and K. P. Michmizos, "An astrocyte-modulated neuromorphic central pattern generator for hexapod robot locomotion on Intel’s Loihi", *Proc. Int. Conf. Neuromorphic Syst.*, pp. 1-9, Jul. 2020](https://arxiv.org/abs/2006.04765)

* Controlled a **fully simulated** hexapod with a CPG network on the Loihi
* Created a new bursting neuron compartment model for Loihi, then used bursting activity as a pacemaker for CPG

![Fig 1 of Polykretis et al. 2020](https://2585980925-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MbT0JvqdvF6kx3gpkbo%2F-MhyDHztnzkG5tXjgo2k%2F-MhyPIvKg4Z8Gtbqz8d9%2FPolykretis%20et%20al%202020%20Fig%201.PNG?alt=media\&token=83c579af-1b6b-410d-95c6-73e5dde79441)

![Fig 2 of Polykretis et al. 2020](https://2585980925-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MbT0JvqdvF6kx3gpkbo%2F-MhyDHztnzkG5tXjgo2k%2F-MhyPjHoUx7i4YIPtID1%2FPolykretis%20et%20al%202020%20Fig%202.PNG?alt=media\&token=2f70a6b2-f1d4-467a-b7c8-99205b373558)

* Actual communication to the simulation is done through ROS
  * Have a "faster than real time" communication speed between ROS and Loihi, since everything is simulated

### [P. Balachandar and K. P. Michmizos, "A spiking neural network emulating the structure of the oculomotor system requires no learning to control a biomimetic robotic head", *arXiv:2002.07534*, 2020](https://arxiv.org/abs/2002.07534v2)

* Developed an oculomotor controller and put it into a robot head
  * Head made with Dynamixel servos
* Controller was developed based on knowledge of existing biological structure of eye
* Included some training component just to see if the system would improve with it; network development strategy did not require any training to implement
* Did not discuss how the Loihi controller interfaces with the servos, but included C++ code in the appendix (which I couldn't immediately find)

![Fig 1 of Balachandar et al 2020](https://2585980925-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MbT0JvqdvF6kx3gpkbo%2F-MhyDHztnzkG5tXjgo2k%2F-MhyRkP9RcC4cx0qFjiM%2FBalachandar%20et%20al%202020%20Fig%201.PNG?alt=media\&token=1f52b272-f679-4992-80ef-2d4d92374cdc)