# Learning to Seek: Autonomous Source Seeking with Deep Reinforcement Learning Onboard a Nano Drone Microcontroller

**URL:** <https://discuss.tinyml.seas.harvard.edu/t/learning-to-seek-autonomous-source-seeking-with-deep-reinforcement-learning-onboard-a-nano-drone-microcontroller/319>\
**Category:** General Discussions\
**Created:** [February 7, 2021, 9:26pm UTC](https://discuss.tinyml.seas.harvard.edu/t/learning-to-seek-autonomous-source-seeking-with-deep-reinforcement-learning-onboard-a-nano-drone-microcontroller/319 "2021-02-07T21:26:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Duisterhof](https://avatars.discourse-cdn.com/v4/letter/d/3da27b/32.png) [@Duisterhof](https://discuss.tinyml.seas.harvard.edu/u/Duisterhof)\
**Post date:** [February 7, 2021, 9:26pm UTC](https://discuss.tinyml.seas.harvard.edu/t/learning-to-seek-autonomous-source-seeking-with-deep-reinforcement-learning-onboard-a-nano-drone-microcontroller/319/1 "2021-02-07T21:26:51Z")

</div>

How can we leverage the power of TinyML to unlock novel applications with constrained flying robots?

In our recent work, we present fully autonomous source seeking onboard a highly constrained nano quadcopter, by contributing application-specific system and observation feature design to enable inference of a deep-RL policy onboard a nano quadcopter. We verify our approach with simulation and in-field testing on a Bitcraze CrazyFlie, a 30g nano quadcopter, using only the cheap and ubiquitous Cortex-M4 microcontroller unit.

We show how ML can be a critical component in application-specific system design for robotics, as our model learns to use lightweight and cheap sensors instead of heavy and expensive cameras. Deploying TinyML onboard a flying robot poses a unique set of challenges, such as severe payload and endurance constraints. Contrary to most TinyML applications, the weight of compute is much more important than its power consumption: every gram counts!

We would love to hear your feedback on this project, that is currently under review at ICRA.

[![](https://canada1.discourse-cdn.com/flex027/uploads/tinyml1/original/1X/6cc08b35da2684c04cd0ecc87407f92bd7dbfd75.jpeg "Learning to Seek") ](https://www.youtube.com/watch?v=wmVKbX7MOnU)

[Paper](https://arxiv.org/abs/1909.11236)

---

<div class="post-metadata">

**Author:** ![liorb](https://avatars.discourse-cdn.com/v4/letter/l/bbe5ce/32.png) [@liorb](https://discuss.tinyml.seas.harvard.edu/u/liorb)\
**Post date:** [February 10, 2021, 2:16pm UTC](https://discuss.tinyml.seas.harvard.edu/t/learning-to-seek-autonomous-source-seeking-with-deep-reinforcement-learning-onboard-a-nano-drone-microcontroller/319/2 "2021-02-10T14:16:58Z")

</div>

Is the NN trained with the all inputs , 1 light and 4 laser reading ?

---

<div class="post-metadata">

**Author:** ![Duisterhof](https://avatars.discourse-cdn.com/v4/letter/d/3da27b/32.png) [@Duisterhof](https://discuss.tinyml.seas.harvard.edu/u/Duisterhof)\
**Post date:** [February 11, 2021, 7:47pm UTC](https://discuss.tinyml.seas.harvard.edu/t/learning-to-seek-autonomous-source-seeking-with-deep-reinforcement-learning-onboard-a-nano-drone-microcontroller/319/3 "2021-02-11T19:47:55Z")

</div>

We feed 4 laser readings and 2 application-specific features into the network. Those 2 features, s1 and s2, are a function of light readings:

 ![image](https://canada1.discourse-cdn.com/flex027/uploads/tinyml1/original/1X/4a8ac5dc830a088a09ba7ccd1e15ed86e1ef1a4d.png)  
Here c is the current light reading, and cf a low-passed version of the light reading. We design custom features to limit the input size of the network, and with that computation.