# How to hide a backdoor in AI software – such as a bank app depositing checks or a security cam checking faces

**URL:** https://discuss.tinyml.seas.harvard.edu/t/how-to-hide-a-backdoor-in-ai-software-such-as-a-bank-app-depositing-checks-or-a-security-cam-checking-faces/667
**Category:** General Discussions
**Created:** [May 5, 2021, 2:34pm UTC](https://discuss.tinyml.seas.harvard.edu/t/how-to-hide-a-backdoor-in-ai-software-such-as-a-bank-app-depositing-checks-or-a-security-cam-checking-faces/667 "2021-05-05T14:34:39Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![W4ym0usa](https://yyz1.discourse-cdn.com/flex027/user_avatar/discuss.tinyml.seas.harvard.edu/w4ym0usa/32/184_2.png) [@W4ym0usa](https://discuss.tinyml.seas.harvard.edu/u/W4ym0usa)
#### Post date: [May 5, 2021, 2:34pm UTC](https://discuss.tinyml.seas.harvard.edu/t/how-to-hide-a-backdoor-in-ai-software-such-as-a-bank-app-depositing-checks-or-a-security-cam-checking-faces/667/1 "2021-05-05T14:34:39Z")

</div>

Hi there everyone,

A colleague of mine showed me this really interesting article on AI. In summary, it describes how the process of pruning and quantizing can be manipulated to the advantage of an attacker, such as a visual recognition app. Interesting stuff and directly related to some project ideas I may have to rethink!

> **[How to hide a backdoor in AI software – such as a bank app depositing checks...](https://www.theregister.com/2021/05/05/ai_backdoors/)**
>
> Neural networks can be aimed to misbehave when squeezed

What does everyone make of that?

---

<div class="post-metadata">

### Author: ![CallaJ](https://avatars.discourse-cdn.com/v4/letter/c/919ad9/32.png) [@CallaJ](https://discuss.tinyml.seas.harvard.edu/u/CallaJ)
#### Post date: [June 4, 2021, 5:50am UTC](https://discuss.tinyml.seas.harvard.edu/t/how-to-hide-a-backdoor-in-ai-software-such-as-a-bank-app-depositing-checks-or-a-security-cam-checking-faces/667/2 "2021-06-04T05:50:23Z")

</div>

I think I’d better augment heartily images in my data set, and with enough noise, in order to build robustness to “seeming”.

---

<div class="post-metadata">

### Author: ![vjreddi](https://yyz1.discourse-cdn.com/flex027/user_avatar/discuss.tinyml.seas.harvard.edu/vjreddi/32/35_2.png) [@vjreddi](https://discuss.tinyml.seas.harvard.edu/u/vjreddi)
#### Post date: [June 12, 2021, 4:07pm UTC](https://discuss.tinyml.seas.harvard.edu/t/how-to-hide-a-backdoor-in-ai-software-such-as-a-bank-app-depositing-checks-or-a-security-cam-checking-faces/667/3 "2021-06-12T16:07:02Z")

</div>

Very interesting article, thanks for sharing! Similar to this, one of my student groups in the Harvard tinyML course showed that quantized/optimized models can also make inference biases more pronounced.
