TinyML Security Applications

At @vjreddi’s suggestion, I’m starting a thread here about cybersecurity and ML.

By way of introduction, I’m a cybersecurity consultant with a background in physical security. One of my interests is deploying ML to the field to help address security issues.

Right now, some endpoint protection (anti-malware/anti-virus) products are using ML models executing on the endpoint to identify malware with great success. However, the vast majority of security products still backhaul data to the cloud for processing. Anyone who has taken the TinyML course will understand that this approach is expensive (power, network traffic, etc.) There are also significant security and privacy implications, especially when network traffic is concerned.

I’m new to ML in general, and I’m very interested in understanding how ML can be applied to network traffic, etc.

In the physical security world, I think there are tremendous applications for TinyML. For example, consider alarm systems. Sensors used in the majority of deployments are low cost and attempt to turn the real world into a boolean. Door sensors complete or break a circuit. IR motion sensors use relatively simple rules to determine the state of a dry contact or send a boolean motion detection report by RF. Some installers are forced to connect a pair of $20 IR motion detectors in parallel to help reduce false alarms. (It’s a bit crazy that in 2021 you still can’t program many alarm systems to require two sensors to trip before sending an alarm. Some have related capability but suffer from usability issues.) The combination of human error and overly simplistic detection results in the vast majority of alarm activations being false alarms. As a result, the police respond at a low priority and leave the owner a bill for the false alarm service.

These are only a few examples – I can’t help but see many opportunities for ML in the security space.

1 Like

Hello @SecurityGuy,

“Industrial strength” security is being addressed at the proprietary level by Microsoft with its Azure Sphere Photon technology. Even the supplier (i.e. Seeed) cannot provide details! :grinning:

Similarly, Arduino has its efforts but the end results are flaky with support issues being constantly logged as “Cannot Repro!”

Ubuntu Core has a solid architecture for this purpose. The backwards compatibility is its strength since industrial IoT devices need a long life-cycle unlike the common Home Automation examples (with default passwords).

Kind regards.

2 Likes

Thank you for introducing this topic, I also am intrigued by the potential for TinyML in physical security. Just the concept of a smart door lock, as simple as it is, would seem to have promise.

I don’t know how real world it is at this point, but I have a vision of using a layered approach. Since the TinyML devices are going to be relatively cheap, using several of them and taking a consensus result could help reduce false alarms and possibly allow for a categorical response instead of a boolean “call the police, don’t call the police” choice.

1 Like

(Sorry this one got very long…I surprised myself…)

@stephenbaileymagic In the physical security world, I think your approach makes a lot of sense. I see two applications of TinyML. The first are the alarm sensors themselves.

For example, passive Infrared (PIR) detectors are very common because of their low cost and longevity. Unlike door/window sensors (which really can be binary), PIRs use a small array and simple rules to attempt to differentiate a human from moving another heat source. They are very low tech.

For example, “pet immune” or “pet-proof” sensors have specific mounting requirements (minimum height above the floor, specified straight-line distance from any place the animal can be, not facing stairs, etc. Then they basically use movement, size, and vertical detection to differentiate between a cat or dog and an intruder.

If installed perfectly, they’re reasonably reliable. But, for example, they can’t point toward ascending stairs, or a 10 lbs cat walking down the stairs looks like a person to the sensor. Similarly, if a cat jumps off an object, off the side of the stairs, etc., as cats love to do, they look very tall and trigger the sensor.

For dogs, PIRs rely mostly on the size of the heat signature. That’s why you’ll see weight limits (e.g. pet proof to 100lbs). They’re essentially betting that an 80-pound criminal isn’t going to crawl like a dog.

PIRs also require a minimum movement speed, generally sideways across the sensor. That helps eliminate false alarms when the furnace kicks in on a cold day, but for example, installers have to be careful that a floor register isn’t going to heat a heavy curtain and move it around. If you want to have some fun, put an alarm on test and walk very slowly in a straight line toward the sensor.

Even when properly installed, PIR false alarms are too frequent. With the rise of DIY alarm installations, things get even worse because people install PIRs without understanding how they work.

Of course in an industrial setting, there are bigger problems. A mouse that runs right in front of a sensor looks massive, and even a large bug crawling across the sensor of some devices can create a false alarm. So, there are so-called dual-tech devices that combine PIR with microwave detection and require two the different detection systems to fire at the same time. However, microwave is much more expensive, active, contributes to RF noise, and can’t run for long on batteries (as compared to passive PIR detectors that can run on a battery for a few years.) Microwave is also not without false alarm issues. A print job on a laser printer in the middle of the night has been known to do the trick.

Acoustic glass break sensors are a good option to deal with areas where motion sensors may be problematic, or as a second intrusion indication (more on that later). They can also cover a much larger area and are sometimes more cost effective and aesthetically pleasing than putting contact sensors on multiple windows. Glass breaks, as they are called in the field, have become more sophisticated, but could be improved. The better ones detect the sound of an impact followed by the specific frequencies emitted by framed glass breaking. In theory, you can use them 24x7 (i.e. no armed/disarmed state needed). However, in practice I’ve seen them triggered by dropping a toilet seat in a power room, putting down a stack of dishes too hard, and most famously by a cat knocking a large plastic bin of dry cat food off the counter onto the floor.

So my first thought is to improve the sensors themselves. Basic wired systems these days still use a closed loop with an end-of-line resistor, and wireless sensors basically send a serial number plus a few bits for different conditions (motion, low battery, etc.). Most wireless alarm sensors in use today have little if any security on the RF side. For example, you can use a cheap SDR setup to monitor all but the latest Honeywell systems, and with a bit of work you can impersonate any of the devices. But with some people looking to ZigBee, etc., in the future more intelligent devices could send more information. Even an integer with a confidence of 0-255 would be an improvement.

The second area is the alarm panel itself. Both installers and alarm monitoring centers are essentially forced to use tradecraft to overcome the limits of binary alarm sensors and the dumb alarm panels that report them.

For example, in a house with a pet, it is unlikely that you’d install a PIR in each room. Take a look around your livingroom, for example, and see if you can find a PIR location that doesn’t face the street (car headlights), doesn’t face a heat register, and where a cat can’t come within 6 feet of the sensor (straight line measurement from sensor in any direction.) Easy if there is no furniture.

I have a love-hate relationship with PIRs. We them to detect movement, otherwise the system is incomplete. We can’t have a situation where an intruder can pry or break in through a window and not be detected. But every PIR installed increases the false alarm potential.

Similarly, unless the house is pre-wired or the basement ceiling is open, it’s difficult to install contact switches (mechanical or magnetic) on a lot of newer windows. Even if the basement ceiling is open, drilling through the window frame into an exterior wall and trying to shove a wire through is nasty business.

So a common solution is to put a contact sensor on each exterior door, install a PIR in a hallway (through which you hope the intruder is forced to pass), perhaps use a glass break detector to cover the most vulnerable areas, and then the lesser known installer trick of wiring internal doors and configuring the alarm panel to believe that the internal door contacts are motion sensors so that they apply the same armed away/armed stay/disarmed logic.

Why? The dirty little secret of alarm monitoring is that they want to be able to ignore (or at least not dispatch the police) when (not if, but when) the PIR triggers a false alarm.

Instead of making a more intelligent alarm panel that understands, for example, that in the absence of any other devices triggering, a single hit from a PIR is almost always a false alarm, the mainstream alarm industry has to work around that limitation.

Unfortunately the person in the monitoring center trying to make that decision doesn’t know anything about the layout of the building. They might have a list of sensors, but they generally don’t have the time or information to follow the logic, “Ok, the front door hasn’t been opened, and the glassbreak covering the back of the house hasn’t been triggered, but the PIR in the hallway just inside the front door detected motion once” vs “hallway PIR tripped multiple times plus bedroom door opened”. In a lot of cases the alarm monitoring center doesn’t want the liability, so in the absence of specific directions they will call a keyholder. They might get someone like me who will say, “One hit on one PIR? Thanks for the call, don’t dispatch, I’m going back to sleep.” or they might get someone who is terrified that their home is being burgled while they are out of town, in which case now the police are going to end up doing a perimeter check and leave them a ticket or send an invoice.

There is such a high false alarm percentage that police generally de-prioritize alarm calls. The public see ads of digital walls protecting them and near instant police response. Reality is very different, most forces won’t accept an alarm call over 911. Some forces will not respond until the alarm monitoring center contacts a keyholder and gets instructions. However, some will prioritize a call that meets certain verification procedures, and in limited cases (for example with video verification) they might even accept a 911 call and respond on a confirmed crime-in-progress basis.

Even worse, some will stop responding at all after a certain number of false alarms. It is not difficult for a malicious person to intentionally create false alarms, but I won’t discuss those techniques here.

There are alarm panel features to help with false alarms. For example, if you disarm the alarm within a certain period of time after alarm activation, the panel can send a “cancel” signal.

Some have a feature called “cross zone” – the concept is that more than one specified zone has to trip before the alarm is activated. But, presumably being risk adverse, some panels will enter a “trouble” state when only one of the configured zones is tripped. While that is an improvement in terms of false alarm reduction, it usually means that authorized people are greeted by a loudly beeping alarm panel warning of “trouble” that they have to manually reset when, in fact, all that has happened is the cross zoning feature prevented a false alarm. Not a winner.

Working with most alarm panels, especially the mainstream ones, is like a trip back to 1980. They’ve added wireless, WiFi, and cellular, but the alarm detection logic is essentially unchanged in decades.

From an ML perspective, a false alarm and a real intruder result in different patterns when sufficient alarm sensors are present. I think there’s an opportunity there, especially if combined with smarter sensors.

1 Like

@baqwas, there are certainly a lot of proprietary projects, but what interests me is leveraging ML to detect or even pre-filter data. While I just wrote a surprisingly long comment on alarm systems, I’ve been in cybersecurity for 25-ish years and the overwhelming problem is that there is too much data. The needle in a haystack analogy doesn’t fit – most of the time we’re looking for a piece of hay in a field full of haystacks.

In the endpoint protection realm, companies like Cylance (naming them because they were first) are successfully applying ML models to detect malware and similar threats. Improvements are required, but overall it is much better approach than traditional AV signatures. Updates are less frequent, lower CPU utilization (looking at behaviour instead of comparing to a huge list of signatures), offline decision making – all the goodness that goes with ML on the endpoint instead of backhauling everything to the cloud.

However, there are other opportunities. A lot of network security products still backhaul all traffic or selected traffic to the cloud for analysis. Web Application Firewalls (WAF) products use static rulesets.

One of my interests is to understand how to take, for example, HTTP events and convert them into a format suitable for machine learning. The HarvardX courses have shown me the basics of how to do this with numberic data, audio, and video, but I haven’t wrapped my mind around how to convert GET /a/b/c?d=eric&e=today&f=12345 and GET /x.html into something that can be fed into an ML model. As a security guy I can often look at these in retrospect and pick out abnormal traffic pretty quickly, but not fast enough to prevent anything. On the surface it seems like a good application for anomaly detection, but I’m stuck on how to take a string of arbitrary length with various parameters and turn it into a fixed-size input. (Any hints on where to start Dr. @vjreddi or anyone else?)

Another area is dealing with email and related user behaviour. While intrusions continue to occur through hacking services exposed to the Internet, one of the largest attack vectors is email. People open files or click on links and trigger the first of a multi-stage infection. It might be ransomware, or it might be a backdoor that gives an intruder their initial entry point into a company. Right now our best defence is to combine security awareness training (users can often spot the emails) with endpoint protection software (preferably ML-based) and DNS filtering services (block known phishing domains). Network-based controls are becoming far less useful. I can’t help thinking that there are some ML applications here. Privacy is a major issue with email, so solutions that work on a user’s PC are required.

1 Like

Thank you very much for your reply @SecurityGuy , I found it very interesting on many levels!

I appreciate the insight in to the real world applications of PIR sensors. I had actually considered them for an unrelated project that never got off the ground, but never took the time to understand the limitations so this is very helpful.

From everything you have described and from what I have learned in the TinyML course so far, it seems that there really is great potential here. The difficulties in tailoring the alarm thresholds to deal with the cats or the heaters of the world sounds exactly like what Laurence talked about in the video in which he described the difficulties of using traditional programming to create a device that can tell the difference between walking and golfing. As you mentioned, even adding a confidence level of some sort would be a big advantage.

My intent after finishing up the HarvardX course is to spend some time experimenting with creating an “anomaly detector” in my house. That is to walk through the pipeline of collecting data about what is normal, build a model and deploy it, then connect the sensors wirelessly to a more powerful device (a Raspberry Pi in this case) which can do further processing/decision making. Obviously, what I am going to do is on a very small scale, but I can hopefully start getting an idea of what would really be involved in this type of system.

In any case, thank you very much for sharing your knowledge! It has given me much to think about.

For:

how to take a string of arbitrary length with various parameters and turn it into a fixed-size input

My reflex response is to hash it - a time honored technique!

Kind regards.

@baqwas For other applications I would agree that a hash is a good way to create a fixed-length representation of data, but I don’t think that will work for ML because the neutral network would have no insight into the difference between components in the URL.

For example,

GET /search?name=bob
GET /search?name=eric
GET /hackingattempt.php?nasty1=nasty2

Would all result in a different output from the hash, and the fact that the first two examples have similar elements would be hidden from the neural network.

Hello @SecurityGuy,

Don’t want to get off-topic in this forum. Don’t want to be a hammer in search of nails but I’ve managed teams doing Bayesian inference with web server log files since 2001 across different web servers and standards.

Kind regards.

@baqwas Not off-topic at all…I’m very interested in how logs can be fed into ML. I’d like to understand how the neural network would be able to learn patterns if the data is hashed. Does the NN ultimately see through the hash alg in some way?

Hello @SecurityGuy,

Apologies for being cryptic. I was alluding to the values not the fields - quantization (I’m sure you recognized my reflexes are not slow but severely mistake prone). The fields/values themselves should be transferred to a NoSQL repository for the traditional data prep operations before you start the ML process. My recommendation is Redis rather than the popular flavors of NoSQL because then you can lower the latency this way (after all even SSDs have MTBF stats).

My other recommendation is that you explore the mining part before embarking on the ML part unless you just have a few types of web server solutions in mind given that different types of traffic need different forensics. For a limited number of web server solutions, as you rightly recognize, jump right into the ML part after you establish a representative set of the heavily used parameter fields. Given your background you know more about the types and commands for web servers than I ever will. I am assuming that your solution will be applied to cases where you have prior knowledge of the scope web server operations (and predominant access methods).

Kind regards.

Just arrived in my mailbox:

https://ubuntu.com/download/iot?utm_source=email&mkt_tok=eyJpIjoiWWpBellqWTFZbU5oT1RkaCIsInQiOiJtSFAxYm4ya21WZmhSTmFkRHpySHVjelhKWnM3azJ0YkE2cXJDbHlPWW91Mkk3SmpCQnRzNWJua3J0U0RPb0Y3OTRpbHg4ekV0RmVuK0xQM1BTM3JncldcL3pTWTZscXBUV2ZrTXQrXC8xRmxPS0ZFUWxyZDZLMU1XZTJaMlNkRm1qIn0%3D#core

Is that a valid link?

@vjreddi I checked the link out – removing the tracking codes, it’s https://ubuntu.com/download/iot

1 Like

Well, since you are our security head, I was brave enough to click it :slight_smile: Thanks!

I happened to have my “test” VM running, so low risk :smile: