Security1 publisher2 min readPublished
A KTH lab agent reboots industrial hosts on six packet counts per interval
The reinforcement-learning defender was trained on 40,000 half-minute traffic samples from a container replica of a segmented plant network, and its best-performing version works from a model of the attacker it is fighting.
The Watch · Security desk
What happened
- KTH Royal Institute of Technology built a container replica of a segmented industrial network, attacked it repeatedly across 14 days of running time, and trained a defense agent on the captured traffic.
- The agent's entire view is six numbers per interval, all packet counts: traffic crossing the network's segments and traffic moving to and from individual machines.
- Its choices are to do nothing, reset one of three supervisory hosts, reset one of two water tank processes, or reset every host in the supervisory and control subnets at once.
- A reset reboots the target, renews its credentials and changes its IP address, and the paper states it can briefly interrupt industrial operations.
- The researchers have released their implementation and plan to test the approach on an industrial testbed with a partner.
Compiled by The WatchSomething wrong?How this is made
Why it matters
- decision Running this in a plant means someone signs off on software bouncing a live process on inference from six counters, and the authors leave operational safety to future work while noting it limits what a defender strategy may do.
- cost The observation model the math calls for would take about 95 years of emulator time to measure at 30 seconds a sample, so any deployment inherits an approximation that nobody can afford to validate the expensive way.
- constraint Reproducing the headline result requires the adversary model as well as the traffic, because the state estimator that got closest to full visibility is fed a description of how the attacker behaves.
- capability The transferable part needs no policy training: an OT team can put per-asset probabilities of scanned, exploited and inspected on a screen and leave the reset decision with a human.
The observation model rests on 40,000 measurement periods of 30 seconds [6], about 333 hours of emulated traffic, which accounts for the 14 days of running time the paper reports [1]. Modeling how traffic varies with the full system state would take roughly 100 million measurements [7]. At 30 seconds each, that is about 95 years of collection [2]. So the researchers fitted the smaller thing: how traffic varies with the attacker's action alone [7]. The sample they did collect is 0.04 percent of what the full-state version would need [3].
Most reinforcement-learning work on industrial intrusion response has assumed the defender can see the system state or the attacker's actions directly. The authors call that unrealistic, and say the studies that do model partial observability tend not to explain where their observation model came from [5].
The best of the three trained agents keeps 500 running guesses about the state of the network, updates them every interval, and hands a compressed version to its policy [8]. It beat the two agents fed raw observation history and came close to a baseline handed full visibility into the system state [9]. To update those guesses, it runs them through a model of how the system evolves, and that model includes how the attacker behaves, so the agent closest to the full-visibility baseline is working from a description of the adversary it defends against [11]. Feeding a policy four intervals of history instead of one made it worse [10].
The network under test is three supervisory hosts, two PLCs and two tanks [12]. Both HMIs run HTTP with weak credentials, and the engineering workstation runs SSH, Telnet and SMB with weak credentials and exposure to CVE-2017-7494 [13]. The researchers did not study whether their model generalizes to other configurations or attack types [14].
One piece transfers without any of the learning machinery. The learned agent and the paper's simpler threshold baseline both maintain a probability distribution over how far an intrusion has progressed on each asset, from undiscovered through scanned, exploited and inspected, weighting each stage by what it would cost [15]. A console showing the probability that a host has been exploited is buildable without reinforcement learning [16].
What to watch
- The planned testbed run with an industrial partner: whether the reset actions survive contact with real process safety limits.
- Whether the released implementation is rerun against attack sequences that differ from the one baked into its transition model.
- Whether the six-feature observation model holds on a network larger than three supervisory hosts, two PLCs and two tanks.