AI in Cyber Defense
The public works department of a small city has one IT technician and one contract programmer. On one Tuesday, an AI-powered security tool produced four outputs for the department. This lesson takes them in order and asks how each one helps defend the department's network, applications, and data.
The first three are recommendations for a person to review. The fourth is the tool's sorting of the day's events.
Security configurations
At 7:40 a.m. the tool finished reviewing the department's firewall rules and account settings. For each setting it judged weak, it recommended a more secure option.
| Item | Finding | Recommendation |
|---|---|---|
| R1 | Firewall rule 12 allows connections to the file server from any address. | Allow connections from the office network only. |
| R2 | Staff accounts stay open after any number of failed sign-ins. | After 5 failed sign-ins, lock the account for 15 minutes. |
| R3 | No staff sign-ins were recorded from midnight to 5:00 a.m. in the last 30 days. | Block staff sign-ins in that window. |
The IT technician reads the three. The file server is used only from the office, and a short lockout costs staff little, so R1 and R2 are applied.
R3 is wrong for this department. The snow-plow crews are dispatched between midnight and 5 a.m., and dispatch runs on staff sign-ins. The last 30 days had no snow, so the sign-in history the tool read shows none of that work. No setting records the plow schedule either. Applied, R3 would block the plow dispatch on the first night of snow, so the technician declines it.
Application code
At 10:15 a.m. the tool finished analyzing the code of the permit-application page at permits.example.org. It listed three vulnerabilities, each with a recommended mitigation.
| Item | Where | Vulnerability | Mitigation |
|---|---|---|---|
| V1 | apply.py, line 88 | The applicant's street address goes into a database query exactly as typed. | Pass the address to the query as a separate value. |
| V2 | upload.py, line 41 | A site-plan upload of any size is accepted. | Refuse uploads over 20 MB. |
| V3 | status.py, line 17 | The status page opens for any application number, whoever is signed in. | Show the page only to the applicant who filed it. |
The contract programmer reads the list and accepts V1 and V2 as written. The mitigation for V3 would lock out the two permit clerks, who open every application's status page to process it. So the programmer writes the fix to admit the applicant who filed it and the clerks' accounts.
Detection rules
At 1:00 p.m. the tool suggested a rule for the department's automated detection system.
Suggested detection rule D7
Watch: the code folder on permits.example.org
Fire: when a file changes through any account but the programmer's
Action: email it-desk@publicworks.example.org
The IT technician, who wrote the department's existing detection rules, reviews D7. It watches the right folder for the right change, since the programmer is the only person who should change the page's code. Its action is the problem. The it-desk address belonged to a technician who left in June, and nobody has read that mailbox since. The tool had no way to know that. The technician points the alert at a mailbox that is read every day, and then adds D7 to the system.
Sorting the day's events
Every sign-in, connection, email and file change at the department is recorded as a digital event. The tool's summary for Tuesday follows.
Event summary, publicworks.example.org, Tuesday 12:00 a.m. to 11:59 p.m.
Events recorded 2,592,000
Not flagged 2,590,760
Flagged as likely malicious 1,240
Handled by corrective action 1,213
Sent to the IT technician 27
A day has 86,400 seconds, so the department recorded 30 events a second on average. Even at one second an event, far too fast for careful examination, one person would need 720 hours to read the day's record. That is 30 days without a break, and a reader never catches up. Events arrive at 30 a second and are read at 1, so the unread record grows by 29 every second. Yet some of those events likely represent an adversary at work.
The tool sorted each event within seconds of its being recorded and flagged 1,240 of them. "Not flagged" means only that: a tool like this can miss a malicious event and flag a harmless one. That's why a rule like D7 and the human alerts below still matter.
Alerts and corrective actions
The tool was programmed with one response for each type of likely malicious activity. The table splits Tuesday's 1,240 events by type.
| Type of likely malicious activity | Events | Programmed response |
|---|---|---|
| Password guesses at the permit page's sign-in, 4 each from 277 addresses | 1,108 | Block each address after its 4th failed guess |
| Email attachment matching a known harmful file | 105 | Delete the attachment before delivery |
| Staff account signed in on two computers at once | 27 | Alert the IT technician |
One of the 27 alerts arrived at 3:02 p.m. A permit clerk's account had signed in on a second computer while the clerk was signed in at the front counter. The technician called the clerk, who hadn't signed in anywhere else, and locked the account and signed out all its sessions at 3:06 p.m. The account can open all 3,412 permit applications on file, with each applicant's name and address.
At 30 events a second, about 1.6 million events came before the 3:02 p.m. sign-in. Reading from midnight at one second an event, a person would reach it after about 450 hours, nearly 19 days. In that time the adversary could copy every applicant's name and address, or change or delete all 3,412 applications. The tool sorted the sign-in within seconds, so the account was locked four minutes after it signed in. In those four minutes the adversary opened two applications and changed none.