← Home
SECURITY BRIEF
By Luke @ Lukata·Updated May 25, 2026·6 min read· Download PDF

AI Safety & Security

AI is starting to touch evidence, money, customer data, business tools, and production code.

Before shipping AI, answer one question:

Can this AI be tricked, trusted, tested, and controlled?

[ Back to Lukata ]

By the numbers

362
AI incidents documented in 2025
233
AI incidents documented in 2024
+55%
Year-over-year increase, 2024 to 2025

Source: Stanford HAI AI Index 2026

Frameworks builders should know

If you ship AI in production, you should be able to explain each of these in one sentence to a non-technical stakeholder.

MODULE 1

OWASP Top 10 for Agentic Applications

AI agents that use tools and take action
Checks
toolsmemorypermissionsactions
What it is

A public list of common risks for AI agents that can use tools, remember context, and act on their own.

What it checks

Whether an AI agent can be tricked, misuse tools, overstep permissions, remember bad information, or take actions the owner did not approve.

Why it matters

If an AI agent can act inside a real product or company system, it needs limits before users rely on it.

MODULE 2

OWASP Top 10 for LLM Applications

Chatbots and LLM apps
Checks
prompt injectiondata leakshidden instructions
What it is

A public list of common risks for apps built with large language models.

What it checks

Whether a chatbot can be tricked, leak private data, expose hidden instructions, trust unsafe content, or use too much freedom.

Why it matters

Most AI apps start as chatboxes. They still need security rules before they touch real users or private data.

MODULE 3

MITRE ATLAS

How AI systems get attacked
Checks
attack patternspoisoningmodel theft
What it is

A public map of attack patterns used against AI systems.

What it checks

How attackers try to trick, poison, steal from, or manipulate AI systems.

Why it matters

Builders need to know what real attacks look like before they decide what to test.

MODULE 4

NIST AI Risk Management Framework

How organizations manage AI risk
Checks
governmeasurereviewreduce
What it is

A voluntary guide many organizations use to manage AI risk.

What it checks

Whether a team has a clear way to name, measure, review, and reduce AI risk.

Why it matters

It gives teams a shared process for deciding when AI is safe enough to use.

Recent incidents

Three recent examples of why AI systems need limits before they touch tools, data, or code.

May 2026

Microsoft Semantic Kernel

Risk type
prompt injectioncode execution
What happened

Microsoft disclosed vulnerabilities where prompt injection could cause an AI agent framework to run code on the machine hosting it.

Why it matters

If AI can use tools or run scripts, bad instructions can become real actions.

Lesson

Do not let AI tools run commands freely. Limit what they can do, log what they do, and require approval for risky actions.

March 2026

Meta rogue AI agent

Risk type
data exposurepermission failure
What happened

A rogue internal AI agent at Meta reportedly exposed sensitive company and user data to employees who did not have permission to access it.

Why it matters

Even internal AI tools can break approval boundaries if permissions are not tightly controlled.

Lesson

AI agents that can share, publish, or access company data need strict limits and human approval.

March 2026

ROME crypto-mining incident

Risk type
resource misusecontainment failure
What happened

Reports said an experimental AI agent tied to Alibaba-affiliated research used training resources to mine cryptocurrency during testing.

Why it matters

AI agents can behave in unexpected ways when their goals, tools, and limits are not controlled tightly enough.

Lesson

Agents need containment, monitoring, and hard limits before they are given resources or system access.

Builder rules before shipping AI

None of this is exotic. These are the operating rules I use.

1

Use public frameworks as the starting point

OWASP Top 10 for Agentic Applications and NIST AI RMF are free, public, and the floor before any agent touches customer data, payments, tools, or production code.

2

Test before real users

No AI agent should reach real users without testing. If it can touch money, data, tools, or customer accounts, it needs stricter review.

3

Tell people when AI is involved

If AI is part of a hiring, legal, medical, or financial decision, the person affected should be told and able to ask for human review.

4

Own what your AI says and does

If a company deploys AI to customers, it should assume responsibility for what the AI says and does. Do not blame the model when users are harmed.

5

Use AI to remove repeated work, not erase people

Take over repetitive tasks, but leave judgment, review, and final decisions to people.

6

Protect the people who report AI bugs

Independent researchers often find serious AI security issues. They need safe ways to report them.

Sources

Every load-bearing claim above traces to one of these public sources.

NEXT
Contact

Shipping AI that touches data, tools, or customers?

I build AI software that sets clear limits before AI touches customers, data, or production code.

Have a case study, a policy proposal, or a correction for this page? Same address: lukatasolutions@gmail.com.