Last week I watched a team win a hackathon with a single idea: stream every 911 call through an AI that transcribes, triages, and pre-fills the dispatch record in real time — while the caller is still speaking. The dispatcher’s screen updates before they finish their sentence.
That couldn’t have been built by a small team, affordably, two years ago.
Something meaningful is happening in a shift in data architecture that redefines real-time product strategy.
What streaming AI actually is
Most applications process data in batches — collect, store, analyze later. Streaming AI does the opposite: it processes data as it arrives, event by event, and applies intelligence in the flow.
The core stack has three layers. Kafka is the event log — a durable nervous system that ingests data from any source at any scale. Flink is the processor — it filters, aggregates, and enriches events in flight. An LLM or ML model sits inside that pipeline, reasoning over each event as it passes through.
The result: insight and action in seconds, not hours. A fraud signal caught before the transaction clears. A stockout flagged before a customer notices. A patient showing early sepsis markers — lactate creeping up, heart rate variability shifting, a nurse note mentioning confusion — flagged six hours before a crisis, not after.
The infrastructure is old. The opportunity isn’t.
Kafka has been around since 2011. Flink since 2014. Real-time fraud detection, ad bidding, recommendation engines — all of these ran on streaming infrastructure long before anyone said “LLM.”
So what changed? The technical catalyst: unstructured data. Before large language models, streaming AI was limited to structured inputs — transactions, clicks, sensor readings, log lines with known schemas. Processing a 911 call, a physician’s note, a shelf image, or a security alert narrative in real time required expensive, purpose-built ML pipelines that took months to train and broke whenever the input changed.
LLMs removed that constraint. They made voice, images, and free text first-class citizens in the stream. They enabled zero-shot pattern detection — no labeled training data required for each new threat type or failure mode. And they lowered the expertise floor enough that a small team can now build intelligent streaming pipelines that would have taken eight ML engineers in 2020.
The infrastructure predates the AI wave. The applications are new.
Evaluating the opportunity landscape
Sixteen use cases, scored across six criteria. The matrix separates genuine step-changes from incremental improvements — and from the traps where large markets are already locked up by incumbents the AI wave didn’t disturb. Sort any column; filter by category.
| # ↕ | Use case ↕ | Mkt ↕ | Diff ↕ | Feas ↕ | LLM ↕ | Reg ↕ | AI ↕ | Score ↓ |
|---|
The top of the table shares a common trait: unstructured data that pre-LLM systems couldn’t process intelligently at streaming speed. The bottom cluster — ad bidding, financial markets, price notifications — are not small markets. The AI wave simply didn’t change their core loop. Those races were run and won before 2024.
The eight use cases most transformed by AI
Each of the following scored 4 or 5 on AI unlock. What distinguishes them isn’t the stack — it’s what becomes possible when an LLM can reason over live, unstructured data in real time.
Where LLMs Changed Everything
Eight use cases that couldn't be built at scale before 2024 — and what makes each one different now.
911 & Public Safety
An LLM transcribes the call, extracts location and incident type, and pre-fills the dispatch record while the caller is still speaking. The dispatcher focuses on judgment, not data entry.
Healthcare Claims Approvals
The moment a prior auth arrives, an LLM reasons across clinical history, payer policy, and current guidelines simultaneously — producing a structured clinical argument, not just approve/deny.
Retail Computer Vision
Multimodal LLMs turned shelf monitoring from expensive per-SKU custom models into general-purpose inference. Cameras stream frames; AI detects stockouts, misplacements, and theft patterns in real time.
Cybersecurity / Threat Detection
LLMs narrate behavioral sequences across logs, entity graphs, and network telemetry — spotting novel attack patterns and explaining them in plain language before damage occurs.
Clinical Patient Monitoring
Vitals, lab results, and nurse notes stream together. An LLM catches the multi-signal pattern that predicts sepsis six hours early — the combination no single threshold alarm catches.
Customer Support Triage
Before the customer types a word, the system has streamed their session and assembled context. The agent receives likely intent and the resolution path most likely to work — already prepared.
Content Moderation
LLMs evaluate each post against account history, thread context, and network-wide behavior in the past 60 minutes — catching coordinated campaigns a single-post classifier would miss entirely.
Behavioral Analytics
Rage clicks and form abandonment become a full diagnosis: the LLM traces the session, identifies the root cause, estimates revenue impact, and drafts the bug ticket — automatically.
1. 911 / Public Safety
Today’s 911 dispatch is a human transcription bottleneck. A caller describes a scene in panicked, fragmented language. A dispatcher manually categorizes and routes. The lag between call and unit dispatch is measured in minutes.
The streaming version reduces that lag. Every call streams live audio into a Kafka topic. A Flink job runs continuous speech-to-text, feeding rolling transcript windows into an LLM that simultaneously extracts incident data, detects secondary signals — sounds of violence, weapon mentions, distress indicators — cross-references open incidents nearby, and pre-populates the dispatch record. The dispatcher’s screen updates in real time. More comprehensive implementations add a second stream scanning social media, traffic cameras, and prior incident history at that address before the first unit arrives. The dispatcher doesn’t get replaced. They get 90 seconds back.
2. Healthcare Claims Approvals
Prior authorization is a significant friction point in American healthcare. A physician submits a request. It sits in a queue. A human reviewer checks it against a policy document days later. Patients wait.
The streaming version treats every clinical event as a Kafka event: lab result, diagnostic code, physician note, pharmacy fill. Flink maintains a continuously updated patient evidence record. The moment a prior auth arrives, an LLM reasons across three streams simultaneously — clinical history, payer policy, and current clinical guidelines. The output isn’t approve/deny. It’s a structured clinical argument: which criteria are met, which are missing, what specific documentation would close the gap. Physicians get a feedback loop in seconds. Payers get auditable decisions.
3. Retail Computer Vision
Focal Systems points cameras at store shelves and detects stockouts. That’s the standard implementation. The more capable version maintains a persistent spatial-temporal model of the store — one that knows not just “this slot is empty now” but “this slot has been declining for four hours and historically empties faster on Tuesday afternoons.”
A multimodal LLM processes frames for things no rule-based system catches: wrong-facing products, price tag mismatches, theft-pattern intrusions versus normal browsing. A second stream tracks associate locations and task queues. The system routes, not just alerts.
4. Cybersecurity / Threat Detection
Existing SIEMs are good at known signatures. They’re poor at two things: novel attack patterns that don’t match existing rules, and explaining what is actually happening to an analyst in plain language.
The streaming layer sits above the SIEM. Flink maintains entity graphs — which IPs talked to which hosts, which accounts touched which files, which processes spawned in which sequence. An LLM consumes rolling windows and narrates the attack: “Over the past 14 minutes, a Finance credential authenticated from an unusual location, queried the HR database schema, and made three outbound connections to an IP with no prior history. This is consistent with early-stage exfiltration.” It also generates hypotheses about what happens next — proactive hunting instead of reactive alerting.
5. Clinical Patient Monitoring
Alarm fatigue is a documented challenge in hospitals. Single-metric thresholds fire constantly. Nurses silence them. The subtle multi-signal pattern that predicts sepsis six hours early goes undetected.
Unifying every patient data stream — continuous vitals, lab results, nurse notes, medications, fluid balance — into a single Kafka pipeline changes that. An LLM reasoning across all streams catches the combination: slight lactate elevation plus heart rate variability shift plus a nurse note mentioning increased confusion. No single alarm triggers. The multivariate pattern does, with a natural-language explanation and recommended interventions.
6. Customer Support Triage
Most AI support tools classify a ticket after it’s submitted. That’s batch logic applied to a streaming problem.
The streaming version starts before the customer types anything. Their session — pages visited, errors encountered, features abandoned — streams as Kafka events. By the time they open a chat widget, the system has a hypothesis about why they’re there. Eight minutes on the billing page plus a history of churning after billing disputes routes them to a retention specialist, not standard support — with a pre-composed context summary including emotional trajectory and the resolution path most likely to work.
7. Content Moderation
Static classifiers fail on context. A post that’s acceptable in one thread is harassment in another. A phrase that’s benign from one account is a coordination signal from fifty accounts created this week.
Each piece of content streams in before it’s published. Flink enriches it in parallel — account age, prior violations, the thread it’s joining, what similar content is doing across the network in the past 60 minutes. The LLM reasons across all of it: “47 nearly identical posts appeared in 20 minutes from accounts created this week, all targeting the same user. This is coordinated harassment, not organic content.” No static classifier reasons across the graph in real time. That capability is new.
8. Behavioral Analytics
Tools like FullStory record sessions well. They diagnose them poorly — connecting a behavioral signal to a root cause and surfacing a fix automatically is where they stop.
Every user interaction streams as a Kafka event. Flink detects the signature — rage clicks, dead clicks on non-interactive elements, form abandonment loops. The LLM reasons across the full session: “User attempted to apply a coupon code three times — all failed with a non-descriptive error — then abandoned. The coupon field rejects valid codes due to a case-sensitivity bug in last Tuesday’s deploy. 340 sessions show this pattern. Estimated lost revenue: $18,400.” From behavioral signal to root cause to business impact to drafted Jira ticket. This automation of the final analysis step is a new capability.
What this means if you’re building
The most useful question isn’t “can I use streaming AI?” It’s “do I have unstructured data that currently sits outside my intelligence pipeline?”
If your product touches voice, images, documents, or behavioral session data — and you’re still analyzing it in nightly batch jobs — you’re accepting unnecessary latency. The gap between what you know and when you know it is now a product decision, not a technical constraint.