AI Security Ops cover art

AI Security Ops

AI Security Ops

By: Black Hills Information Security
Listen for free

Summary

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).© 2025 Black Hills Information Security Politics & Government
Episodes
  • Agent Pentest Benchmarking | Episode 52
    May 14 2026
    In this episode of BHIS Presents: AI Security Ops, the team breaks down a new benchmarking framework designed to evaluate AI pentesting agents against real-world offensive security scenarios.What began as experimental evaluation of “can AI hack?” has quickly shifted into something much closer to operational reality. Organizations are now seeing a surge in agentic tooling and automated pentesting workflows, where human-guided AI systems consistently outperform fully autonomous agents in complex, unsupervised environments.As AI tooling evolves, teams must balance speed with validation, monitoring, and oversight as offensive capabilities outpace defenses.We dig into:The new “AutoPenBench” framework for benchmarking AI pentesting agentsWhy fully autonomous AI hacking only achieved a 21% success rateHow human-assisted AI workflows increased success rates to 64%Testing AI agents against Log4Shell, Heartbleed, Spring4Shell, and classic web exploitsWhy modern offensive AI systems still require heavy human oversight and validationHow custom internal AI frameworks are already finding vulnerabilities humans missedThe operational role of prompt engineering, scaffolding, and agent memoryReal examples of AI agents mis-scoping infrastructure and chasing irrelevant targetsHow AI lowers the barrier for ransomware operations and offensive capability developmentWhy defensive teams need stronger edge visibility, packet capture, and AI-aware monitoring strategies⸻📚 Key Concepts & TopicsAI Pentesting & Agentic SecurityAutonomous AI hacking agentsAgentic AI workflowsAI-assisted penetration testingOffensive security automationBenchmarking & EvaluationAutoPenBenchAI security benchmarkingHuman-in-the-loop validationLong-horizon task evaluationOffensive Security OperationsSQL injectionPath traversalLog4Shell / Heartbleed / Spring4ShellKali Linux offensive toolingAI Infrastructure & Model OperationsPrompt engineeringPersistent agent memoryRoleplay jailbreak techniquesGuardrail reduction strategiesDefensive Security StrategyDefense in depthEdge network monitoringZeek network analysisPacket capture visibilityIndustry & Threat ImplicationsAI-enabled ransomware operationsAI-assisted red teamingInfrastructure scoping failures Operational scalability challenges#AISecurity #CyberSecurity #Pentesting #AIAgents #RedTeam #EthicalHacking #CyberDefense----------------------------------------------------------------------------------------------(00:00) - Video Intro and Sponsor (01:20) - Al Pentesting Benchmark Overview (02:11) - How AutoPenBench Works (03:44) - Real World Results and Experience (05:16) - Real World Results and Experience (06:48) - Human and Al Collaboration (07:38) - Improving Al Agent Workflows (08:56) - Model Limitations and Updates (10:35) - Jailbreaks and Model Guardrails (13:16) - Provider Controls and Trust Factors (14:41) - Lower Barrier for Cyber Attacks (15:39) - Defensive Security Implications (16:59) - Why Red Teams Need Al NowClick here to watch this episode on YouTube. Creators & Guests Brian Fehrman - HostDerek Banks - HostBrought to you by:Black Hills Information Security https://www.blackhillsinfosec.comAntisyphon Traininghttps://www.antisyphontraining.com/Active Countermeasureshttps://www.activecountermeasures.comWild West Hackin Festhttps://wildwesthackinfest.com🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.
    Show More Show Less
    18 mins
  • AI and Bug Bounties | Episode 51
    May 11 2026

    In this episode of BHIS Presents: AI Security Ops, the team breaks down a growing problem in cybersecurity: AI-generated bug bounty “slop” overwhelming the system.

    What started as a powerful way to crowdsource vulnerability discovery is now hitting a breaking point. Programs like cURL’s bug bounty and platforms like HackerOne are seeing a massive surge in submissions — but fewer and fewer of them are actually valid.

    The result? Security teams spending hours reviewing reports that go nowhere, while real vulnerabilities risk getting buried in the noise.

    We dig into:
    • Why cURL shut down its bug bounty program after years of success
    • How valid reports dropped from 1-in-6 to 1-in-20
    • What “death by a thousand slops” actually looks like in practice
    • How AI is flooding programs with low-quality vulnerability reports
    • The difference between “theoretical” vs. exploitable vulnerabilities
    • Why reviewing findings is now harder than generating them
    • How HackerOne is responding to the surge in submissions
    • Whether AI can be used to filter AI-generated noise
    • The role of reproducibility and proof-of-impact in triage
    • Why human expertise still matters in vulnerability validation

    This episode explores a critical shift in security operations: when vulnerability discovery becomes cheap and automated, validation and triage become the real bottleneck.

    📚 Key Concepts & Topics

    Bug Bounty Programs & Triage
    • Submission quality vs. volume imbalance
    • Signal-to-noise challenges in vulnerability pipelines
    • The growing burden of manual validation

    AI in Vulnerability Discovery
    • Automated scanning vs. real exploitability
    • AI-generated findings and false positives
    • The “editor’s dilemma” — review vs. generation

    AI Security Risks
    • Lower barrier to entry for vulnerability discovery
    • Over-reliance on AI without domain expertise
    • Flooding systems with low-quality submissions

    Defensive Strategy
    • Requiring reproducible steps and proof-of-impact
    • Using AI to pre-filter vulnerability reports
    • Combining human expertise with AI tooling

    Industry Impact
    • cURL bug bounty shutdown
    • HackerOne submission pause
    • Shifting economics of vulnerability research

    #AISecurity #BugBounty #CyberSecurity #LLMSecurity #ArtificialIntelligence #InfoSec #BHIS #AIAgents #AppSec
    ----------------------------------------------------------------------------------------------

    • (00:00) - Intro: Bug Bounty Burnout & AI Noise
    • (01:14) - cURL Kills Its Bug Bounty Program
    • (02:05) - “Death by a Thousand Slops” Explained
    • (03:42) - AI vs Vulnerability Scanners: Signal vs Noise
    • (04:38) - HackerOne Pauses Submissions & Industry Impact
    • (05:41) - Can AI Filter AI? Proposed Solutions
    • (07:49) - Why Humans Still Matter in Validation
    • (12:55) - Final Takeaway: AI as a Tool, Not a Replacement

    Click here to watch this episode on YouTube.

    Creators & Guests
    • Ethan Robish - Guest
    • Bronwen Aker - Host
    • Brian Fehrman - Host
    • Derek Banks - Host

    Brought to you by:

    Black Hills Information Security

    https://www.blackhillsinfosec.com


    Antisyphon Training

    https://www.antisyphontraining.com/


    Active Countermeasures

    https://www.activecountermeasures.com


    Wild West Hackin Fest

    https://wildwesthackinfest.com

    🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits
    https://poweredbybhis.com

    Click here to view the episode transcript.

    Show More Show Less
    14 mins
  • Vercel Breach | Episode 50
    May 1 2026

    In this episode of BHIS Presents: AI Security Ops, the team breaks down the Vercel breach — a real-world incident that shows just how fragile modern security has become in the age of AI integrations and SaaS sprawl.

    What started as a simple Roblox cheat script downloaded on a work laptop quickly escalated into a multi-hop compromise involving OAuth permissions, an AI productivity tool, and access into Vercel’s internal systems.

    This wasn’t a zero-day or advanced nation-state exploit. It was a chain of everyday decisions: installing software, clicking “Allow,” and trusting third-party integrations.

    The result? Allegedly $2M worth of data listed for sale, including API keys, internal data, and employee records — all from a breach path that most organizations aren’t even monitoring.

    We dig into:
    • What Vercel is and why it’s such a high-value target
    • How environment variables become the “keys to the kingdom”
    • The full attack chain: Roblox malware → Context.ai → Vercel
    • What infostealers like Lumma actually do (and how cheap they are)
    • How OAuth permissions become persistent backdoors
    • Why AI productivity tools introduce hidden risk
    • The rise of “shadow AI” inside organizations
    • How supply chain attacks continue to scale across ecosystems
    • The role of AI in accelerating attacker speed and capability
    • Why this type of breach is becoming the new normal

    This episode highlights a critical shift in cybersecurity: you don’t have to get hacked directly anymore — attackers just need to compromise something you’ve already trusted.

    📚 Key Concepts & Topics

    Attack Chain & Initial Access
    • Lumma infostealer and malware-as-a-service
    • Credential theft: passwords, cookies, OAuth tokens
    • Low-cost, high-impact compromise paths

    OAuth & Identity Risk
    • “Allow All” permissions and persistent access
    • OAuth tokens as long-lived entry points
    • Lack of visibility into third-party integrations

    AI Security Risks
    • Shadow AI and unsanctioned tool adoption
    • Deep integrations with Google Workspace and SaaS
    • AI tools as new supply chain attack surfaces

    Supply Chain Attacks
    • Multi-hop compromise paths across vendors
    • Real-world parallels (Trivy, LiteLLM)
    • Interconnected ecosystems increasing blast radius

    Threat Landscape Evolution
    • AI accelerating attacker speed and scale
    • Lower barrier to entry for complex attacks
    • Criminal groups operating as decentralized “businesses”

    Defensive Strategy
    • Auditing OAuth integrations and permissions
    • Enforcing least privilege across SaaS tools
    • Segmenting sensitive data and reducing blast radius
    • Avoiding risky behavior on corporate devices

    ⏱️ Chapters

    • (00:00) - Intro & Breach Overview
    • (00:21) - Sponsors & Show Setup
    • (01:29) - What Vercel Is & Why It Matters
    • (02:31) - Initial Compromise: Roblox Script & Infostealer
    • (05:03) - OAuth Permissions & Pivot into Vercel
    • (08:04) - AI Tools, Over-Permissioning & Supply Chain Risk
    • (09:53) - AI Acceleration of Attacks & Ecosystem Impact
    • (13:34) - Threat Actors, Attribution & Key Takeaways

    Click here to watch this episode on YouTube.

    Creators & Guests
    • Brian Fehrman - Host
    • Ethan Robish - Guest

    Brought to you by:

    Black Hills Information Security

    https://www.blackhillsinfosec.com


    Antisyphon Training

    https://www.antisyphontraining.com/


    Active Countermeasures

    https://www.activecountermeasures.com


    Wild West Hackin Fest

    https://wildwesthackinfest.com

    🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits
    https://poweredbybhis.com

    Click here to view the episode transcript.

    Show More Show Less
    18 mins
adbl_web_anon_alc_button_suppression_c
No reviews yet