I build AI systems that follow rules, adapt, self-correct, show their work, converge, never guess and self-improve. follow rules.

The Fermi paradox is premised on expansionism being intrinsic to civilizational development, most explicitly in Hanson's grabby aliens model. There's a Great Filter provided by expansionism itself that doesn't center on scientific and technological progress that I think is worth exploring. It's about the destabilizing nature of expansion events and the precarity of an expansionist disposition, and survives the standard counter that civilizational natural selection maximizes expansionism in the limit. It's a cosmic-scale cousin of habryka's Do not conquer what you cannot defend, where the concern is defending what's taken. Mine is about integrating it.

Questioning expansionism

Before and unrelated to that, a brief skepticism about the premise feels due and will be useful context. I don’t really believe that expansionism is a necessary civilizational posture, in particular in the long-arc limit relevant to Fermi time scales, and might be a rare/never phenomenon.

The argument for it seems to be from intuition, positing that selection pressure made organisms expansionist in the sense of always consuming what resource is available to support population growth. That tendency inheres genetically, hence something something cultures and civilizations must be like that on balance/in great number and also in the limit.

Recent memory certainly aligns. Technological progress has been regularly opening new avenues for resource acquisition and human civilization acts on them.

I don’t buy it. Cultures throughout history have not been uniformly expansionist and the rapid and steady influx of low hanging tech progress acts as confounder to our intuitions, which are based on recent times. We’ve seen that pattern for a long time, therefore it’s always like that.

But, counter to that, an awareness of the externalities of expansionist tendencies is already normative — the collective has metabolized that wielding technology to absorb newly accessible resource can have substrate-undermining externalities. This is a recent trend too and my reasoning here comes from intuition also (hence has that same recency confound), but I think this latter intuition has better foundations.

Externality awareness is not a naked observation about sociocultural trends. It fits into developmental models, which more or less all posit that a growing sphere of concern, a receptivity and saliencing of context and externalities, is constitutive of development. This observation is thus structural in kind, matching models of the development of other intelligent systems, whereas the linear intuition that “replicators expand therefore all the higher order structures constituted of them will tend to do the same” ignores that structural regularity. Also and to a lesser degree, that linear intuition is undermined/countered by noting that civilization comes to persist exactly by civilizing older base energies and drives. Take what you can for yourself, the atom of expansionism, is about the most primitive force at the individual level that coordination supportive of civilization suppresses. So I don’t think these two intuitions are on equal footing. A growing enacted concern for externalities has a precedented and principled significance.

Assuming expansionism

“Is your WAF ready for frontier AI models?” We keep hearing this question from our customers, so we decided to find out.

When it comes to exploiting applications, what LLMs are really good at is iterating and mutating attack payloads faster than any human hacker could do. LLMs can use real-time responses to iterate and change their techniques by, for example, testing different encodings, sending the payload in a different part of the HTTP request, or moving to the next vulnerability to test.

Even before LLMs were around, security engineers used two common approaches to test applications: static and dynamic application security testing. The former analyzes code without executing it to identify vulnerabilities, while the latter probes running applications to find runtime flaws. There are plenty of works scanning code with frontier AI models, including details on how to build your own harness.

For the project described in this blog post, we took a dynamic approach: making the LLM act as if it was a hacker to evaluate whether a WAF is doing its job. The LLM had no visibility into source code, no view of the WAF's rules, and could only see selected HTTP response data.

We built a WAF tester that starts from known exploits and then iterates by changing how it is encoded or delivered, sends it again, and uses the response to choose the next variation. A request that was not blocked became a lead for human review, not a confirmed exploit.

We ran the tester against an authorized customer staging environment across six attack categories and recorded 1,107 attempts. After reviewing the non-blocked requests and removing malformed, benign, duplicate, and out-of-scope observations, the vast majority of the attacks were blocked by the Cloudflare WAF. The requests that got through helped us create new detections to harden our security to benefit all Cloudflare customers.

Here we will explain how we set up the system, the types of attacks we tested, which attack vectors bypassed the WAF more easily, and how we fixed it. Most importantly, we share what we learned from this process and how this exercise is becoming a foundational building block of our WAF development lifecycle.

Finally, we offer guidance to help you correctly deploy your WAF in front of your application and, most importantly, patch your software. A payload that bypasses the WAF still needs an exploitable application to succeed, so keeping your stack up-to-date remains one of the strongest defenses against attackers.

How the adaptive loop works

To test our WAF with frontier models, we built a system that iterates over multiple scenarios. A scenario means choosing one attack category, placing the input in a specific part of the request, starting with a version the WAF already blocked, and giving the tester a fixed number of attempts to try other variations. The loop runs LLM models twice: the first is the proposal call, the second is the review call.

The first call receives the starting request, the context, a short history of earlier results, and suggests the next variation, then the code builds and sends the request. The review call receives the request context, response status, selected headers, and the response body. The loop stops when mutations stop producing useful variations or when a hard coded attempt limit has been reached.

Anthropic has formally warned investors that its technology may pose “existential risks to humanity” in a long-awaited initial public offering prospectus circulated with a small group of partners in recent days.

The nearly $1 trillion AI start-up led by Dario Amodei devoted almost a third of its lengthy S-1 filing to detailing “risk factors,” including the potential of increasingly advanced AI models to manipulate, blackmail, and exhibit other unpredictable behaviors.

Anthropic outlined more prosaic risks, including the extreme concentration of its customer base, with close to a quarter of revenue last year coming from just two clients, according to people familiar with the filing.

Read full article

Comments

In July, AI agents testing new cybersecurity models compromised parts of OpenAI’s infrastructure and Hugging Face’s production environment.

We've all just witnessed one of the first AI-driven successful cyber attacks. When given a task, the agents ignored existing guardrails and autonomously discovered previously unknown vulnerabilities, recovered exposed credentials, moved between cloud environments and coordinated their work through communication channels they created themselves.

The speed of the final compromise was incredible. In under 13 hours, the agents went from executing code on a Hugging Face worker to gaining admin-level access across multiple clusters. But the incident had been brewing for much longer. Responders found clues of activity tracing back to May (agents created an unauthorized message board), to June (internal network scanning) and early July. The relationship between these events was understood only on July 20.

The lesson here is not that AI agents exploit vulnerabilities. That’s not news; human attackers already do that. The change is that agents can work persistently, test multiple paths simultaneously, share discoveries, and chain vulnerabilities, credentials, and permissions into sophisticated attacks.

The incident also shows why application security cannot depend on single tools. For example, network restrictions were bypassed by services connected to the Internet; valid credentials were used to perform unauthorized actions. Rebuilding Artifactory removed one attack path, but agents found another. The key insight is that individual alerts identified pieces of the activity without revealing the complete campaign. OpenAI reached a similar conclusion in its report: organizations need overlapping and independent controls across prevention, detection, and mitigation, continuous validation of security boundaries, and faster mechanisms to correlate and contain suspicious behavior.

We address this challenge by connecting application security across four activities that are too often separated: discovering which risks matter, governing what humans and agents may do, protecting applications at runtime, and turning every investigation into stronger protection. Cloudflare can deliver this framework because of its broad security portfolio and visibility across a vast share of Internet traffic.

Alongside the framework, we connect existing Cloudflare solutions with new capabilities across each stage. These include: using Large Language Models (LLMs) to conduct a penetration test of our Web Application Firewall (WAF), expanding threat intelligence to all customers, and a new feature to automate deploying positive security.

What has changed

The security landscape is shifting. These are the emerging trends we see:

• The way we build software has fundamentally changed. AI-assisted development allows engineers to produce and deploy software faster outside traditional engineering workflows. That speed creates both more code and more opportunities for vulnerabilities to reach production.
• Software composition risk is still a risk: applications depend on large chains of open-source libraries, packages, and operating-system components that are intrinsically trusted and are difficult for any team to inspect. What’s new is that AI is now importing libraries that we might not be aware of.
• Techniques and tactics are changing. LLMs can chain vulnerabilities and use feedback in real time to mutate payloads, evade defenses, and make decisions autonomously. They can operate continuously and at machine speed. Patching faster remains important, but patching alone cannot close the gap. Attackers are always going to be faster than you can update your systems.
• Agentic traffic. In the past, automation was a synonym for malicious activity. Today, a request generated by an agent or bot may be malicious automation, a search crawler, or an agent purchasing a product on behalf of a customer.
• Compromised servers, residential proxies, IoT devices, and cloud resources allow attacks to move quickly across infrastructure and identities. A coordinated attack can leverage a number of devices, making it difficult to be identified as a unique campaign.

Before we start: given this article is about native mobile development, I want to offer my 2021 ebook, ‘Building Mobile Apps at Scale: 39 engineering challenges’, for free to all readers. (normally costs $20). The book remains relevant on the challenges to solve for large-scale mobile applications, and lists the technologies covered below in this article, Kotlin Multiplatform included.

Claim your free copy here

This offer is valid until Friday, 2 October. On checkout, simply select “The ebook: PDF & EPUB versions of the book.” No credit card required! Feel free to share the link with colleagues who build mobile apps, or work on them.

Recently, Shopify dropped the bombshell announcement that Native is now the future of mobile development at the e-commerce platform. This triggered a lot of reaction in the mobile development community, on the scale of the stir Shopify generated when it originally moved over to React Native six years ago.

But times have changed, and we all know the agent of change this time: AI agents! This latest generation of models has improved in coding capability to the extent that Shopify no longer is satisfied with React Native (RN), as it was only last year.

This article looks into what led here, why Shopify made the change, and also the broader mobile ecosystem. We cover:

1. Why Shopify chose React Native in 2020. Android apps took too long to build, first and foremost, and it was nice to have consistent iOS and Android apps.

2. Still happy five years later. Shopify moved all six apps over to RN and most things were going well, and the 2020 move was seen as a good move.

3. Why ditch React Native now? AI is now very good at writing mobile code as well as backend code, while React Native adds abstractions that native does not. Shopify rewrote the Shop app to native in just 12 weeks(!!). Additional details from the Shopify mobile team.

4. Haven’t we seen this back-and-forth before? Airbnb did something similar by adopting React Native in 2016 before moving back to native just two years later. The underlying reason was the same: performance.

5. Going native since 2019. Moving to native used to take a long time, and Notion has been at it for seven years. When Notion finishes the process, a fair question will be whether a native editor actually slows down their web iteration speed, with or without AI.

6. Kotlin Multiplatform (KMP) would like a word. KMP allows the writing of business logic in Kotlin and sharing it across iOS and Android while each platform stays native. It’s a technology that feels relevant, and could become more popular with AI.

7. What’s next? It’s easier to write native iOS and Android apps than before, but that’s also true for React, Flutter, and KMP apps.

1. Why Shopify chose React Native in 2020

Let’s rewind to 2020, when Shopify decided to go all-in on React Native (RN). Back then, mobile was a very important channel for the platform, with seven out of 10 customers using it on mobile devices. As Head of Engineering, Farhan Thawar wrote at the time:

“Each quarter, the majority of buyers purchase on mobile (with 71% of our buyers purchasing on mobile in Q3 of last year [in 2019]). Black Friday and Cyber Monday (together, BFCM) are the busiest time of year for our merchants, and buying activity during those days is a bellwether. During this year’s BFCM, Shopify merchants saw another 3% increase in purchases on mobile, an average of 69% of sales.”