How They Know You’re a Bot
Run this and watch it fail:
curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/124.0" https://example.comYou told the server you’re Chrome on Windows. It reads that, glances at the rest of your connection, and blocks you anyway. Nothing about the User-Agent was wrong — it’s just the one part of an HTTP request the client gets to invent, so no serious detection system gives it any weight. It’s a name tag you write yourself in crayon.
The interesting question is what the server looks at instead. The answer is everything below the part you control.
The handshake you didn’t write
Before a single byte of HTTP moves, your client and the server do a TLS handshake, and the very first message — the ClientHello — is a detailed confession. It lists the TLS versions you support, your cipher suites in your preferred order, your extensions, your elliptic curves. None of that is content you chose. It’s whatever your TLS library compiled in.
In 2017 some engineers at Salesforce noticed this was as good as a license plate and built JA3: take those ClientHello fields, concatenate them, hash with MD5, and you have a short string that’s stable across connections and wildly different between a real Chrome and a script using OpenSSL. Chrome ships BoringSSL; curl usually ships OpenSSL; Python’s requests ships something else again. They produce different handshakes for the same reason different car factories stamp different VINs.
So when your curl claims to be Chrome in the User-Agent but presents OpenSSL’s ClientHello, the server isn’t fooled — it’s amused. The application layer says Chrome and the transport layer says curl, and that contradiction is a louder bot signal than an honest curl/8.4.0 would ever be.
The HTTP/2 tells
TLS isn’t the only layer that leaks. HTTP/2 lets a client open the connection with a SETTINGS frame announcing its preferences — header table size, max concurrent streams, initial window size — followed by a WINDOW_UPDATE and, crucially, the order in which it sends the pseudo-headers :method, :authority, :scheme, :path. Akamai documented all of this in a paper on passive HTTP/2 fingerprinting. Chrome sends those four in one order; a Go program sends them in another; the difference is invisible to you and obvious to the server.
Stack the layers and the picture sharpens. A request that’s Chrome in the User-Agent, OpenSSL in the TLS handshake, and Go in the HTTP/2 frames isn’t a browser having an identity crisis. It’s a bot wearing three different costumes at once.
The JavaScript confession
If the connection survives all that and actually runs a browser, the browser tells on itself. The W3C WebDriver spec requires an automated browser to set navigator.webdriver to true — an honesty flag, so pages know they’re being driven by a script. Naturally a whole ecosystem of stealth plugins now exists to patch that flag back to false, which is its own confession: a real Chrome never needs to lie about being a real Chrome. Add the small tells of a headless browser — missing plugins, odd window dimensions, a renderer string that reads SwiftShader instead of a real GPU — and the JavaScript environment is one more fingerprint.
Why the arms race never ends
This is where it gets genuinely hard, and where the honest people and the bot-runners blur together. There’s a tool called curl-impersonate that rebuilds curl against the same TLS library as a real browser, in the same cipher order, with matching HTTP/2 settings, specifically so the fingerprints line up. It works. The whole game is making every layer agree.
The browsers aren’t standing still either, and their moves cut both ways. Chrome 110, in early 2023, started randomizing the order of its TLS extensions on every connection — partly to fight protocol ossification, partly to resist fingerprinting. That broke JA3, whose hash depends on extension order, so the same Chrome now produces a different JA3 every time. The answer was JA4, which sorts the values before hashing so order stops mattering. GREASE (RFC 8701) sprinkles deliberately random values into the handshake for the same anti-ossification reasons, and fingerprinters simply learned to filter them out. Every move spawns a counter, and every counter spawns a counter to that.
What’s actually being measured
Step back and the real signal isn’t any single fingerprint. It’s consistency. A genuine browser is internally coherent all the way down: the User-Agent, the TLS handshake, the HTTP/2 frames, and the JavaScript environment all agree on exactly which browser, which version, which platform. A bot is a stack of layers assembled from different libraries, and the seams show.
Which means the only honest way to not look like a bot is to actually be the thing you claim — same engine, same library, same defaults, top to bottom. You can’t fake your way past it with a header, because the header was never the thing being read. That’s the part worth remembering the next time you reach for -A "Chrome" and wonder why it didn’t work: you edited the one field nobody was looking at.