Here is a failure that never gets filed as a bug, because nobody ever sees it.
A team turns on IPv6. They add AAAA records, flip on dual-stack at the load balancer, and ship it. Somewhere in the path — a firewall rule that drops inbound IPv6, a router with no return route, a security group that only ever got IPv4 rules — the v6 path is a black hole. Packets go in and nothing comes back. And yet: no tickets. No slow-page complaints. Dashboards green. The IPv6 rollout is declared a success and everyone moves on.
Then one day someone writes a small backend service — a health checker, a cron job, a script that hits the same hostname with a bare socket — and it hangs for thirty seconds every single time before it works. Now there’s a bug. Except the bug isn’t in the new service. The bug was there all along, in production, for months. The new service is just the first client naive enough to tell you about it.
The thing that hid it is called Happy Eyeballs, and it is one of the most quietly consequential algorithms your browser runs.
The problem it was built to solve
When a hostname has both an A record (IPv4) and a AAAA record (IPv6), the client has to choose which one to connect to. For years the rule, per the address-selection logic in RFC 6724, was simple: prefer IPv6. Try it first.
Which is exactly right when IPv6 works, and a catastrophe when it doesn’t. If the v6 path is broken, “try IPv6 first” means opening a TCP connection to an address that will never answer, then waiting for that connection to time out before falling back to IPv4. TCP connection timeouts are measured in tens of seconds. So the user experience of a subtly broken IPv6 network in, say, 2011 was not an error page. It was: the whole internet is mysteriously, intermittently slow. Some sites hang for twenty seconds, then load fine. Nobody could explain it, because from the user’s chair it looked like flaky Wi-Fi, not a routing bug two hops away.
That’s a miserable thing to debug and an impossible thing to ask users to tolerate, and it was a real drag on IPv6 deployment — every operator who turned on v6 risked making their dual-stack users’ experience worse, so many just didn’t.
Don’t wait. Race.
The fix, first written down as Happy Eyeballs in RFC 6555 (2012, by Dan Wing and Andrew Yourtchenko), is almost insultingly simple in hindsight: don’t wait for the IPv6 connection to fail. Start an IPv4 connection shortly after, in parallel, and use whichever one completes first. Cancel the loser.
RFC 8305 (2017) turned that idea into a real specification — Happy Eyeballs version 2, which obsoletes the original — and the details are where the craft lives.
You fire off both DNS queries, AAAA and A, at the same time. But you don’t block waiting for the AAAA answer: the RFC recommends a Resolution Delay of 50 milliseconds, after which, if the A record is already back and the AAAA isn’t, you start connecting on IPv4 rather than stalling. You sort the resulting addresses so the families interleave — IPv6, then IPv4, then IPv6 — with a deliberate bias toward trying one IPv6 address first. Then you start connecting to the first address and set a timer. That timer, the Connection Attempt Delay, defaults to 250 milliseconds (the spec pins it to a minimum of 100ms and a maximum of 2 seconds). If the first attempt hasn’t completed in that window, you don’t wait around — you fire the next address, then the next, staggered by the same delay, all racing at once. First handshake to finish wins; the rest get torn down.
The 250ms number is the whole trick, and it’s a genuinely good piece of design. Make it too long and a broken v6 path costs the user real, perceptible time before v4 takes over. Make it too short and you’d hammer every server with a needless duplicate IPv4 connection even when IPv6 was about to succeed, throwing away the efficiency of preferring v6. A quarter second is long enough that healthy IPv6 almost always wins the race outright — so you still get the v6-first behavior you wanted — and short enough that when v6 is a black hole, the user pays 250 milliseconds instead of thirty seconds. They never notice. That’s the point.
The catch nobody puts on the slide
Happy Eyeballs is excellent engineering. It is also, if you care about the actual health of the IPv6 internet, a quiet disaster — for exactly the reason it’s excellent.
It works so well that it removed the pressure to fix the underlying disease. Before Happy Eyeballs, broken IPv6 was loud: it made your site slow, users complained, and you had a reason to go find the dropped route. After Happy Eyeballs, broken IPv6 is silent. Every mainstream browser papers over it in 250 milliseconds. Your users are happy. Your metrics are fine. And your IPv6 path can be completely, totally broken — dropping every packet — while you never hear a word about it.
Multiply that across the industry and you get the situation we’re actually in: an unknown but real fraction of the AAAA records on the internet point at paths that don’t work, maintained by people who have no idea, because the clients are engineered to succeed anyway. The algorithm designed to make IPv6 adoption painless also made IPv6 breakage undetectable from the one vantage point that matters — the client that’s supposed to use it. Measurement projects that probe this (APNIC has published ongoing analysis of how inconsistently Happy Eyeballs is even implemented across clients) keep finding that the real-world behavior is far messier than the RFC’s clean pseudocode.
And here’s where it bites you specifically. The safety net is not universal. Browsers implement Happy Eyeballs, and implement it reasonably. Almost nothing else does it as well. Plenty of language HTTP clients, database drivers, command-line tools, and hand-rolled connect() calls either don’t implement it at all or implement a broken half of it. So the code with no happy eyeballs — the backend microservice, the monitoring probe, the migration script, the thing running in a cron slot at 3 a.m. — is precisely the code that inherits the full thirty-second hang your users were shielded from. You will debug it as an application bug, or a DNS bug, or a “the network is flaky” non-answer, and you will lose an afternoon, because the tool that would have shown you the truth is the browser, and the browser lied to you by working.
What to actually do
The lesson generalizes past IPv6: a fallback that works too well hides the thing it’s falling back from, and hidden failures are the expensive kind. Resilience and observability pull against each other, and Happy Eyeballs is the cleanest example of the trade you’ll find — it bought resilience by spending your ability to notice.
So if you run dual-stack, don’t trust that green dashboard. Test the IPv6 path directly and on its own, forcing v6 with no v4 escape hatch, from a network that actually has working IPv6 — because that’s the only way to see what your users’ browsers are hiding from you out of politeness. If a client hangs and then recovers, stop suspecting the application and go look at whether one address family is a black hole. And know that the version numbers are still moving: there’s an active Happy Eyeballs v3 draft in the IETF refining the resolution and address-selection details, which tells you the “solved” problem was never quite as solved as the smooth 250-millisecond connection made it look.
The connection feels instant. That good feeling is the algorithm doing its job. It is not evidence that anything underneath it is actually working.