Get started in 5 minutes!

Why ordering HTTP headers is important

RFC 2616 in decay - picture by Olli Homann @ https://flic.kr/p/RyDVkc

RFC 2616 in decay - picture by Olli Homann @ https://flic.kr/p/RyDVkc

If you code against Akamai hosted sites, you could be rejected because your HTTP library sends request headers in the wrong order. In fact, most libraries use undefined order, as the IETF specification says it doesn’t matter.

In casu:

$ URL=http://www.bulgari.com
$ UA="User-Agent: Mozilla/5.0 My API Client"
$ ACCEPT="Accept: */*"

$ curl -v -H "$UA" -H "$ACCEPT" $URL |& grep '< HTTP'
< HTTP/1.1 403 Forbidden

$ curl -v -H "$ACCEPT" -H "$UA" $URL |& grep '< HTTP'
< HTTP/1.1 302 Moved Temporarily

My guess: they identified that major browsers send HTTP headers in a specific order, and they implemented this trick to fend off spammers.

Update After some more experimenting, it appears that this behaviour depends on order and the Accept header:

$ ACCEPT="Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"

$ curl -v -H "$UA" -H "$ACCEPT" $URL |& grep '< HTTP'
< HTTP/1.1 302 Moved Temporarily

$ curl -v -H "$ACCEPT" -H "$UA" $URL |& grep '< HTTP'
< HTTP/1.1 302 Moved Temporarily

Also, no block without Mozilla/5.0 in the User-Agent.

Conclusion: they will block your request if:

  • Mozilla User-Agent, and
  • */* Accept, and
  • the Accept is sent after the User-Agent header

Update 2 Other sites at Akamai don’t expose this behaviour, so it could be a single site issue and/or a configurable setting.

Prevent Magecart attacks,
protect your web store
with Sansec's eComscan

Since our discovery of MageCart attacks in 2015, we've investigated thousands of hacked Magento web stores. All our research findings are added to our automated malware and vulnerability scanner for Magento and Adobe Commerce.
Merchants using eComscan are protected against the latest malware attacks and all known backdoors. eComscan provides an hourly vulnerability audit on all Magento versions, configurations, and extensions.

Try our malware scanner