The official diagrams are not always very explicit (at least for me).
Especially about where Azure ExpressRoute starts…
In fact, ExpressRoute starts at the Inernet Service Provider level, not at the customer level.
And here is a self-explicit picture.
And if you want to compare the network path with Site-to-Site VPN or Peer-to-Site VPN, here is another picture.