Opened 2 weeks ago

Closed 10 days ago

#1781 closed defect (fixed)

http-client: Always include a path component when sending a proxy request

Reported by: sjamaan Owned by: sjamaan
Priority: major Milestone: someday
Component: unknown Version: 5.2.0
Keywords: Cc:
Estimated difficulty: easy

Description

See this issue from mitmproxy. While debugging an issue I had to locally patch my mitmproxy to be able to get it to work...

Change History (4)

comment:1 Changed 2 weeks ago by Vasilij Schneidermann

I suspect the same issue keeps me from using Burpsuite unless I enable "invisible proxying". This happens only for https:// URLs though, not http:// ones.

edit: Nevermind, this is its invisible proxying tolerating clients that don't implement CONNECT, so it's a separate issue.

Last edited 2 weeks ago by Vasilij Schneidermann (previous) (diff)

comment:2 Changed 2 weeks ago by sjamaan

Should be fixed with f3efa21fb, will make a release later after verifying that this fixes the situation under mitmproxy.

My interpretation of the RFCs is that the old version of mitmproxy was incorrect, but it's still fine to update the path to have a trailing slash, because it would be added by a well-behaving proxy anyway, so there should be no observable difference in that case.

comment:3 Changed 2 weeks ago by sjamaan

Nope, that didn't fix it because uri-common has a hateful behaviour where it tries to "normalize" the path in a way that doesn't actually match the real situation.

This is not ideal for this particular piece of code, but it is helpful when trying to manipulate paths in a sane way. And since uri-common makes the promise/guarantee never to mutate the input unless you explicitly do so yourself, it can't really avoid doing this at the readout level (note that when you write an empty path it will normalize it too, so then it will have the right value). So while I'm not 100% sure we should keep it, it's probably still fine.

Anyway, I've worked around this by also checking for the empty faked out root path in 4c32441.

There's still some behaviour I want to investigate before making a new release - it looks like the client is disconnecting when it is receiving a 404. A quick study of the http-client code seems to indicate we're not doing it there, so maybe it's mitmproxy (or the upstream server even?) which is disconnecting.

comment:4 Changed 10 days ago by sjamaan

Resolution: fixed
Status: assignedclosed

Fixed in 1.2.1

Note: See TracTickets for help on using tickets.