I’m an idiot — I’m still confused by requests and httpx

It’s Friday afternoon for me in London. So I’m making time for a little personal therapy session. Today’s topic — Why is it that I still have a mental block when using Python’s requests or httpx?

I keep getting it wrong. Am I just a numpty?

The nicest Python libraries to work with are those with simple and clear APIs. They are intuitive. They don’t require you to write much code. They just do the job with no fuss.

Some of the best are requests, httpx (an async enabled version of requests), and FastAPI. I like them all and use them all a lot.

But I seem to have a chronic problem with requests / httpx when making POST or PUT requests. I keep getting mixed up. It’s clearly my mental block. Perhaps some other people have the same confusion too?

Here is the issue — when making a POST or PUT request to an API we pass the data in the body. We can pass that body data in either data= or json= parameter.

requests.post("https://example.com/endpoint/", data=thing ... or ... json=thing)

Now there are really only four possible options :

1. Pass the thing as a dictionary to the data= parameter
2. Pass the thing as JSON using json.dumps() to the data= parameter
3. Pass the thing as a dictionary to the json= parameter
4. Pass the thing as JSON to the json= parameter

Two of these work. Two of these are wrong.

That’s a 50/50 chance. But I keep getting it wrong! Surely it can’t be that hard?

I’m guessing what happens in my head is that my brain says that data= sounds like something I should pass a dictionary to, and json= sounds like something I should pass JSON to. That sounds logical to me? But it’s wrong.

So I check the documentation. The documentation does exist here but this doesn’t exactly jump off the page .

Or I spin up the Python console and test it out by hand until I find the right one.

Or I just roll the dice, write some code, and wait to see if my tests fall over. Which they usually do.

This really should not be that difficult for me. So rather than wasting 5 minutes each time I mess it up, I have spent the 20 mins now to create a simple Jupyter notebook on GitHub that will remind me what is the right way to do it.

And because humour is more memorable, I thought a little visual cartoon might nudge me in the right direction too …

Thanks for being part of this short therapy session. If you have the same issue, then maybe it is nice to know that you are not alone.

Here is the likely irony of spending the hour to create this post … I probably won’t now have this misunderstanding issue ever again! You can ask me in six months’ time if that’s true.

Enjoy your weekend.