Debugging HTTP::Tiny Calls Deep Inside Your Stack
LWP::ConsoleLogger is one of the little libraries that I am most proud of. First, because I think it’s clever pats self on back but second because I have found it to be incredibly useful over many years of working with libwww-perl based UserAgents like LWP::UserAgent and WWW::Mechanize.
I initially wrote about it in UserAgent Debugging Made Easy and then later I talked about an even cleverer module called LWP::ConsoleLogger::Everywhere which allows you to debug UserAgents you don’t even know about.
In the intervening years Mojo::UserAgent support was added and now we also have HTTP::Tiny.
"Pimp My Desktop 10" by joergermeister is licensed under CC BY 2.0.
#The magic
You can either add UserAgent debugging to your code directly or, through the
magic of perl’s -M you can invoke it on the command line without touching a
single line of code. Let’s look at a very basic example to get you started.
We’ll make a request to the MetaCPAN API, using
MetaCPAN::Client, which defaults
to an HTTP::Tiny UserAgent.
$ perl -MMetaCPAN::Client \
-E 'say MetaCPAN::Client->new->distribution("LWP-ConsoleLogger")->metacpan_url'
https://metacpan.org/release/LWP-ConsoleLogger
That prints the distribution’s URL. We’ve established that it works. Now let’s add the debugging:
$ perl -MMetaCPAN::Client \
-MLWP::ConsoleLogger::Everywhere \
-E 'say MetaCPAN::Client->new->distribution("LWP-ConsoleLogger")->metacpan_url'
With -MLWP::ConsoleLogger::Everywhere we have now invoked the magic and we
get some insights into what the UserAgent is actually doing, all without
knowing (or caring) in which packages the UserAgent calls are actually taking
place.
GET https://fastapi.metacpan.org/distribution/LWP-ConsoleLogger
+---------------------------------+---------------------------+
| Request (before sending) Header | Value |
+---------------------------------+---------------------------+
| Host | fastapi.metacpan.org |
| User-Agent | MetaCPAN::Client/2.043000 |
+---------------------------------+---------------------------+
+--------------------------------+---------------------------+
| Request (after sending) Header | Value |
+--------------------------------+---------------------------+
| Host | fastapi.metacpan.org |
| User-Agent | MetaCPAN::Client/2.043000 |
+--------------------------------+---------------------------+
==> 200 OK
+-----------------+------------------------------------------------------------+
| Response Header | Value |
+-----------------+------------------------------------------------------------+
| Accept-Ranges | bytes |
| Cache-Control | private |
| Connection | keep-alive |
| Content-Length | 417 |
| Content-Type | application/json; charset=utf-8 |
| Date | Sat, 19 Sep 2026 02:29:26 GMT |
| Server | nginx |
| Vary | Accept-Encoding |
| Via | 1.1 varnish, 1.1 varnish, 1.1 varnish |
| X-Cache | MISS, MISS, MISS |
| X-Cache-Hits | 0, 0, 0 |
| | |
| X-Served-By | cache-lhr-egll1980058-LHR, cache-lhr-egll1980058-LHR, cach |
| | e-lhr-egll1980058-LHR, cache-fra-etou8220106-FRA |
| | |
| X-Timer | S1789784967.661973,VS0,VE72 |
+-----------------+------------------------------------------------------------+
+--------------------------------------------------------------------+
| Content |
+--------------------------------------------------------------------+
| { |
| "bugs" : { |
| "github" : { |
| "active" : 0, |
| "closed" : 35, |
| "open" : 0, |
| "source" : "https://github.com/oalders/lwp-consolelogger" |
| } |
| }, |
| "name" : "LWP-ConsoleLogger", |
| "repo" : { |
| "github" : { |
| "stars" : 6, |
| "watchers" : 2 |
| } |
| }, |
| "river" : { |
| "bucket" : 1, |
| "bus_factor" : 5, |
| "immediate" : 7, |
| "total" : 7 |
| } |
| } |
+--------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| Text |
+------------------------------------------------------------------------------+
| Printing in line 482 of /home/olaf/.plenv/versions/5.42.2/lib/perl5/site_per |
| l/5.42.2/LWP/ConsoleLogger.pm: |
| { |
| bugs => { |
| github => { |
| active => 0, |
| closed => 35, |
| open => 0, |
| source => 'https://github.com/oalders/lwp-consolelogger', |
| }, |
| }, |
| name => 'LWP-ConsoleLogger', |
| repo => { |
| github => { |
| stars => 6, |
| watchers => 2, |
| }, |
| }, |
| river => { |
| bucket => 1, |
| bus_factor => 5, |
| immediate => 7, |
| total => 7, |
| }, |
| } |
+------------------------------------------------------------------------------+
Note the Printing in line 482 of …LWP/ConsoleLogger.pm: line above. That’s a side effect of me having
the following in my Data::Printer config.
Put something like this into your ~/.dataprinter file:
caller_info = 1
That’s an implementation detail that is leaking through to the terminal, but I really like it because it gives me just a little more context as to what is going on under the hood. I don’t know how many other languages have this kind of tooling, but I do miss it when debugging UserAgents in Golang. I tried to do something even more convoluted for Playwright. That one even spits out images in formats that are friendly to being viewed in your terminal. That was always going to be a niche use case that it is hard to explain to someone who isn’t me, but I had a lot of fun building it as well.
Related posts: