Perl Toolchain Summit 2018 Wrap-up Report

Perl Toolchain Summit 2018 Wrap-up Report

Getting There

This year I had the pleasure of attending the Perl Toolchain Summit in Oslo, Norway. Because of a conflict in my schedule, I initially didn’t think I’d be able to attend. After a lot of mental back and forth, I decided I’d try to make it work. The compromise was that this year I would leave on Sunday morning rather than on Monday. That meant I wasn’t able to participate in the last day of the summit, but I’d still have 3 entire days to get things done.

I left Toronto on Tuesday evening and arrived in Frankfurt on Wednesday morning. I got breakfast in Germany and after a two hour layover I was on a short flight to Oslo. From the Oslo airport I took a train into town.

I got into my tiny, tiny room and changed into my running clothes. I spent the next couple of hours running mostly along the water and making lots of stops along the way to take pictures and hang out. That was enough to burn off any excess energy.

In the evening I met up with everyone else at a bar called RØØR. Here it finally became clear to my why Salve had said alcohol would be expensive. (I hadn’t really believed him when he originally let us know). I had a great time regardless and it was nice to see a lot of familiar faces and catch up.

Day One

On Thursday morning we had an 8:30 breakfast at the venue and then 9:00 AM standup, where everyone had a chance to introduce themselves and talk about what they intended to work on. Leo and Mickey had a claimed a small room for our group, which meant that for the span of the summit we would all be able to sit around the same table and whiteboard. Some folks dropped by early in the morning to talk about a few items on their wish list. We also made a game plan for the summit and made various work assignments amongst ourselves. These sorts of face to face meetings are what make the summit invaluable. The first morning meeting allowed us to get everyone on the same page and then get to work.

This year we didn’t come into the summit with any one grand project to complete. Rather, we spent some time on smaller items, bug fixes, attending to outstanding issues on Github and infrastructure work. Specifically, we’re generally working towards having a better disaster recover plan and failover capacity for MetaCPAN. Leo did a lot of work on that, so I’ll let him blog about his contributions. For my part, I picked back up on the work that I had started at meta::hack this past November.

At meta::hack I had spent a good chunk of time upgrading our Debian from Wheezy to Stretch. This is an upgrade of two major releases and it turns out to be non-trivial in our case. A large part of the problem is that with our current setup we are on a version of Puppet (3.8.7) which is no longer supported. Since Wheezy has a newer Puppet (4.8.2) by default, I took this as an opportunity to upgrade Puppet. This meant that some of our Puppet config needed updating and that we also need to update our PuppetForge modules (3rd party dependencies). This was complicated enough that I was going to need a lot of help from Leo, so I sat myself beside him in order to make bothering him more convenient for the both of us. It turned out that I bothered him a lot. I shudder to think how long it would have taken me to work through this remotely. I can’t say that I enjoyed the work, but it was much more enjoyable to do this face to face and it was a much better use of everyone’s time.

Thursday also happened to be Leo’s birthday so we went out for drinks and various foods made of pork at a bar called Crowbar. Marcus Ramberg and Batman joined us there and eventually a good chunk of the attendees came along as well.

Day Two

I had gotten the bulk of the upgrade done on Thursday. On Friday I got up early and went for a run. Then, at the venue we continued tying up some loose ends on the Debian/Puppet upgrade and getting Leo involved in testing it out. I should add that part of this upgrade also included an overhaul of how we build our Vagrant boxes. Currently if you want to deploy a Vagrant instance you download a pre-built box which we have uploaded to one of our servers. Moving forward you’ll be able to build your own box on demand. It will take slightly longer to get set up, but you’ll have the latest and greatest software right away and you won’t need to rely on any one of us to upload an updated box for you to download. In the evening we all went out to a restaurant where there were speeches, excellent food and great conversations.

Day Three

On Saturday morning, Mickey and I met up at 6:30 AM for a quick run. At 7:30 AM we met up with Leo and Sawyer and rented some bikes. We cycled along the waterfront by the fortress and the Opera House before making it back just in time for standup. By now we were upgrading Debian and deploying the new Puppet on two of the three machines which Bytemark generously donates to us. This turned out to be non-trivial as the first machine we upgraded would not reboot. After some excellent technical support from Bytemark we were able to get past this as well.

I also added a URL mapping for static files to the MetaCPAN API. This is a first step in self-hosting some API documentation, likely using Swagger in some way. This is partly what’s preventing the MacOS Dash application from having more Perl documentation available.

In the evening most of us hung out at a local hacker/maker space. For me, it was a nice way to wrap up the summit.

In General

While all of this was going on, we also had a good debate about sponsorship of our project. We’ve been getting a lot of interest from potential sponsors over the last year or so and we’ve mostly been approaching it on a case by case basis. Having everyone in on a group discussion allowed us to set a sponsor policy moving forward that we think will be good for both us and our sponsors. (If your company would like to sponsor MetaCPAN or our next meta::hack conference, please don’t hesitate to get in touch with me).

I also fit in the usual code reviews over the 3 days and also tried to catch up on some outstanding Github issues. It turns out I had missed some entirely. There were issues going back as far as January which had not yet been responded to. 🙁

As part of some of the other work going on, there was progress made towards a tighter integration of the CPAN river data in the MetaCPAN API. Also, we now have access to Neil Bowers’ CPAN river data generator. There’s a plan to have MetaCPAN directly generate this data, rather than having us pull the data from him.

There was also a fair bit of discussion as to where to host the 2018 meta::hack. Our TODO list is long enough that the Perl Toolchain Summit isn’t quite enough for us to make the progress that we think we need to make. So, hopefully, we’ll be able to get together once more this year to continue to make progress.

Additionally Babs Veloso showed us a proposal for a redesign of the MetaCPAN author pages. It looks great. Now we just need to find someone to implement it. We also spent a fair amount of time on a part of the project which I can’t actually tell you about just yet, mostly because the details haven’t all been finalized yet. If all goes as expected, we’ll announce that in the coming months.

Getting Home

My flight back early on Sunday was uneventful. I spent an hour in Copenhagen and then was back home on Sunday afternoon in time to take the kids to the park. I got into an Eastern Standard Time sleep schedule by Sunday night. I dropped the kids off at school on Monday morning and then flew off to Boston for meetings until the following Friday. Since I telecommute, I rarely need to travel (or even wear shoes), but for some reason I ended up with back to back trips this year. In the end, it all worked out.

Thanks

On the whole this year’s PTS was very productive, fun and very well organized. I’d like to thank Salve J. Nilson, Neil Bowers, Stig Palmquist, Philippe Bruhat, Laurent Boivin and anyone else I may have missed. The venue, food, level of organization (and hoodies) were all excellent. It was a smooth experience from start to finish. I feel like I was able to get a lot of good work done and I am particularly pleased that I was able to finish what I had set out to do. Collectively, there was much more work completed on MetaCPAN than what I’ve described.

I would not have been able to make this trip without the support of MaxMind (my employer). MaxMind has sent me to PTS without hesitation each time that I’ve been invited and has also been a sponsor of the event for these past two years. Thanks, MaxMind!

Additionally, I’d like to thank all of our sponsors, without whom this would not have been possible:

MaxMind,
NUUG Foundation,
Teknologihuset,
Booking.com,
cPanel,
FastMail,
Elastic,
ZipRecruiter,
MongoDB,
SureVoIP,
Campus Explorer,
Bytemark,
Infinity Interactive,
OpusVL,
Eligo,
Perl Services,
Oetiker+Partner.

WWW::Mechanize Best Practices

fields

Recently at $work we were discussing some of the behaviours of WWW::Mechanize when submitting forms. For instance, when you pass the fields parameter to the submit_form() method, Mechanize might take a very lax approach to submitting your data. Imagine the following form:

Now take the following code:

Mechanize will happily post *all* of these fields to the form for you, even though the form doesn’t contain an input with the field “C”. If there is no server side validation which checks for unknown fields, you’ll likely get a 2XX status code in your response and all will appear to be well. This can lead to some confusing and hard to debug situations, especially if you’ve done something as subtle as misspelling an input name. You could be banging your head against the wall for quite some time.

with_fields

You can protect yourself from this scenario via with_fields, which will only submit forms which contain all of the provided fields.

If you try to run this code, Mechanize will die with a message like “There is no form with the requested fields at…“. This is already a big improvement.

(Note that form_id is optional in this case. If you leave it out then Mechanize will only look for a for which contains *all* of the fields provided by with_fields. If we provide form_id then Mechanize will want a form which matches the provided id and which also provides all of the required fields).

strict_forms

Can we do any better than this? You bet. We can supply a strict_forms parameter to submit_form. This switches HTML::Form’s strict behaviour on. From HTML::Form’s docs:

Initialize any form objects with the given strict attribute. If the strict is turned on the methods that change values of the form will croak if you try to set illegal values or modify readonly fields.

That’s above and beyond what with_fields brings to the table. Handy stuff. Try to do this wherever possible if you want to make your life easier.

If strict_forms finds a problem, your code will die with something like “No such field ‘C’ at…“.

Notice that the above example uses fields and not with_fields. I would encourage you to use the above incantation rather than the one below whenever possible. The reason is that with_fields will throw an exception *before* strict_forms. So, in many cases you’ll end up with the less helpful error message — the one which doesn’t name the offending field(s). You may not be able to use with_fields in all cases, so when you do have to use with_fields keep in mind that strict_forms can still find additional issues (like trying to set readonly fields). My advice would be to use strict_forms whenever possible.

autocheck

autocheck can save you the overhead of checking status codes for success. You may outgrow it as your needs get more sophisticated, but it’s a safe option to start with. Consider the following code:

This code doesn’t die or warn, since it assumes you will “do the right thing” by checking status codes etc. Now try this:

If you run the above code you’ll get something like “Error GETing foobar.comcom: URL must be absolute at…“, which can also save you a lot of heartache.

HTTP::CookieJar::LWP

You are encouraged to install Mozilla::PublicSuffix and use HTTP::CookieJar::LWP as your cookie jar. This allows you to take advantage of HTTP::CookieJar which is more modern than HTTP::Cookies and “adheres as closely as possible to the user-agent rules of RFC 6265“.

See also https://metacpan.org/pod/HTTP::Cookies#LIMITATIONS

protocols_allowed

How about restricting the protocols your agent might follow? Let’s look at protocols_allowed:

This option is inherited directly from LWP::UserAgent. It allows you to whitelist the protocols you’re willing to allow.

This will prevent you from inadvertently following URLs like file:///etc/passwd

protocols_forbidden

If you don’t want to whitelist your protocols, you can blacklist them instead. Unsurprisingly, this option is called protocols_forbidden:

This option is also inherited directly from LWP::UserAgent. It allows you to blacklist the protocols you’re unwilling to allow.

This will prevent you from inadvertently following URLs like file:///etc/passwd

Creating a Stricter Agent

If we put together all of these together we get something like:

Using these settings in conjunction with strict_forms can help you with debugging, security and also your own sanity.

Making it Official

Much of what has been discussed here is now documented at https://metacpan.org/pod/WWW::Mechanize#BEST-PRACTICES.

Edit: It has been pointed out to me that readers of this article may also benefit from my previous UserAgent Debugging Made Easy post.

See Also

If WWW::Mechanize does not fit your use case, see also WWW::Mechanize::Chrome, WWW::Mechanize::Firefox, WWW::Mechanize::Cached, LWPx::UserAgent::Cached and Mojo::UserAgent.

My “Go for Perl Hackers” Cheatsheet

Last year I found myself working on some Go code at $work. When I’m trying to pick up constructs in a new language, I find it helpful to see how I would have done the same things in Perl. This sheet is far from complete, but I think it’s already helpful. You can find it at https://github.com/oalders/go-for-perl-hackers. Comments, critique and pull requests are welcome. I’ve already had some helpful feedback via Twitter which I’ve incorporated.

As an aside, I did this a long back for Objective-C.

New defaults for Perl Linting in Vim’s Ale Plugin

In my previous post, I talked about using Ale with vim for linting and syntax checking. Since that time, the Ale defaults for Perl have changed. Perl::Critic checks are still on by default, but you will need to enable the syntax and compile checks that are run via the perl binary itself.

The reasoning for the new default is described in the Ale docs. If you want to (re-)enable your Perl checks, you can follow the example in my dot files.

vim, Ale, Syntastic and Perl::Critic

As a vim user, I’ve used Syntastic for a long time. It’s a great tool for syntax checking. However, I was recently introduced to Ale. Ale does a lot of what Syntastic does, but it does it asynchronously. The practical benefits are

  • You should experience less lag when editing large files
  • Ale flags problematic lines containing errors and warnings in a gutter, making it easy to find problems
  • Detailed information about errors and warnings appear at the bottom of your buffer

I may actually be underselling it. Ale is almost a drop-in replacement for Syntastic. (At least it was for me). Try it out. I don’t think you’ll go back to Syntastic once you’ve tried Ale.

Ale Configuration

let g:ale_perl_perl_options = '-c -Mwarnings -Ilib -It/lib'

This is what I’m currently using. (Note the Ale defaults to -c -Mwarnings -Ilib). Often I’m working with a t/lib directory. Having this included by default means less chance of my code not compiling when it’s run by Ale. Of course, pass in any flags which you may require here. You’ll likely want to keep the -c since you want to compile the code rather than run it. Keep in mind that code in BEGIN blocks will still be executed under the -c flag, so there can be security implications to opening random Perl files with this setting. Explicitly enabling the warnings pragma at the command line will cover you for cases where you haven’t already enabled warnings in your code. Your needs may vary depending upon your environment, so keep in mind that your vimrc can source other files. You can add something like the following to your vimrc:

source ~/.local_vimrc

Perl::Critic Configuration


let g:ale_perl_perlcritic_showrules = 1

When this is enabled, you’ll be shown which Perl::Critic rules which have been violated by your code. This helps you to a) fix the issue or b) copy/paste the rule class name so that you can whitelist the code in question.  For example, you may be told that you’re violating Perl::Critic::Policy::Modules::ProhibitEvilModules.  If you feel you need to embrace the evil, this makes it easy to add a ## no critic (ProhibitEvilModules) to the code in question.   Trust me, this is much easier than digging around to figure out exactly which policy you’ve violated.

Lastly, the default behaviour of Ale’s Perl::Critic linting is to display all violations as errors. I find this confusing, because if something is not preventing my code from compiling, I do not consider this to be an error. In order to set Perl::Critic violations to be displayed as warnings, just add the following to your .vimrc:

let g:ale_type_map = {
\ ‘perlcritic’: {‘ES’: ‘WS’, ‘E’: ‘W’},
\}

The above is a map, so you can add config options for other Ale linters to this map as well.

Unfortunately, the current incarnation of Ale can’t tell the difference between a Perl error and a warning. I have started a pull request to try to improve this somewhat, but I got stuck on the Docker end of things and haven’t gotten back to it yet. If you’d like to pitch in, I wouldn’t mind the help. 🙂

Announcing meta::hack v2

It’s that time of year again. We did a bunch of hacking on MetaCPAN at the Perl Toolchain Summit and we got a lot done, but it wasn’t nearly enough. Our TODO list never gets shorter and there are lots of folks willing to pitch in, so today I’m announcing that meta::hack v2 will take place from Nov 16-19, 2017 at Server Central in Chicago. As a reminder of how things went with meta::hack v1, please refer to my wrap-up report from that event.

The attendees this year are:

This group basically represents the MetaCPAN core contributors. We are restricting meta::hack to this core group again purely so that we can focus on getting the maximum amount of work done over the short time that we have together. If you are interested in joining a future meta::hack, now is a good time to start contributing to the project. Our intention is not exclude willing participants, but to keep the group limited to people who are already up to speed on contributing to the project. Having said that, if any hackers in the Chicago area want to hang out, we’re happy to go out to dinner with you while we are in town.

An event like this does cost money and we are still looking for some more sponsors. Some real estate on the front page of MetaCPAN will be used to recognize sponsors. If you or your company are interested in supporting this event, please contact us so that we can send you a copy of the sponsor prospectus.

Preparing for LWP Hack Night

I’ve had a couple of people ask me how they can prepare for LWP Hack Night, so I thought I’d just give a quick introduction to the set of modules.

I whipped up a graph of the various GitHub repositories to give you an idea of which are the most popular and which have the most open issues. Those stats seem to roughly correspond.

If you want to poke around the repositories on GitHub, that will give you an idea of where you can start.

Now, a lot of these modules have open issues in RT as well, so don’t let the GitHub numbers fool you. You can find the RT bugs for the various libraries here:

If you’ve looked at RT and GitHub, you can see there’s a monster amount of work to be done here, including but not limited to:

  • Establishing which bugs are still bugs
  • Establishing which patches could possibly be applied
  • Adding tests for existing pull requests
  • Rebasing some existing pull requests which have merge conflicts
  • Identifying bugs which are possibly in the wrong queue
  • Performing code review of existing pull requests which look like they are close to a state where they could be merged

How you might like to go about this is entirely up to you. If you have time before the meeting to identify some bugs which you may like to approach or comment on, please feel free to get started now. When we’re at the meeting, we can work out a plan to divide and conquer. There’s more than enough work to go around. We won’t (and can’t) clear this all up in one evening, but the point here is to make incremental improvements and learn something useful in the process.

Please feel free to get in touch with me in advance if you have any questions about this. The best way to do this would be via the Toronto Perl Mongers email list.

Introducing LWP::ConsoleLogger::Everywhere

In an earlier post, I introduced you to LWP::ConsoleLogger. I’ve been using it heavily since then, but one thing I didn’t tackle was how to debug a user agent you can’t easily get it. Some modules don’t provide a public API which allows you to access their user agent. Or, maybe the user agent which you want to debug is so far removed from your code that you can’t easily access its public API. Previously, this was not an easy problem to solve. However, this is no longer the case. simbabque was kind enough to write LWP::ConsoleLogger::Everywhere.

It’s quite simple to use.

Simply add this line to your code and run it. Any objects of the LWP::UserAgent family should now dump extensive logging information to your terminal. It can get a bit fancier than that, but this is really all you need to know in order to get started debugging 3rd party LWP::UserAgent-based HTTP requests.

meta::hack Wrap-up Report

meta::hack v1

Earlier this month (Thu, Nov 16 – Sun, Nov 20) I had the pleasure of meeting up with 7 other Perl hackers at ServerCentral’s downtown Chicago offices, in order to hack on MetaCPAN. Before I get started, I’d like to thank our sponsors.

This hackathon wouldn’t have been possible without the overwhelming support of our sponsors. Our platinum sponsors were Booking.com and cPanel. Our gold sponsors were Elastic, FastMail, and Perl Careers. Our silver sponsors were ActiveState, Perl Services, ServerCentral and Advance Systems. Our bronze sponsors were Vienna.pm, Easyname, and the Enlightened Perl Organisation (EPO). Please take a moment to thank them for helping our Perl community.

For the past 2.5 years, we’ve been working off and on at porting MetaCPAN from Elasticsearch 0.20.2 to 1.x and (eventually) 2.x. There were enough breaking changes between the versions to make this a non-trivial task. We had made very good progress over the past two QA hackathons, but the job was just too big to finish in the hours that we had available.

After the QA Hackathon in Rugby, I spoke to Neil Bowers about how we might go about doing some fundraising. Neil was so kind as to offer to help. His offer to help soon evolved into him taking on all of the work (thanks Neil)! Neil worked his magic and got the event fully funded. I know there was a lot of work invovled, but he made it look easy. Mark Keating and the Enlightened Perl Organization kindly took on the financial side of things, invoicing and accepting payment from sponsors. Without EPO and Neil, this event never would have taken place. (Please do take a moment to thank them).

While this was going on, we began searching for a venue. Joel Berger offered to host us at ServerCentral in Chicago and we immediately took him up on the offer. After that it was just a matter of folks booking plane tickets and getting approval from employers for the time off.

The final list of invitees was:

  • Brad Lhotsky (San Francisco)
  • Doug Bell (Chicago)
  • Graham Knop (Baltimore)
  • Joel Berger (Chicago)
  • Leo Lapworth (London)
  • Mickey Nasriachi (Amsterdam)
  • Olaf Alders (Toronto)
  • Thomas Sibley (Seattle)

The event was invitation only. We did this in order to maximize the amount of work we’d be able to finish at the event. [Insert reference to “The Mythical Man Month”]. Everyone who participated was already up to speed on the internals of the project or has an area of expertise which we needed in order to complete our goal of launching fully with v1 of the API. Because everyone already had a working VM and working knowledge of the project, we were able to tackle the problems at hand right from the first morning.

As far as living space goes, we initially had looked at renting hotel rooms, but the cost would have made it almost prohibitive to meet in Chicago. After doing some research, we booked two apartments (each with 3 bedrooms) on the same floor of a condo building in the Lincoln Park area of Chicago. We booked the accommodations via booking.com of course. 🙂 I think we were happy with the housing. Everyone had their own room and we had big enough living rooms for all of us to meet up mornings and some evenings. At the end of the day the rental was a fraction of the price of a Chicago hotel. I’ve also made a mental note not to be the last one to arrive in town. Apparently it also means you get the smallest room.

Each day we took the subway downtown to ServerCentral. We had a dedicated boardroom in the office with a large TV that we could use for sharing presentations, IRC chat or error logs. ServerCentral also sponsored lunch each day of the event. Extra monitors were also available for those who wanted them. (Lots of Roost laptop stands were to be seen. Also lots of people who couldn’t figure out how to open them after having collapsed them for the first time in forever).

After settling in at the office we’d discuss our plans for the day and map out goals for that day. We had breakout discussions where appropriate but the time spent not writing code was minimal. Generally, as a group, we worked well into the evenings. We didn’t get the full Chicago experience, but we got a lot done. We did make it to the Chicago Christkindlmarkt, which was a few blocks from the office and we went out for a breakfast and a dinner as well. Minimal downtime, but the breaks we had were lots of fun.

Day one was spent removing anything which was blocking the API upgrade. Wishlist items were ignored and as a group we worked really well. Lots of pull requests were created, reviewed and merged.

By day two of the hackathon we flipped the switch and went live with the new API. We could have waited a bit longer, but we opted to make the change earlier so that we could troubleshoot any issues as a group and watch the error logs in real time. There were no showstopping bugs and the transition was actually pretty smooth.

Day three was spent squashing some of the bugs which came up after the upgrade. We also started to tackle some wishlist items.

Day four was a slightly shorter day. We wrapped around 4 PM. Some of us went to check out “the Bean” before flying out while Leo and I headed right for our respective airports.

This list is by no means exhaustive, but over this long weekend we:

  • moved ++ data to v1 of the API.
  • moved https://metacpan.org to v1 of the API.
  • implemented load balancing via Fastly, our CDN sponsor.
  • reduced noise in the logs by squashing bugs which generated warnings or exceptions.
  • updated our API documentation as well as the metacpan-examples GitHub repository from v0 to v1.
  • published an upgrade document which explains to how upgrade your query syntax and configuration for v1.
  • moved http://explorer.metacpan.org to v1 of the API.
  • began work on streaming logs to Elasticsearch.
  • began moving the query logic that metacpan.org uses over to the API so that other clients can use this same logic.
  • began porting author queries from metacpan.org to the API as well.
  • added a meta::hack event page along with sponsor info to metacpan.org.
  • continued work on adding a /permission endpoint which will provide access to the data in 06perms.txt.
  • added more tests for the /download_url endpoint which translates module names into download URL. Specifically this is meant to be used by cpanm.
  • added snapshotting of Elasticsearch indices in v1 so that we can easily restore from backup.

/permission is something I spent a fair bit of my time working on over the last two days. Having 06perms.txt data in the API will mean that we can display a list of all authors who have maint on a module on metacpan.org. This will make it easier to track down authors who can release a module, particularly for those who aren’t familiar with the way PAUSE works. I think this branch is probably about 1.5 years old, so I was happy to get the time to try to finish it off. I didn’t quite get there, but that’s okay. It was a wishlist item and it’s actually quite close to being released.

Also of note is the fact that we’ve now officially deprecated the v0 API. There is a 6 month runway to move clients over to v1 and v0 will be taken offline on or after June 1, 2017.

Since https://metacpan.org now uses v1 of the API, results for v0 are no longer available. If you have a client which uses v0 of the API, please feel free to reach out to us with any concerns you may have about making the switch.

If you rely on updated ++ data, you’ll need to switch to v1 now, as ++ data in v0 is no longer being updated. The indexer is, however, still running on v0, so it will still find and index new CPAN uploads. v0 development is officially closed. Any v0 bugs (barring catastrophic issues) will likely not be addressed. v0 has been around for just over 6 years now. It has served us well, but it’s time to let it go. [Insert musical scene with a talking snowman, an ice queen and her loyal sister.]

UserAgent Debugging Made Easy

Earlier today I saw a recent blog post from Gabor Szabo. In it, he shows a very concise way to handle Basic Authentication using LWP::UserAgent. Now, what if you had a problem running the script? How might you go about debugging it? You could add a bunch of print statements. Maybe dump the request and the response objects. That’s entirely valid, but I want to show you a slightly simpler way of going about it, using LWP::ConsoleLogger::Easy.

Gabor’s original script looks like this:

Let’s run it to see what the output looks like.

Here’s the debugging version. Note the important changes are on lines 4 and 9.

The output we get is:

You can see that the debugging version is just one line longer. I added 2 lines and removed a print statement. It prints out a whole pile of (nicely?) formatted information. Let’s try running it with valid credentials. (Brace yourself, there’s going to be a lot of output.)

You can see that I ran the script with LWPCL_REDACT_HEADERS='Authorization'. That’s a handy flag to use if you want to copy/paste an example when asking for help publicly. It replaced the Authorization header value with [REDACTED]. That’s maybe not a big deal here, but there are cases where it’s more important. See also LWP_REDACT_PARAMS.

Let’s make it prettier. We’ll do this by installing HTML::FormatText::Lynx.

Let’s run it again. I’ll only show you the changed part. Instead of just displaying the text with the HTML stripped away, we get something nicer to look at.

Now, we can also turn down the verbosity of the script by passing a flag to debug_ua(). Any integer from 0-8 will do the trick. Let’s try 6.

Let’s see what we get:

That’s far easier to read now.

This is just a very basic example of what you can do with LWP::ConsoleLogger::Easy. There’s a lot more you can do with it and it’s all laid out for you in the documentation. It really shines when you have a user agent which is going through multiple links or if you’re debugging someone else’s API calls. Have fun with it. It beats inserting arbitrary print statements and it could save you from pulling a lot of your own hair out someday.