Getting Started with perlimports

A version of this post was previously published in the 2023 Perl Advent Calendar. I have significantly revised it for publication here.

What’s in a Package, Anyway?

#

Perl, like many other languages, exposes the functionality to export functions (and other things) from one package into another. The functions that are exported by package A are the same functions that are imported into package B. Consider the following code:

package WhereAmI;

use 5.38.0;

use Git::Helpers qw( checkout_root );

say checkout_root(); # or WhereAmI::checkout_root()

In this example, Git::Helpers is exporting the checkout_root function into WhereAmI. Conversely, WhereAmI is importing the checkout_root function from Git::Helpers. Importing and exporting are two sides of the same coin. The important thing to note is we now have a new function available in the WhereAmI package.

Today we’ll mostly be referring to imports, but in many cases we could just as well be framing our discussion in terms of exports instead.

What is perlimports?

#

I have been interested in code quality for many years. This is something I touched on in Find and Fix More Typos and Finding Unused Perl Variables. As part of that interest, I wrote a tool to help automate cleaning up cases where code is being imported. This tool is called perlimports. perlimports is a tidier, which means it can rewrite our code for us. It’s also a linter, so we can use it to report on problems without rewriting our code. With this power comes great responsibility. perlimports tries to be responsible.

Inspired by goimports, perlimports is an opinionated tool which tries to force its opinions onto our code. What we get in return is a tool which takes much of the burden of managing imports out of our hands.

If you’re interested in a more thorough discussion of how perlimports and Perl’s use and require work, I spoke about this in depth at The Perl Conference in 2021: perlimports or “Where did that symbol come from?”.

Why Tidy Imports at all?

#

You may quite correctly be asking yourself if perlimports is worth the bother. Why even bother? After having used this tool in anger for about 3 years now, I’ll try to sum up the best points.

Consistent Style

#

Just like any other tidier, using perlimports will automate and enforce a consistent style for your imports. This has the following advantages:

Code is Easier to Read

#

When your imports are laid out in a consistent manner, your code can become easier to scan. You’ll have consistent spacing, the imports for a given module will be alpha-sorted and you’ll have a standard quoting style. This should reduce the cognitive load when you’re looking at the first few lines of your files.

Diffs Become Easier to Read

#

Once code layout is consistent, code changes can become easier to read, since you’re less likely to be confronted with formatting changes. This can also reduce the size of your diffs, which also reduces cognitive load.

Code Reviews are Easier

#

Hopefully code reviewers will no longer have to comment on how someone used the wrong quotes or how they imported something they’re not actually using. If your linting and tidying is automated, code contributors can now also stop worrying about these little things.

Writing Code is Less Onerous

#

Once you have a tidier to do the work for you, you can be a fair bit sloppier when writing code. Write it quickly, get it out of your system and then tidy the file and watch the magic happen. Not having to worry about the mundane frees up mental energy for other things.

Improved Dependency Management

#

Once you start automatically removing from your code the imports (and even the modules) which you are not using, it should be easier for you to evaluate which dependencies you actually need. You may find that you’ve been installing modules which you don’t actually use. Fewer dependencies leads to a better security posture (since you’ve reduced your attack surface), faster setup time (because you now have fewer things to install) and fewer headaches around not being to install this or that module in a certain environment. Dependencies are great when you need them, but limiting your dependencies can also have some real benefits. Unfortunately, linting Perl’s imports is not always as easy as we’d like. Let’s try to contextualize the problem by taking a quick look at the tools module authors might use to export code into yours.

There’s More Than One Way to Do It

#

If you’ll allow a small digression, Perl’s import() is an incredibly powerful tool that does little in the way of imposing restrictions on the implementor when it is run. This means that over the years CPAN authors have allowed themselves to be very creative when it comes to what may or may not get imported into our code.

This liberal approach to code import has also allowed for the proliferation of modules which authors can use to manage what their code exports into ours.

• https://metacpan.org/pod/Exporter
• https://metacpan.org/pod/Sub::Exporter
• https://metacpan.org/pod/Exporter::Tiny
• https://metacpan.org/pod/Export::Declare
• https://metacpan.org/pod/Exporter::Auto
• https://metacpan.org/pod/Exporter::Easy
• https://metacpan.org/pod/Exporter::Lite

This is not an exhaustive list, but it’s also the kind of TIMTOWTDI that might leave a Perl newbie scratching their head in confusion and wonder.

The upshot of all of these different methods of exporting is that a static code analysis is not an effective way of discovering what is being imported into your code. perlimports needs to eval the package imports and then (in many cases) see what actually changed in the symbol table. This means two things:

• We need to install dependencies and be able to eval them in order to see what’s going on
• A speed penalty is imposed on perlimports for this approach

Strategy

#

As we saw above, there is an embarrassment of riches when it comes to modules to manage code exports. Some of these exporters work in very different ways, which makes the job of perlimports tricky. If we accept that it can be difficult to predict what some code is actually importing, we will probably want to be very careful when rewriting imports. The more “important” the code is, the more careful we’ll want to be. perlimports makes a best effort, but it simply cannot be correct in every case.

I find that a good approach is to start with lower impact code and move on from there, once we’ve established that we’re on a good path. For example, we could first run perlimports as a linter, just to see what it might change. Or we could get some code which is under version control and run perlimports as a tidier on it and just diff the code to see what has changed.

I would probably start by applying perltidy to the test suite (assuming it exists) and then running the tests to see what the impact is. From there I might move to some code which has good test coverage. Eventually I would make my over to other code which is lacking in coverage and/or is harder to unit test. Working from lower to higher impact code allows for a gentle introduction of our new linting and tidying tool. We’ll explore this approach in more detail below, after we cover installation.

Installation

#

Let’s take a look at a typical getting started workflow. First we’ll install the package from CPAN. I like to use https://metacpan.org/pod/App::cpm, but feel free to use your CPAN installer of choice:

  cpm install -g App::perlimports

(If you’re using https://github.com/tokuhirom/plenv to manage your Perl installations, don’t forget to run plenv rehash after cpm install.)

Once that is done, we are ready to try perlimports. The quick way to run it on a new repository might look something like this:

Getting Started

#

• Clone a repo
• Install *all* of the repository’s dependencies (including recommended)
• Run the test suite
• Ensure all of the tests are passing. If there are test failures, fix those first
• Run perlimports with the --lint flag to see what changes it might make
• Tweak the configuration until we’re happy with the linting results
• Apply perlimports to the tests. We can do this via perlimports -i t
• Ensure all of the tests are still passing
• Commit the changes
• Move on to other parts of the code, e.g.: perlimports -i lib

In a best case scenario, this “just works”. Let’s try it on a really old repository of mine.

$ git clone https://github.com/oalders/acme-odometer.git
$ cd acme-odometer/
$ cpm install -g --with-recommends --cpanfile cpanfile
$ yath t
$ perlimports --lint t

I’m not including the command output, but I ran this locally and all of the steps “just worked”. I ran the test(s) via yath and they passed. Why did I use yath rather than make test or prove? I certainly could have done either of those things, but I’m trying to get in the habit of using more modern tools. yath comes bundled with Test2::Harness. If you’d like to learn more about some of the features which Test2 provides, please see Santa’s Workshop Secrets: The Magical Test2 Suite (Part 1) and Santa’s Workshop Secrets: The Magical Test2 Suite (Part 2).

Now, let’s see what the linting looks like:

$ perlimports --lint t
❌ Test::Most (import arguments need tidying) at t/load.t line 1
@@ -1 +1 @@
-use Test::Most;
+use Test::Most import => [ qw( done_testing ok ) ];

❌ Acme::Odometer (import arguments need tidying) at t/load.t line 3
@@ -3 +3 @@
-use Acme::Odometer;
+use Acme::Odometer ();

❌ Path::Class (import arguments need tidying) at t/load.t line 4
@@ -4 +4 @@
-use Path::Class qw(file);
+use Path::Class qw( file );

We can see three suggestions have been made. In the first suggestion, perlimports has detected that done_testing and ok are the only functions exported by Test::Most which the test is using, so it has made this explicit.

In the second suggestion perlimports has detected that the test is not importing any symbols from Acme::Odometer, so it has made this explicit by adding the empty round parens following the use statement.

In the third suggestion we see that some whitespace padding has been added to the Path::Class import.

If we don’t like these changes, we can tweak the configuration. To tell perlimports to ignore Test::Most, we can change our incantation:

perlimports --lint --ignore-modules Test::Most t

If we also don’t like the additional padding, we can turn that off:

perlimports --lint --ignore-modules Test::Most --no-padding t

Applying these settings we now get:

$ perlimports --lint --ignore-modules Test::Most --no-padding t
❌ Acme::Odometer (import arguments need tidying) at t/load.t line 3
@@ -3 +3 @@
-use Acme::Odometer;
+use Acme::Odometer ();

It’s time to update the actual file. We’ll use -i for an inplace edit:

perlimports -i --ignore-modules Test::Most --no-padding t

The result is:

git --no-pager diff t
diff --git a/t/load.t b/t/load.t
index 503d560..d19688f 100644
--- a/t/load.t
+++ b/t/load.t
@@ -1,6 +1,6 @@
 use Test::Most;

-use Acme::Odometer;
+use Acme::Odometer ();
 use Path::Class qw(file);

 my $path = file( 'assets', 'odometer' )->stringify;

Are the tests still passing?

yath t

** Defaulting to the 'test' command **

( PASSED ) job 1 t/load.t

                                Yath Result Summary
-----------------------------------------------------------------------------------
     File Count: 1
Assertion Count: 3
      Wall Time: 1.00 seconds
       CPU Time: 1.42 seconds (usr: 0.32s | sys: 0.09s | cusr: 0.77s | csys: 0.24s)
      CPU Usage: 142%
    --> Result: PASSED <--

Excellent. Let’s add the changes via git and commit them. After that, let’s turn to the lib directory.

$ perlimports --lint --ignore-modules Test::Most --no-padding lib
❌ namespace::clean (appears to be unused and should be removed) at lib/Acme/Odometer.pm line 9
@@ -9 +8,0 @@
-use namespace::clean;

❌ GD (import arguments need tidying) at lib/Acme/Odometer.pm line 11
@@ -11 +11 @@
-use GD;
+use GD ();

❌ Memoize (import arguments need tidying) at lib/Acme/Odometer.pm line 12
@@ -12 +12 @@
-use Memoize;
+use Memoize qw(memoize);

❌ Path::Class (import arguments need tidying) at lib/Acme/Odometer.pm line 14
@@ -14 +14 @@
-use Path::Class qw( file );
+use Path::Class qw(file);

Now, we already see some issues. First off, perlimports doesn’t seem to know about namespace::clean. That’s ok. We can ignore it.

perlimports --lint --ignore-modules namespace::clean,Test::Most \
  --no-padding lib

As an aside, we could also update the code to use namespace::autoclean while we’re poking around, but we’re trying to make minimal changes in this first iteration.

The last suggestion is to remove the padding from the Path::Class imports. It’s good to be consistent. The second and third suggestions look to be solid. Let’s make this change.

perlimports -i --ignore-modules namespace::clean,Test::Most --no-padding lib

That gives us:

$ git --no-pager diff lib
diff --git a/lib/Acme/Odometer.pm b/lib/Acme/Odometer.pm
index 7fee773..cb1734e 100644
--- a/lib/Acme/Odometer.pm
+++ b/lib/Acme/Odometer.pm
@@ -8,10 +8,10 @@ package Acme::Odometer;
 use Moo 1.001;
 use namespace::clean;

-use GD;
-use Memoize;
+use GD ();
+use Memoize qw(memoize);
 use MooX::Types::MooseLike::Numeric qw(PositiveInt PositiveOrZeroInt);
-use Path::Class qw( file );
+use Path::Class qw(file);

That’s pretty good. Do the tests still pass? Yes, they do. So, we can commit this change as well. Not every introduction of perlimports will be this easy, but it’s nice to start off with a win. Can we improve our experience? I think so. Paring down our use of the command line switches would be a good start. We can do that via a config file.

A Configuration File

#

Aside from just running perlimports with fewer switches at the command line, what if we wanted to run perlimports via the Perl Navigator Language Server? It would be better if we didn’t have to worry about the custom command line switches there as well. This sounds like a good time to create a config file.

perlimports --create-config-file perlimports.toml

Nice! We have a stub configuration file. Let’s see what’s inside perlimports.toml

# Valid log levels are:
# debug, info, notice, warning, error, critical, alert, emergency
# critical, alert and emergency are not currently used.
#
# Please use boolean values in this config file. Negated options (--no-*) are
# not permitted here. Explicitly set options to true or false.
#
# Some of these values deviate from the regular perlimports defaults. In
# particular, you're encouraged to leave preserve_duplicates and
# preserve_unused disabled.

cache = false # setting this to true is currently discouraged
ignore_modules = []
ignore_modules_filename = ""
ignore_modules_pattern = "" # regex like "^(Foo|Foo::Bar)"
ignore_modules_pattern_filename = ""
libs = ["lib", "t/lib"]
log_filename = ""
log_level = "warn"
never_export_modules = []
never_export_modules_filename = ""
padding = true
preserve_duplicates = false
preserve_unused = false
tidy_whitespace = true

Let’s commit the stub file to git and then let’s move our command line switches to the config file. The diff should look something like this:

$ git --no-pager diff perlimports.toml
diff --git a/perlimports.toml b/perlimports.toml
index d631998..1e54c9e 100644
--- a/perlimports.toml
+++ b/perlimports.toml
@@ -10,7 +10,7 @@
 # preserve_unused disabled.

 cache = false # setting this to true is currently discouraged
-ignore_modules = []
+ignore_modules = ["namespace::clean", "Test::Most"]
 ignore_modules_filename = ""
 ignore_modules_pattern = "" # regex like "^(Foo|Foo::Bar)"
 ignore_modules_pattern_filename = ""
@@ -19,7 +19,7 @@ log_filename = ""
 log_level = "warn"
 never_export_modules = []
 never_export_modules_filename = ""
-padding = true
+padding = false
 preserve_duplicates = false
 preserve_unused = false
 tidy_whitespace = true

We can iterate on this config file as we start tidying more of our code, but this is already an excellent start. Now we’re ready to start setting up editor integrations.

Neovim

#

nvim-lint

#

Users of Neovim can enable perlimports linting via https://github.com/mfussenegger/nvim-lint

Language Server Protocol (LSP)

#

Perl Navigator

#

If you’re using an editor with LSP support (like Neovim or VS Code), you can hopefully get perlimports running via the Perl Navigator Language Server. This language server has perlimports disabled by default, so we’ll need to switch it on in the configuration and also make sure that we have the perlimports binary installed and in our $PATH.

{
    perlimportsProfile = 'perlimports.toml',
    perlimportsLintEnabled = true,
    perlimportsTidyEnabled = true,
}

In addition to editor configuration, we can now think about adding perlimports to our Continuous Integration and pre-commit hooks as well, so that we maintain the changes we’ve just imposed. We could do this via precious.

The Power of precious

#

I talked about using Code::TidyAll in How Santa’s Elves Keep their Workshop Tidy. tidyall is a wonderful tool that solves a lot of problems, but its design was not perfect and it’s looking for a new maintainer. In the meantime, precious has drawn inspiration from tidyall and can be regarded as its spiritual successor, even if it’s written in Rust. If we want to run perlimports along with other fixing and linting tools, we can use precious for this.

We won’t cover installation here, but after installing precious we can generate a stub config file:

$ precious config init --component perl

Writing precious.toml

The generated precious.toml requires the following tools to be installed:
  https://metacpan.org/dist/Perl-Critic
  https://metacpan.org/dist/Perl-Tidy
  https://metacpan.org/dist/App-perlimports
  https://metacpan.org/dist/Pod-Checker
  https://metacpan.org/dist/Pod-Tidy

Let’s have a look at the created file:

excludes = [
    ".build/**",
    "blib/**",
]

[commands.perlcritic]
type = "lint"
include = [ "**/*.{pl,pm,t,psgi}" ]
cmd = [ "perlcritic", "--profile=$PRECIOUS_ROOT/perlcriticrc" ]
ok_exit_codes = 0
lint_failure_exit_codes = 2

[commands.perltidy]
type = "both"
include = [ "**/*.{pl,pm,t,psgi}" ]
cmd = [ "perltidy", "--profile=$PRECIOUS_ROOT/perltidyrc" ]
lint_flags = [ "--assert-tidy", "--no-standard-output", "--outfile=/dev/null" ]
tidy_flags = [ "--backup-and-modify-in-place", "--backup-file-extension=/" ]
ok_exit_codes = 0
lint_failure_exit_codes = 2
ignore_stderr = "Begin Error Output Stream"

[commands.perlimports]
type = "both"
include = [ "**/*.{pl,pm,t,psgi}" ]
cmd = [ "perlimports" ]
lint_flags = ["--lint" ]
tidy_flags = ["-i" ]
ok_exit_codes = 0
expect_stderr = true

[commands.podchecker]
type = "lint"
include = [ "**/*.{pl,pm,pod}" ]
cmd = [ "podchecker", "--warnings", "--warnings" ]
ok_exit_codes = [ 0, 2 ]
lint_failure_exit_codes = 1
ignore_stderr = [
    ".+ pod syntax OK",
    ".+ does not contain any pod commands",
]

[commands.podtidy]
type = "tidy"
include = [ "**/*.{pl,pm,pod}" ]
cmd = [ "podtidy", "--columns", "80", "--inplace", "--nobackup" ]
ok_exit_codes = 0
lint_failure_exit_codes = 1

We can see that the config already includes a linting and tidying configuration for lots of helpful Perl linters and tidiers, including perlimports. Now, we can run precious tidy --all or precious lint --all to run all sorts of checks in pre-commit hooks and other places where code quality needs to be ensured.

precious is a powerful tool and it merits its own blog post, but let’s leave this as a quick introduction. Perhaps you’ll feel inspired to try it out.

That’s a Wrap

#

I currently have some capacity in my schedule for client work. If you or your team need help with Perl dependency management and/or integrating perlimports, precious and other code quality tools into your environment, I’m available for hire.

We’ve covered a lot of ground here today. If you have any comments or suggestions for improvements, please do reach out to me. I’d love to hear from you.