Sorting and Merging Unix Password Files

The chances are that your password database is in something of a state of disrepair. In the past people used to maintain the password file (/etc/passwd, /etc/shadow and /etc/group) by hand. That's what the vipw and vigr commands were for: locking the files while they were being edited to make sure that sysadmins didn't overwrite each other's work. And because they were hand edited, people tended to keep the files ordered by increasing user or group id.

These days, though, it's more likely that you'll be using a tool like useradd to put entries in these files, and these tools invariably plonk new entries at the end of the file. Adding and removing packages (with tools like rpm) can also create and delete users and groups, so after a few years of system upgrades you can finish up with files that are anything but sorted. That can be a real problem if you want to make sure two password files on different systems contain the same entries, for example.

Sorting the password and group files is actually really easy, because the Unix sort command is so flexible. The following command will sort the lines of the password file into ascending order by user id:

$ sort -g -t : -k 3 /etc/passwd
and this will do the group file:
$ sort -g -t : -k 3 /etc/group
those with sharp eyes will notice that the commands are the same: the user id and group id are the third fields in both files, and they both use : as a field separator, so the arguments to sort are the same.

Ok, so that sorts the file, but the results just get printed to the terminal. How do we sort the password file in place? Well, you could play around with redirecting the output into a temporary file and then renaming it, but we can return to our old friends vipw and vigr for a slightly tidier solution. Run vipw to load the password file into vi, then type !G to filter the whole of the file through a command. You'll be prompted to enter the command at the bottom of the screen (in vim the prompt looks like :.,$!); enter sort -g -t : -k 3 and press return. The password entries should now be sorted into ascending order. If all seems well save the file and quit the editor. You can then run vigr to do the same transformation on your group file.

What's that? Oh, vipw and vigr are prompting you to edit the shadow password and shadow group files, and they're not in order either? Ah, well that a more tricky problem - neither of these files contains the user id, so we really need to sort them so that the user names (or group names) are in the same order as the names in the password (or group) file, and that's a bit more than we can get sort to do.

There is another way to do all this though: the pwck and grpck commands have a little-known -s option that will sort their respective databases. So another way to get things sorted out would be:

$ pwck -s
$ grpck -s
which is very brief, but we don't find out how to use sort that way...

$Id: passwd-munge.html,v 1.2 2004/07/20 10:09:50 mhw Exp $