Frequently Asked Questions About Fetchmail

Before reporting any bug, please read G1 for advice on how to include 
diagnostic information that will get your bug fixed as quickly as possible.

If you have a question you think needs to be added to this list, mail it
to esr@thyrsus.com.

General questions:

G1.  I think I've found a bug.  Will you fix it?
G2.  I have this idea for a neat feature.  Will you add it?

Build-time problems:

B1. My C compiler libraries don't seem to have atexit().

Configuration questions:

C1. Why do I need a .fetchmailrc when running as root on my own machine?
C2. Why does my .fetchmailrc from 2.8 or earlier no longer work?
C3. How can I arrange for a fetchmail daemon to get killed when I log out?
C4. How do I know what interface and address to use with --interface?
C5. How can I get fetchmail to work with ssh?
C6. The .fetchmailrc parser won't accept my all-numeric user name.
C7. How can I set up support for sendmail's anti-spam 571 response? 

Runtime problems (all installations):

R1. I think I've set up fetchmail correctly, but I'm not getting any mail.
R2. Fetchmail isn't working, and -v shows `SMTP connect failed' messages.
R3. Why is fetched mail being logged with my name, not the real From address?
R4. Spurious blank lines are appearing in the headers of fetched mail.

Runtime problems (multidrop only):

M1. I tried to run a mailing list using multidrop, and I have a mail loop!
M2. My multidrop fetchmail seems to be having DNS problems.
M3. I'm seeing long DNS delays before each message is processed.

-----------------------------------------------------------------------
G1.  I think I've found a bug.  Will you fix it?

Yes I will, provided you include enough diagnostic information for me
to go on.  When reporting bugs, please include the following:

1. Your operating system and compiler version.
2. The release and patch level of the fetchmail you are running.  You can see
   your patchlevel by typing `fetchmail -V'.
3. The output of fetchmail -V (this will not reveal your password).
4. Any command-line options you used.

It is helpful if you include your .fetchmailrc, but not necessary
unless your symptom seems to involve an error in configuration parsing.

A transcript of the failed session with -v on is almost always useful.
If the bug involves a core dump or hang, a gdb stack trace is good to have.
(Bear in mind that you can attach gdb to a running but hung process by
giving the process ID as a second argument.)

Best of all is a mail file which, when fetched, will reproduce the bug.

-----------------------------------------------------------------------
G2.  I have this idea for a neat feature.  Will you add it?

Probably not.  Most of the feature suggestions I get are for ways to
set various kinds of administrative policy or add more spam filtering
(the most common one, which I seem to get about four million times a week
and am *really* tired of, is for tin-like kill files).

You can do spam filtering better with procmail or mailagent on the server
side and (if you're the server sysadmin) sendmail.cf domain exclusions.
You can do other policy things better with the MDA option and script
wrappers around fetchmail.  If it's a prime-time-vs.-non-prime-time issue,
ask yourself whether a wrapper script called from crontab would do the job.

I'm not going to do these; fetchmail's job is transport, not policy, and I
refuse to change it from doing one thing well to attempting two things badly.
One of my objectives is to keep fetchmail simple so it stays reliable.

All that said, if you have a feature idea that really is about a transport
problem that can't be handled anywhere but fetchmail, lay it on me.  I'm
very accommodating about good ideas.

-----------------------------------------------------------------------
B1. My C compiler libraries don't seem to have atexit().

Your compiler libraries are deficient (this has been reported from a
bunch of older Solaris and SCO boxes).  The atexit(3) function is part of
the ANSI C standard and should be there.

You may be able to find a linkable object for atexit(3) in your C++ 
library.

If you can't, atexit(3) source is easily obtained.  You need both
atexit(3) and exit(3), since the latter provides necessary support for
the former.  Glenn E. Thobe <thobe@lafn.org> tells us he found source
code for both in directory /cdrom/usr/src/lib/libc/stdlib on his
FreeBSD disc (BSD 4.4-lite).

-----------------------------------------------------------------------
C1. Why do I need a .fetchmailrc when running as root on my own machine?

Ian T. Zimmerman <itz@rahul.net> asked:

On the machine where I'm the only real user, I run fetchmail as root
from a cron job, like this:

 fetchmail -u "itz" -p POP3 -s bolero.rahul.net

This used to work as is (with no .fetchmailrc file in root's home
directory) with the last version I had (1.7 or 1.8, I don't
remember).  But with 2.0, it RECPs all mail to the local root user,
unless I create a .fetchmailrc in root's home directory containing:

 skip bolero.rahul.net proto POP3
        user itz is itz

It won't work if the second line is just "user itz".  This is silly.  

It seems fetchmail decides to RECP the `default local user' (ie. the
uid running fetchmail) unless there are local aliases, and the
`default' aliases (itz->itz) don't count.  They should.

Answer:

No they shouldn't.   I thought about this for a while, and I don't much
like the conclusion I reached, but it's unavoidable.  The problem is
that fetchmail has no way to know, in general, that a local user `itz'
actually exists.

"Ah!" you say, "Why doesn't it check the password file to see if the remote
name matches a local one?"  Well, there are two reasons.

One: it's not always possible.  Suppose you have an SMTP host declared
that's not the machine fetchmail is running on?  You lose.

Two: How do you know server itz and SMTP-host itz are the same person?
They might not be, and fetchmail shouldn't assume they are unless
local-itz can explicitly produce credentials to prove it (that is, the
server-itz password in local-itz's .fetchmailrc file.).

Once you start running down possible failure modes and thinking about
ways to tinker with the mapping rules, you'll quickly find that all the
alternatives to the present default are worse or unacceptably
more complicated or both.

-----------------------------------------------------------------------
C2. Why does my .fetchmailrc from 2.8 or earlier no longer work?

The `interface', `monitor' and `batchlimit' options have changed.

They used to be global options with `set' syntax like the batchlimit
and logfile options.  Now they're per-server options, like `protocol'.

If you had something like

	set interface = "sl0/10.0.2.15"

in your .fetchmailrc file, simply delete that line and insert 
`interface sl0/10.0.2.15' in the server options part of your `defaults'
declaration.

Do similarly for any `monitor' or `batchlimit' options.

-----------------------------------------------------------------------
C3. How can I arrange for a fetchmail daemon to get killed when I log out?

Fetchmail versions before 2.3 actually used SIGHUP as a wakeup signal.
Newer versions use SIGUSR1 for wakeup and ignore SIGHUP entirely in
order to avoid any potenntial confusion about logout-time behavior.
The right way to dispatch fetchmail on logout is to arrange for the
command `fetchmail -q' to be called on logout.

Under bash, you can arrange this by putting `fetchmail -q' in the file
`~/.bash_logout'.  Most csh variants execute `~/.logout' on logout.
For other shells, consult your shell manual page.

-----------------------------------------------------------------------
C4. How do I know what interface and address to use with --interface?

This depends a lot on your local networking configuration (and right
now you can't use it at all except under Linux).  However, here are
some important rules of thumb that can help.  If they don't work, ask
your local sysop or your Internet provider.

First, you may not need to use --interface at all.  If your machine
only ever does SLIP or PPP to one provider, it's almost certainly by a
point to point modem connection to your provider's local subnet that's
pretty secure against snooping (unless someone can tap your phone or
the provider's local subnet!).  Under these circumstances, specifying
an interface address is fairly pointless.

What the option is really for is sites that use more than one
provider.  Under these circumstances, typically one of your provider
IP addresses is your mailserver (reachable fairly securely via the
modem and provider's subnet) but the others might ship your packets
(including your password) over unknown portions of the general
Internet that could be vulnerable to snooping.  What you'll use
--interface for is to make sure your password only goes over the 
one secure links.

To determine the device:

1. If you're using a SLIP link, the correct device is probably sl0.

2. If you're using a PPP link, the correct device is probably ppp0.  

3. If you're using a direct connection over a local network such as
   an ethernet, use the command `netstat -r' to look at your routing table. 
   Try to match your mailserver name to a destination entry; if you don't
   see it in the first column, use the `default' entry.  The device name
   will be in the rightmost column.

To determine the address and netmask:

4. If you're talking to slirp, the correct address is probably 10.0.2.15,
   with no netmask specified.  (It's possible to configure slirp to present
   other addresses, but that's the default.)

5. If you have a static IP address, run `ifconfig <device>', where <device>
   is whichever one you've determined.  Use the IP address given after
   "inet addr:".  That is the IP address for your end of the link, and is
   what you need.  You won't need to specify a netmask.

6. If you have a dynamic IP address, your connection IP will vary randomly
   over some given range (that is, some number of the least significant bits
   change from connection to connection).  You need to declare an address with
   the variable bits zero and a complementary netmask that sets the range.

To illustrate the rule for dynamic IP addresses, let's suppose you're
hooked up via SLIP and your IP provider tells you that the dynamic
address pool is 255 addresses ranging from 205.164.136.1 to
205.164.136.255.  Then

	interface "sl0/205.164.136.0/255.255.255.0"

would work.  To range over any value of the last two octets
(65536 addresses) you would use

	interface "sl0/205.164.0.0/255.255.0.0"

-----------------------------------------------------------------------
C5. How can I get fetchmail to work with ssh?

This is a lightly edited version of a recipe from Masafumi NAKANE.

1. You must have ssh (the ssh client) on the local host and sshd (ssh
server) on the remote mail server.  And, you have to configure ssh so
you can login to the sshd server host without a password.  (Refer to ssh
man page for several authentication methods.)

2. Add something like following to your .fetchmailrc file: 

poll localhost port 1234 with pop3:
        preconnect "ssh -f -L 1234:mailhost:110 mailhost sleep 20 </dev/null >/dev/null";

(Note that 1234 can be an arbitrary port number.  Privileged ports can
be specified only by root.)  The effect of this ssh command is to
forward connections made to localhost port 1234 (in above example) to
mailhost's 110.

This configuration will enable secure mail transfer.  All the
conversation between fetchmail and remote pop server will be
encrypted.

If sshd is not running on the remote mail server, you can specify
intermediate host running it.  If you do this, however, communication
between the machine running sshd and the POP server will not be encrypted.
And the preconnect line would be like this:

preconnect "ssh -f -L 1234:mailhost:110 sshdhost sleep 20 </dev/null >/dev/null"

You can work this trick with IMAP too, but the port number 110 in the
above would need to become 143.

-----------------------------------------------------------------------
C6. The .fetchmailrc parser won't accept my all-numeric user name.

So put string quotes around it. :-)

-----------------------------------------------------------------------
C7. How can I set up support for sendmail's anti-spam 571 response? 

Rachel Polanskis <r.polanskis@nepean.uws.edu.au> writes:

Basically you need to use the "check_*" rules in sendmail.
These are rules introduced since version 8.8.2

The idea is to generate a list of domains and addresses that are placed into 
a file - I call mine "sendmail.rej" and you place just one domain 
or email address on each line.   During the SMTP transaction, this file
is checked and if there is a match, the message is refused, with
a suitable "Service not available" message sent back to the sender.

With the feature enabled in fetchmail, the mail is simply deleted, 
with no further processing.

The only drawback when blocking spam with fetchmail is that you 
do not get the satisfaction of sending an error back to the sender.

To actually use the check_mail rules in sendmail 8.8.2 or better, 
you need to know how to generate a sendmail.cf file from the m4 
config files distributed with sendmail.

The actual rules can be found at the following URLS:

http://www.informatik.uni-kiel.de/%7Eca/email/check.html

By Claus Assman, who has documented more of sendmail then I can digest!

The actual setup I used though was by David Begley, who has put together 
a WWW page describing how to quickly implement these rules yourself.

http://www.nepean.uws.edu.au/users/david/pe/blockmail.html

David's pages could be moving shortly.  I will post an update if it happens.

Remember, when copying these rulesets off the web, that there are tabs 
embedded in them, that may not be preserved.  You *must* reintroduce
these tabs into the rules to make them work properly.  

Once you have your ruleset in place, and have generated a nice sendmail.cf
file, and the list of blocked sites,  try telneting to your
SMTP port to test it, and send a message with a blocked address in it.

You should see a message similar to:

			"571 unsolicited email is refused"

Next, if you have access to a host that you can send mail from, that is *not* 
your mail host, add that host to your spamlist and restart sendmail.

Send a message to your mailing address from that host and then pop off
the message with fetchmail, using the -v argument.  You can monitor
the SMTP transaction, and when the FROM address is parsed, if sendmail
sees that it is an address in spamlist, fetchmail will flush and
delete it.

Under no circumstances put your *mailhost* or any host you accept 
mail from using fetchmail into your reject file.   You *will* lose mail!!!

The check_ rules work, and they work well. Coupled with fetchmail's
ability to respond to the appropriate error messages, you can be assured
of never seeing a spam from any address you put in the reject list.

The only thing that is missing, as mentioned previously, is the ability
to allow sendmail to process the message further and generate an error
message to the sender.  

-----------------------------------------------------------------------
R1. I think I've set up fetchmail correctly, but I'm not getting any mail.

Maybe you have a .forward set up that you've forgotten about.  You
should probably remove it.

Or maybe you're trying to run fetchmail in multidrop mode as root
without a .fetchmailrc file.  This doesn't do what you think it
should; see question C1.

Or you may not be connecting to the SMTP listener.   Run fetchmail -v
and see the next question.

-----------------------------------------------------------------------
R2. Fetchmail isn't working, and -v shows `SMTP connect failed' messages.

Fetchmail is working, but your SMTP port 25 listener is down or inaccessible.
The first thing to check is if you can telnet to port 25 and get a greeting
line from the listener.  If the listener is down, bring it back up.

If the listener seems to be up when you test with telnet, it could
have had a momentary problem due to resource exhaustion (process table
full or some other problem that stopped the listener process from
forking).  If your SMTP host is not `localhost' or something else in
/etc/hosts, the glitch could also have been caused by transient
nameserver failure, or the SMTP host's actually being down.

If the listener tests up, you can usually ignore the glitch (except as
a symptom of other problems) because a future fetchmail run will get
the mail through.  If this is a recurring or constant failure mode,
OTOH, you may have more serious problems in your network layer which I
can't diagnose in this FAQ.

One way to work around chronic SMTP connect problems is to use --mda.
But this only attacks the symptom.  You should really try to figure
out what's going on underneath before it bites some other way. 

We have one report from a Linux user of 2.1 who solved his SMTP
connection problem by removing the reference to -lresolv from his link
line and relinking.  Apparently in some recent Linux distributions the
libc bind library version works better.

As of 2.2, the configure script has been hacked so the bind library is
linked only if it is actually needed.  So under Linux it won't be, and
this particular cause should go away.

-----------------------------------------------------------------------
R3. Why is fetched mail being logged with my name, not the real From address?

Because logging is done based on the address indicated by SMTP MAIL FROM,
and some listeners are picky about that address.

Some SMTP listeners get upset if you try to hand them a MAIL FROM
address naming a different host than the originating site for your
connection.  This is a feature, not a bug -- it's supposed to help
prevent people from forging mail with a bogus origin site.

Since the originating site of a fetchmail delivery connection is
localhost, this effectively means these picky listeners will barf on
any MAIL FROM address fetchmail hands them with an @ in it!

In versions up to 1.9.9 this led to pesky errors at some sites.
Because of this, I hacked 2.0 to just use the calling user ID
as the MAIL FROM address.

Versions 2.1 and up try the header From address first and fall back to the 
calling-user ID.  So if your server isn't picky, the log will look right.

-----------------------------------------------------------------------
R4. Spurious blank lines are appearing in the headers of fetched mail.

What's probably happening is that the POP/IMAP daemon on your
mailserver is inserting a non-RFC822 header (like X-POP3-Rcpt:) and
something in your delivery path (most likely an old version of the
`deliver' program, which sendmail often calls to do local delivery) is
failing to recognize it as a header.

This is not fetchmail's problem.  The first thing to try is installing a
current version of deliver.  If this doesn't work, try to figure out
which other program in your mail path is inserting the blank line and
replace that.  If you can't do either of these things, pick a different
MDA (such as procmail) and declare it with the `mda' option.

-----------------------------------------------------------------------
M1. I tried to run a mailing list using multidrop, and I have a mail loop!

This isn't fetchmail's fault.  Check your mailing list.  If the list
expansion includes yourself or anybody else at your mailserver (that is, not on
the client side) you've created a mail loop.  Just chop the host part off any
local addresses in the list.

If you use sendmail, you can check the list expansion with sendmail -bv.

-----------------------------------------------------------------------
M2. My multidrop fetchmail seems to be having DNS problems.

We have one report from a Linux user (not the same one as in R2!) who
solved this problem by removing the reference to -lresolv from his
link line and relinking.  Apparently in some recent Linux
distributions the libc bind library version works better.

As of 2.2, the configure script has been hacked so the bind library is linked
only if it is actually needed.  So under Linux it won't be, and this problem
should go away.

-----------------------------------------------------------------------
M3. I'm seeing long DNS delays before each message is processed.

Use the aka option to pre-declare as many of your mailserver's DNS names
as you can.  When an address's host part matches an aka name, no DNS lookup
needs to be done to check it.

Sometimes delays are unavoidable.  Some SMTP listeners try to call DNS
on the From-address hostname as a way of checking that the address is valid.

-----------------------------------------------------------------------
