Joseph K. Myers

Sunday, October 26, 2003

Night Get, Downloading Files Through Confusing, Unreliable PPP Dialup Connections

(And how many keywords can you stick in h1?)

When you try to download files like

qt-mac-free-3.2.2.sit, 15809242 bytes,

then you will find out on my internet connection that between 50 and 70 minutes the download will peter out.

It'll just stop. Quit. Abscond. Desist without your will to desist.

However, a dialup connection can be script-urally manipulated on Mac OS X / Darwin.

I'm not sure of exactly the way to really do it right (the equivalent of choosing "Connect" from the menu), but I know that on Jaguar 10.2, at least, I can use a script like the following to connect, knowing my particular dialup numbers, etc.

Like so, conn.sh:

pppd serviceid 2 plugin \
/System/Library/SystemConfiguration/PPPController.bundle/Contents/PlugIns/PPPDialogs.ppp \
logfile /tmp/ppp.log plugin PPPSerial.ppp device modem modemscript \
'Apple Internal 56K Modem (v.90)' nomodemsound modemreliable \
modemcompress modemtone modemdialmode 0 remoteaddress 219-4747 \
altremoteaddress 283-0215 redialcount 6 redialtimer 5 idle 300 \
lcp-echo-interval 10 lcp-echo-failure 4 mru 1500 mtu 1500 receive-all \
0:0 noipdefault ipcp-accept-local ipcp-accept-remote usepeerdns noauth \
user [] password [] forcedetach noccp looplocal \
noaskpassword

Of course, you can find the variant fields fairly easy, I think from observation. This is exactly what I use to dial up, with only the user and password removed with brackets.

(OK, I know it is a real problem. The right way to do it is to let the computer run its own high-level setup of whatever happens when a dudette clicks on her connect button. I.e., that command should be let alone to be generated dynamically by the computer's own knowledge of your connection gyrations. Except I never looked how.)

But at any rate, there are three more scripts.

disconn.sh:

kill `cat /Users/jmyers/b/wget.pid`
kill `cat /var/run/ppp0.pid`

You've got to hard-code your path, I think. The deal with the technique of killing wget like it was a road kill is due to the fact that I use wget to download the file.

If you don't want to use wget... but believe me, you do want to use wget. It is so much more flexible and lazy to use, although someone does need to rewrite it, I think.

night.get:

source /Users/jmyers/.bashrc
if test ! -f /var/run/ppp0.pid
then
/Users/jmyers/bin/conn.sh
sleep 140
fi
su jmyers /Users/jmyers/bin/night.wget

This script is run as root under cron, so that it can connect. (Another thing in favor of connecting the right way, because the way that the connection menu works, an ordinary one-year-old user can connect--not having to be root.)

night.wget:

cd /Users/jmyers/b
if test -f wget.pid
then kill `cat wget.pid`
fi
wget --passive-ftp -cb -i newlist | perl -pe 'm/pid (\d+)/; $_ = $1' > wget.pid

Then the crontab is generated by this script in perl, make24hr-crontab.pl:

print STDERR "Enter the hours to begin and end (0-23): ";
chomp($_ = <STDIN>);
my ($beg, $end) = split;
$end ||= $beg;
for (0..23) {
next unless $_ >= $beg && $_ <= $end;
for $r (rand()*10, 30+rand()*10) {
$r = int($r);
print "$r $_ * * * /Users/jmyers/bin/night.get\n";
}
}
$end++;
print "0 $end * * * /Users/jmyers/bin/disconn.sh\n\n";

So for example, you do

% perl make24hr-crontab.pl

Which might create a run like this:

Enter the hours to begin and end (0-23): 2 5
3 2 * * * /Users/jmyers/bin/night.get
37 2 * * * /Users/jmyers/bin/night.get
7 3 * * * /Users/jmyers/bin/night.get
39 3 * * * /Users/jmyers/bin/night.get
6 4 * * * /Users/jmyers/bin/night.get
37 4 * * * /Users/jmyers/bin/night.get
0 5 * * * /Users/jmyers/bin/night.get
30 5 * * * /Users/jmyers/bin/night.get
0 6 * * * /Users/jmyers/bin/disconn.sh

that you can use in your crontab (root crontab).

Notes:

The assumption is that you have a directory ~/b which holds files, and that newlist is the file containing lines of download URLs. Also, bash is used, and ~/.bashrc is run, just in case there are is any special $PATH set from that file.

P.S. If you find/know the right connect script, that one will I use in favor of this one. To the general user, my connect script is as non-portable as heaven. (Not that you would want heaven to be portable, kinda like you don't want anybody else to use your girlfriend.)