www.delorie.com/gnu/docs/wget/wget_21.html

search

	Buy GNU books!

GNU Wget Manual

5.1 Time-Stamping Usage

The usage of time-stamping is simple. Say you would like to download a file so that it keeps its date of modification.

wget -S http://www.gnu.ai.mit.edu/

A simple ls -l shows that the time stamp on the local file equals the state of the Last-Modified header, as returned by the server. As you can see, the time-stamping info is preserved locally, even without `-N' (at least for HTTP).

Several days later, you would like Wget to check if the remote file has changed, and download it if it has.

wget -N http://www.gnu.ai.mit.edu/

Wget will ask the server for the last-modified date. If the local file has the same timestamp as the server, or a newer one, the remote file will not be re-fetched. However, if the remote file is more recent, Wget will proceed to fetch it.

The same goes for FTP. For example:

wget "ftp://ftp.ifi.uio.no/pub/emacs/gnus/*"

(The quotes around that URL are to prevent the shell from trying to interpret the `*'.)

After download, a local directory listing will show that the timestamps match those on the remote server. Reissuing the command with `-N' will make Wget re-fetch only the files that have been modified since the last download.

If you wished to mirror the GNU archive every week, you would use a command like the following, weekly:

wget --timestamping -r ftp://ftp.gnu.org/pub/gnu/

Note that time-stamping will only work for files for which the server gives a timestamp. For HTTP, this depends on getting a Last-Modified header. For FTP, this depends on getting a directory listing with dates in a format that Wget can parse (see section 5.3 FTP Time-Stamping Internals).