Methodology
In the current backup schema, all paper paths are of the form <year>/<num>_<timestamp>{.pdf,.ps,.ps.gz}, where <timestamp> is the Unix epoch (in seconds) of the time that the file was uploaded to ePrint. For each snapshot, only the newest version is retained. For most papers, the latest version is retrieved by going to the "version history" page on the ePrint site and downloading the latest version marked "PDF Update" ("metadata update" entries do not change the PDF). This methodology does not work for all papers, though. Here are the exceptions, and what we chose to do about them:
- When the paper is Postscript, then the version page says "metadata update" for all updates, even if the Postscript file was updated (eg
2004/028). We name all Postscript papers the timestamp of the latest metadata update.
- Some papers have no version page (eg
2000/010). For these we set the timestamp to be midnight UTC on the date of the last revision on the website (this coincides with the date_last_revised field in the metadata JSONL)
- The version page on ePrint does not always contain all the versions of the paper. Sometimes the default download (the one you first see on the paper page) is distinct from all the PDFs on the version page. Sometimes it is newer (
2018/596) and sometimes it is older (2017/267). When the latest-update version is clearly the newest, we simply use that. We explain all other cases below:
- The latest-update PDF of
2023/1523 is empty. So we use the second-latest-update version, which matches the default download.
- The latest-update version PDF of
2020/1033 is an invalid PDF. So we use the default one and estimate the timestamp by PDF inspection.
- The default downloads of
2015/1208 and 2018/596 have a later creation date than the latest-update version. So we use the later ones and estimate the timestamps.
- For some papers, the latest-update version is marked withdrawn, but the paper is not marked withdrawn in ePrint's own metadata:
2002/191, 2005/037, and 2005/185. Here, we use the default download and estimate the timestamp.