I saw this too; yesterday I tried manually accessing the page to explore just how many there are. Seems like some of the pages are duplicates (I was simply comparing the last listed file name and content between some of the first 10 pages, and even had 1-2 duplications.)

Far as maximum page number goes, if you use the query parameter ?page=200000000 it will still resolve a list of files. — actually crazy.

https://www.justice.gov/epstein/doj-disclosures/data-set-9-files?page=200000000

Can also confirm, receiving more chunks again.

EDIT: Someone should play around with the retry and backoff settings to see if a certain configuration can avoid being blocked for a longer period of time. IP rotating is too much trouble.

I also was getting the same error. Going to the link successfully downloads.

Updating the cookies fixed the issue.

I'll work on the second –startByte 110131937280 --endByte 120424759295 (9.59 GB)

EDIT: I’m probably at 20-30 passes by now. Got squat.

Do you think this is a bug, or is it possible the chunk is not there?

Nor I. I got a single chunk back before never getting anything again.

Gonna grab a some tea, then get back at it. Will update when I have something.

Thanks for this!

EDIT: This works quite well. Getting chunks right off the bat. About 1 per second, just guessing.

If we could target different byte ranges, having 10-20 different people spaced through the expected range could cover a lot of ground!

Absolutely

I had the script crash at line 324: BadStatusLine: HTTP/1.1 0 Init

EDIT: It’s worth noting that about every time I (re) start it after seemingly been blocked a bit, I get about 1gb more before it slows WAY down (no server response).

EDIT: It looks to me, that if I'm getting only FAILED: No server response, stopping the script for a minute or two and restarting immediately garners a lot more results. I think having a longer pause with many failures might be worth looking at. -- I'll play around a bit.

8 passes in, still haven't gotten a single chunk

My IP appears to have been completely blocked by the domain. Multiple browsers, devices, confirm it.

If anyone has any suggestions for other options, I’m listening.