DRIVE_DOWNLOADER

# Drive Downloader

**v1 · Mantra Productions · Zagreb**

Pull a whole Google Drive folder onto this machine, over a connection that
drops, without ever starting a large file again from the beginning.

Replaces the `dialog` + `rclone` shell script it grew out of. The reason it
replaces it rather than wrapping it: **rclone does not resume part of a single
file.** It retries whole files. On a 4 GB video over a connection that drops at
3.5 GB, that costs 3.5 GB every time. This fetches the missing bytes and appends
them.

---

## What it does

- **Byte-level resume.** Every file downloads into `name.part`. On a fresh
  attempt the app asks Google for the bytes *after* the ones it already holds
  (`Range: bytes=N-`) and appends them. There is no second file and no seam —
  a half file and a whole file are the same file at two moments.
- **Survives a kill.** Progress lives in SQLite, not in memory. Pull the plug at
  any instant; the next start reads the same state off disk.
- **Believes the disk, not itself.** Before every run it walks the destination
  and corrects its own ledger from what is actually there. A row claiming 400 MB
  means nothing if the file holds 12 MB.
- **Verifies.** Every finished file is checked against Drive's own MD5 before it
  is given its real name. A file under its real name has been verified.
- **Knows how much is left.** Bytes done, bytes remaining, files done, speed,
  time left, and a per-file list.

## What it deliberately does not do

- **Google Docs, Sheets and Slides cannot resume.** They have no bytes until
  Google exports some, the export has no length known in advance, and the export
  endpoint ignores `Range`. They are fetched whole or not at all, and the page
  says so on every one of them. They are excluded from the byte total rather
  than blended into it, because a percentage that quietly leaves files out is a
  lie told confidently.
- **One job at a time.** Parallel folder downloads are not implemented.

---

## Three ways in

Exactly one is in force at a time.

### 1. Shared folder — *the simple one*

A **service account** is a Google account with an email address and no person
behind it. You share a Drive folder with that address, the same way you would
share it with a colleague, and the app reads it.

No browser. No consent screen. Nothing expires. It can see the folders you
shared with it and nothing else — which is also what makes it the safe one.

- Google Cloud Console → new project → enable the **Google Drive API**
- IAM & Admin → Service Accounts → Create → Keys → Add key → **JSON**
- Save it as `credentials/service_account.json`
- The page then shows you the address. Share your folder with it, as **Viewer**.

### 2. My whole Drive — OAuth

For reaching everything your account can see rather than one shared folder.

- Google Cloud Console → OAuth consent screen → External → add yourself as a
  test user
- Credentials → Create → OAuth client ID → **Desktop app**
- Save it as `credentials/client_secret.json`

One browser trip, then `token.json` appears beside it and refreshes itself.

### 3. Public link — API key

For a folder set to *anyone with the link*. Credentials → Create → **API key**,
then paste it into the page. It grants nothing on your account, so this is the
one to hand to somebody else for a test.

---

## Install

macOS:

```bash
bash 1sh_i_drive_downloader_macos.sh
drive-downloader
```

Then open **http://127.0.0.1:5055**. The first screen sets a password for the
page. That password protects the page, not your Google account; it is stored as
a PBKDF2-SHA256 hash at 210,000 iterations in `config.json`, mode 600.

The app binds to `127.0.0.1` — it is reachable from this machine only. Change
`bind` in `config.json` to expose it, and think about that before you do.

---

## The four tests

```bash
python3 tests.py
```

    Four tests: 24, 6, 22, 9 passed, 0 failed

1. **The mechanism** — link parsing, filename safety, backoff, ledger arithmetic
2. **The real thing** — a real HTTP round trip with real `Range` headers, and a
   checksum agreeing with a source we did not write
3. **The ugly cases** — the connection cut twice mid-file and the reassembled
   file bit-identical to the original; a server that ignores `Range`; a corrupt
   fragment; a wrong checksum; a zero-byte file; an empty job; a 404
4. **The upgrade** — a folder the old rclone script already downloaded, with one
   file left half-finished

**What is not tested:** the Google endpoints themselves. Test 2 and Test 3 run
against a local server that speaks the same HTTP, which proves the resume logic
and nothing about Google's own behaviour on very large files or under quota
pressure. The OAuth browser flow is code inspection only. See
`WHAT_IS_NOT_TESTED.md`.

---

## The bug this found

`iter_content(1 MiB)` does not hand over a partial block. It waits for a whole
megabyte, and if the connection dies at 700 KB it raises with those 700 KB still
inside the library — nothing reaches the file. On a link that drops every half
megabyte the download would make **no progress at all, for ever, while looking
busy.**

The chunk size is therefore not a tuning knob. It is the size of the loss. It is
64 KiB.