How to track upload progress?

[wuyuanyi135]

Hello, I have read the documentation about how to track the progress of downloading. I want to implement a similar functionality for post uploading. However, the following code did not track the progress in realtime. Instead, it finishes much faster than the actual uploading. I suspect that the library will try to read and cache the data from the io so I could not get the real upload progress. Can you suggest a proper way to implment this functionality? Thanks!

    content = firmware.read()
    firmware_size = len(content)

    class ProgressMemoryReader(BytesIO):
        def __init__(self, data, read_callback=None):
            super().__init__(data)
            self.read_callback = read_callback
            self.length = len(data)

        def read(self, size=None):
            if self.read_callback:
                self.read_callback(self.tell(), self.length)
            return super(ProgressMemoryReader, self).read(size)

    client.post(
        "/ota",
        content=ProgressMemoryReader(content, progress_callback),
        headers={'Content-Type': 'application/octet-stream'}
    )

[slophisticated]

A read callback measures how far the request producer has progressed, rather than how many bytes the device has received. It can reach 100% while data is still buffered by the transport/OS, or while the device is processing the upload. Moving the callback alone cannot give you a device-side acknowledgement.

For a bounded-memory upload, the documented request-content generator approach works with your synchronous client:

import os

def upload_chunks(firmware, total, progress_callback):
    completed = 0
    while chunk := firmware.read(64 * 1024):
        yield chunk
        completed += len(chunk)
        progress_callback(completed, total)

with open("firmware.bin", "rb") as firmware:
    total = os.fstat(firmware.fileno()).st_size
    response = client.post(
        "/ota",
        content=upload_chunks(firmware, total, progress_callback),
        headers={
            "Content-Type": "application/octet-stream",
            "Content-Length": str(total),
        },
    )
    response.raise_for_status()

This avoids reading the whole firmware into memory first. The explicit length also avoids relying on chunked request encoding when an OTA endpoint requires a known length. Each callback happens after the yielded chunk has been consumed and iteration resumes.

I tested this pattern with HTTPX 0.28.1 against a local HTTP server: a 2 MiB payload arrived intact, with 32 monotonically increasing progress updates. The final update still preceded server completion. Label that progress as bytes supplied for upload; wait for the successful response to report request completion. If you need progress for bytes actually received or firmware written, the device/API needs to expose that information.