What is the network protocol overhead of game downloads? And how does Steam do it?
I am asking because, if my observations were correct, I downloaded a game on Steam and it said that the data transferred was 16.6-16.7 GB - I got two slightly different infos from the client at different points.
But I watched my OS (Kubuntu Linux) network throughput logging and it showed 18.5 GB down and 0.4 GB up for that download alone AFAIR. That would be 13% overhead on the usable data. Is this normal?
< >
Showing 1-7 of 7 comments
Steam downloads work using HTTP; it grabs a manifest, which maps the files to be installed to a bunch of ~1 MB chunks of data, and then it downloads each of those 1 MB chunks with a series of HTTP GETs.

13% seems strangely high to me. The HTTP headers shouldn't come close to that on 1 MB file transfers (10% HTTP overhead would imply ~100KB headers, which seems implausible), and IP and TCP headers should only account for about 2.5% overhead (about 20 bytes each on a typical 1500 byte MTU)

So, I don't know what's going on; which may just reflect my ignorance.
Last edited by aiusepsi; 1 Sep @ 2:45am
Cray 1 Sep @ 4:17am 
It'll be asked at some point anyway, so to get it over with:

Did you check that neither you nor the respective softwares mixed up GiB and GB?

And how exactly did you view the network throughput logging? Because if it's total throughput for the whole OS it's really hard to say what's what or if something else is muddying it.

Maybe a filtered pcap dump and some wireshark trawling would give a cleaner view?
There is also packet loss and security protocols to consider (the second one can add a lot of overhead).
Last edited by MunsterZX; 1 Sep @ 6:11am
Originally posted by Cray:
It'll be asked at some point anyway, so to get it over with:

Did you check that neither you nor the respective softwares mixed up GiB and GB?

And how exactly did you view the network throughput logging? Because if it's total throughput for the whole OS it's really hard to say what's what or if something else is muddying it.

Maybe a filtered pcap dump and some wireshark trawling would give a cleaner view?

OS 'background' throughput is negligible.
Linux states GiB, Steam GB. If those were referring to different units, then the OS would have to show LESS data transferred. (1 GB ~ 0.93 GiB).

When the opportunity arises (I'm on limited data plan), I'll do more tests to see whether my measurement was off. But I don't think so.

Originally posted by MunsterZX:
There is also packet loss and security protocols to consider (the second one can add a lot of overhead).
Relevant amount of packet loss should be an exception, no? And what do you mean by security protocols that can add a lot?
If these GET requests comes over HTTPS then TLS support could have a small overhead. But I don't think it will be significant.
Unless I’m missing something about TLS, the main source of overhead with TLS is the setup of a connection.

In this case, the connections are long-lived and used for lots of GETs each so the cost should be amortised down to practically zero. But I’m not a TLS expert.
Originally posted by aiusepsi:
Unless I’m missing something about TLS, the main source of overhead with TLS is the setup of a connection.

In this case, the connections are long-lived and used for lots of GETs each so the cost should be amortised down to practically zero. But I’m not a TLS expert.
Yea, the handshake.
< >
Showing 1-7 of 7 comments
Per page: 1530 50