And if someone will hit the same issue, here is the story:

It looks like Zaram (at least with current firmware) isn’t the best choice. Combination of Zaram with everything else (on my side and Odido side)  ends up in so called “bufferbloat”, when due to overloading of buffers latency spikes and throughput goes to low values.

Solutions:

1. use BBR congestion control algorithm on all clients behind router. Works with linux (sysctl net.ipv4.tcp_congestion_control=bbr) and FreeBSD-13 (tcp_bbr module), macOS does not have this option, and I have no clue about windows.
2. traffic limiting with “fq_codel” to avoid overload of transmission chain. (but sadly pf/opnsense have long standing BSD bug, which does not allow limiting fro values higher than 4Gbps)
3. route I went, just set on x7100  max_tx_rate to 5Gbps (higher it is getting unstable).

Hi, also in possesion of a Zaram XGS-PON SFP+ which is currently connected to Caiway over Delta fiber.. Will be switching to Odido over Delta fiber shortly and was wondering how you got this working with the Zaram module. Did you have to clone something from the Huawei/Nokia ONT or were you able to register the Zaram SN directly at Odido?

For registration you can ask @Demi van Odido or @Tommie van Odido .they will register your Zaram PON-ID.

at what speed do you have it working now? Which router? What is your Speedtest or iperf3 results?

Thanks for the info. They are approchable through this forum? For now its just 400Mbit but with the new Odido connection I should have 2Gbit. SFP+ module is connected into a Ruckus 7150 switch and vlan termintated on a Intel Nuc12 running Pfsense on Proxmox. Inter vlan routing is over 8gbit so should be good for the Odido  circuit.

Hi @debink, you can send me a PB with the PON ID if needed, I will register your Zaram module! 

Ok, if some will be interested in this topic later, here is little update.

I am running some tests with both, Zaram and Huawei ONTs and bufferbloat (later I will use abbreviation “BB”).

Upload:

Zaram: Clearly, there is a severe risk of BB in Odido’s (or KPN?) network. Zaram does nothing about it, and it hits BB full strength, reducing throughput 4x (!!!). At the same time, if I limit TX rate on my SFP+ NIC (which allows that)  to 5200mbps (so, traffic is limited before it hits Zaram), then I have perfect upload with low latency and low latency deviation. But at 5200. And this is I assume we can call “real useable speed”

Huawei: It is very visible that Huawei implements some kind of SQM in firmware. And not trivial traffic limiter, but something more elegant. It seems that they don’t limit maximal bitrate, but rather prevent quick rise of bitrate, which, in case of Odido network, shows to be enough to avoid BB and at the same time, achieve maximal upload speed providing that upload goes of long time to reach upper limit.

I tried to reproduce the same with the changing gradually TX rate on Zaram and yes, it works exactly the same way as Huawei and also allows to hit maximum, but not immediately. In this case we clearly see that  BB goes wild only if one have sudden surge in. But then, it is really really severe case of BB.

So, in case of Huawei approach, for achievable maximal speed we pay price, that this speed never can be achieve of short surges and that latency is still very very unstable under heavy load (Zaram limited to 5.2G is nearly perfect in that sense).

In principal, I suggest everyone who has an option of limiting bitrate stream to ONT, should use it, even if you have huawei ONT.   For that you have various options:

1. if you run DIY setup, some NICs allow TX limiting. 
2. if you run linux-based router like OpneWRT or VyOS, they both have SQMs, look for keyword “fq-codel” (in case of OpenWRT can also look for “cake”, which might be better also in some situations)
3. if you run Ubiquity router, they have also some form on protection against BB in form of SQM, don’t recall what it is called.
4. if you run FreeBSD based router like pfSense/OPNSense….then you have no luck, as “dummynet” is still limited to 4Gbps due to 32-bit integer value. Issue is known for many years, but nothing happens. I am trying to poke developers on this matter now.

of course, you test locally what what value for the limiter works best for you.

Download:

In case of download, both, Zaram and Huawei, show similar behavior. Looks like there is nothing special done like SQM, expect, maybe, sending ECNs. Both of them are able to achieve maximal download rate.  But both severely suffer  form high spread of latencies under load, and Huawei here even worse, on multiple occasions I’ve seen worst latency to go above 1 second ( in plan English: forget about competitive gaming or high frequency trading from home. And yes, “competitive gaming” is not something person older than 21 is capable of! ) .

for this side I didn't experiment with solutions, but we can’t do much beside introduction of some kind of limiting with SQM also :)

couple of side notes:​​​​​​

1. if you do nothing about fighting BB on router side, you can, at least, use on endpoints different congestion control algorithm. Good suggestion will be Bottleneck Bandwidth and Round-trip propagation time (BBR) algorithm.  Although, it is available only for Linux and BSD.  For linux : “sysctl net.core.default_qdisc=fq && sysctl net.ipv4.tcp_congestion_control=bbr” , for FreeBSD (assuming you have kernel with built BBR algo) “kldload tcp_bbr && sysctl net.inet.tcp.functions_default=bbr”. In general, it is probably not bad idea to use BBR in any case if you want to get maximum from network. (but, again, your mileage might vary and, particularly, in case of BSD one can hit some issues with packet filter). But keep in mind, congestion control like BBR must be enabled on end-points. Router with enabled BBR will not help your end-points.
2. I am sure it is not something “on my side of ONT”, as I tested it with quite a variation of hardware and software setups. I tested VyOS, pfSense, OPNSense, OpenWRT routers on bare metal and virtual ones (on different hardware). I tested direct connection to ONT from endpoint, without routing involved, I tested ONT attached via different NICs and/or though intermediary switches etc. It is always the same outcome. But, in general, it is not “Odido is doomed” either. With high speed networking, congestion control is a big headache and problem, this is why things like BBR  algorithm are in active development. World suffers from BB :)

To be continued :)

Sorry, I didn’t write this is most simple language, but I assume that someone who might be interested in BB, knows some terminology. But just in case, if needed, I can explain in more plain English :)