libtgvoip/README.md

# Daniil Gentili's submission to the VoIP contest 3

In this submission, I created a new and improved protocol that unifies all flags, lengths that were previously sparsed around the protocol into one single data payload.

This allows saving around 6 bytes for each data packet, and enormously reduces complexity of existing code, splitting business logic and (legacy) presentation/transport details into separate classes.
All signaling packets (including init and initAck) are now unified into extra signaling fields. 
A TL-B scheme for the new protocol can be viewed in `schemeNew.tlb` (compared to the old protocol in `scheme.tlb`).  

Constructor serialization and parsing is implemented using a manually-written polymorphic approach, inspired by Telegram's C++ applications (tdesktop, tdlib).  
My implementation of polymorphic TL serialization and parsing can be viewed in `controller/protocol/protocol/*` and `controller/protocol/packets/PacketStructs.*`.  

**The new libtgvoip is 100% backwards compatibile with previous versions, keeping compatibility even with layer 8 and below, while enormously reducing complexity, encapsulating all legacy behaviour implementation details into a single `Legacy.cpp` file.**  

Among other features, all packet data buffers are moved around as `unique_ptr`s instead of being copied: previously, feeding a packet to the OPUS decoder through the jitter required **two** copies, and allocation of a slot in a buffer pool; now, a single `unique_ptr` (generated when initially parsing the packet) is moved around, instead (`protocol/Protocol.cpp`).  
Similarly efficient move-logic is used for outgoing audio buffers (EC & main), network socket payloads, and reliable packet resending (see below).


The new protocol also allows for (backwards-compatible) transport multiplexing for streams, simply by making use of the packet seq (in newer protocols) or the audio PTS (in older protocols), without using additional fields.  

This allows `AudioPacketSender.cpp` to reliably determine whether a certain packet was received by the other end or not, and resend only the missing audio pieces, especially in the case of Error Correction (EC) packets generated by the secondary low-bitrate opus encoder: only packets that are known to be lost (as of now) are re-sent, by using a Mask array type instead of a basic Array, skipping audio pieces that were already acknowledged for the current stream by the other endpoint.

I've also kept the reliable packet resending for the main audio stream, by further building onto and refactoring the reliable packet logic, which was now merged with the extraSignaling reliability logic (`Reliable.cpp`).

I've fixed a few nasty jitter buffer bugs:
- Fixed a bug where the jitter buffer would wrongly reset in presence of EC packets (5ba6e99882bdc045ae64a3b4f65039910677edf0; for latest commit JitterBuffer.cpp line 142-151, wrongly interpreted EC packets as late main packets)
- Fixed a minor bug where the jitter buffer would backtrack to an invalid timestamp when trying to fill gaps in transmission (457576e93faf8e99c49f936685d5c1582ca221bc; for latest commit JitterBuffer.cpp line 314, delay not multiplied by OPUS step)

Among other changes, the data structure representing incoming and outgoing streams was split into several polymorphic data structures and classes (`protocol/packets/PacketStructs.h`, `protocol/Stream.h`) to save memory and properly separate business logic from datastructures.


The commit history will be available @ https://gitlab.com/danog/libtgvoip as usual.

Please note that commit 2252eb1a2056dc63ec09e4f1acc1c7bfa6fec9e4 contains an initial final version that suitably fixes one specific issues with audio quality: in case of sudden but short packet loss bursts (like when switching from WiFi to mobile network), libtgvoip would **usually** enable redundant EC, and most importantly **decrease the bitrate of both main OPUS stream to the absolute minimum**.  
This causes hearable and persistent audio quality issues even if packet loss is reduced right afterwards (since it takes a while for the congestion controller to raise the audio quality again).  
In that commit, I **completely disable manual OPUS bandwidth regulation**, instead relying on EC packets to fill in the gap during the short period of downtime: this yielded **excellent results** in my tests (I used a real-life device, switching constantly from wifi to weak 4g, with only very brief switches to EC hearable in the audio stream, automatically switching back to the high-quality 20kbps stream right afterwards).  
The same results were confirmed by the `tgvoip-test-suite`: after re-enabling congestion control in the **next commit**, albeit with some fixes to account for packet resending: this also yielded good results even in low-bandwidth conditions.

If I have time until the end of the submissions, I'll refactor the jitter buffer to avoid excessive switching between the main and EC audio streams in case of packet loss in low-bandwidth conditions.

In the meantime, just wanted to say that I really enjoyed working on this contest: as usual, I have many more ideas for improvements, but for now, I'm pretty satisfied of the result.

Thank you.

---

# Daniil Gentili's submission to the VoIP contest 2

My submission consists of a major refactor (to C++) of the existing libtgvoip library.

The existing code was somewhat readable, even if extremely bulky, all in one single controller class, and had way too many C-isms, assertions, unmanaged pointers and then unoptimized C++ code progressively added into the library at later stages.

I've removed many C-isms within the main controller, switching entirely to smart pointers for the management of internal objects, reducing to 0 the number of `delete`s in the destructor; with multiple refactoring passes I've optimized many places of the network loop with modern C++ data structures and general logical optimizations.
I've also applied several optimizations to the code of the controller and helper utilities especially in the MessageThread, buffers and main network threads, switching to the C++ STL for many otherwise highly unefficient and unsafe operations (like throwing assertions if the `delet`ion order of objects using buffers from bufferpool isn't exactly right, easily fixed with smart pointers and a lambda).