Vegvisir

An automated testing framework for web applications using QUIC, using a containerized client-shaper-server architecture.

Presented in the paper “Vegvisir: A testing framework for HTTP/3 media streaming” by Joris Herbots, Mike Vandersanden, Peter Quax and Wim Lamotte, in the Proceedings of the 14th ACM Multimedia Systems Conference (MMSys ’23), Vancouver, Canada, pages 403-409.

Paper (ACM) GitHub

Vegvisir Logo

Info > Running quic-go over tc-netem against aioquic
Info > TimeoutSensor timeout triggered [60sec]
Info > Running quic-go over tc-netem against lsquic

[quic-go]===[tc-netem]===[lsquic]

Vegvisir’s output while it runs quic-go against lsquic over tc-netem.

Summary

Assessing media streaming performance takes reproducible network conditions and a varied set of media, which is hard to set up. QUIC makes it harder still, because it has many tunable features and is encrypted, so it is difficult to analyze. The paper introduces Vegvisir, an open-source framework that automates media streaming experiments over HTTP/3.

How it works

Vegvisir runs QUIC clients and servers in Docker containers, sends their traffic through a shaper that imposes network conditions, and collects the logs. Two JSON files steer it: one lists the available implementations, the other describes the experiment, which is a list of clients, shapers and servers to combine. Every combination is run as a test, so three clients, three shapers and three servers give 27 tests.

Vegvisir architecture: clients send traffic through a shaper to servers, and all of them write logs that qvis and Wireshark can open

It works with most QUIC implementations from the QUIC interop runner, and with two shapers, tc-netem and ns-3.

Getting started

Vegvisir is open source (Apache 2.0). It needs Linux on x86-64, Python 3.10 or newer, and Docker. An experiment runs with:

python -m vegvisir run -i implementations.json experiment.json

Cite

@inproceedings{herbots2023vegvisir,
  author    = {Herbots, Joris and Vandersanden, Mike and Quax, Peter and Lamotte, Wim},
  title     = {Vegvisir: A Testing Framework for {HTTP/3} Media Streaming},
  booktitle = {Proceedings of the 14th ACM Multimedia Systems Conference (MMSys '23)},
  address   = {Vancouver, Canada},
  pages     = {403--409},
  month     = jun,
  year      = {2023},
  publisher = {ACM},
  doi       = {10.1145/3587819.3592550},
}
BibTeX