API, Challenges and Licensing

Sometimes, technical problems, licensing issues and the need to eat collide. API access to LibreQoS was one of those uncomfortable areas. In this post, we’ll talk about our design philosophy, competing needs that lead to the separation of the API, and how we’re working to find a happy medium in release 2.2.

“We have our own graphing system!”

We hear you. Ever since the original LTS, and Insight, we’ve run into people who would rather use their own graphing and statistics systems. That’s fine (although you’re missing out on stats that aren’t misleading you with multi-minute averages), but it presents us with a few challenges.

Technical Challenges

The first is technical. From talking to users of versions 1.4 and 1.5, people were reaching into the node_manager internal HTTP API, and grabbing data from it. On a secondary level, they were doing so without telling us — or if they did tell us, it was right after we changed something and broke their setup (sorry, not sorry!). From a development point of view, we literally can’t freeze the internal APIs used to display the web interface. If we do that, development either grinds to a halt, or we have to maintain this incredible spaghetti-monster of versioned calls preserving backwards compatibility for all of the things we’ve ever done - even the mistakes. Look at Microsoft Windows for an example of how that can hurt; and we have a budget many, many times smaller than Microsoft’s!

Many of the technical changes were absolutely necessary, too. Version 1.4 used the Rocket webserver. Unfortunately, Rocket became largely unmaintained. That happens (especially in Open Source, where we’re all really relying on volunteers). So we switched over to Axum. Axum is fast, nice to work with, actively maintained (and supported by the Tokio people), and is becoming a bit of an industry standard. Awesome. Unfortunately, that broke a few people’s code that relied on our internal APIs… and nobody actually told us about it until months later.

If you’re working on your own LibreQoS tooling, we would love to hear about it. Open Source is meant to be collaborative!

Other technical changes happened because they make sense for how LibreQoS works. For example, we’ve largely switched from polling HTTP(s) calls to multiplexed websockets and a pub/sub architecture. We even use binary, compressed CBOR rather than JSON; the bandwidth utilization of the web interface and general responsiveness benefit from this a LOT. And yes - that broke a few more integrations, mostly ones nobody told us about until they started grumbling on social media. As you can tell, that fills the development team with joy.

We aren’t deliberately breaking your stuff! But we want to work with you to make sure that your stuff works, and we provide a stable way to give you what you need.

The same is true of lqos_bus. It’s a great integration point, but we just can’t promise a stable contract for it. If we add features — the bus needs them. If we fix a security issue, the bus probably changes. It’s really volatile. We obviously can’t stop you from using it, but be prepared for a lot of pain and disappointment.

So that covers some of the technical challenges. Let’s talk about something that always makes developers uncomfortable: having to afford dinner.

Financial Challenges

As I mentioned in our “Free as in Freedom” post, we can’t develop this for free. The number of Internet Service Providers with multi-thousand circuits (accounts with all devices inside them) who donated was about enough to buy a pizza from time to time. So, we took the hard step of adding a payment requirement to the published builds at 1,000 circuits. We also added an API-only tier (more on that soon) for a fraction of the price.

And yes, we’re absolutely, 100% committed to Open Source. The source code with full history is right here on GitHub. You’re absolutely free to use it, modify it. You can even make your own versions, so long as you abide by the GNU General Purpose License.

We had some early interactions with some rather unpleasant operators who demanded that we write a system for them to extract all the stats without Insight, and without ANY sort of donation. They have tens-of-thousands of users. We’re not going to name names, but if you have an ISP at that scale and want data from LibreQoS - either write it yourself, or pay us for the development! Anyway, this soured our stomachs and made us question continuing to donate time, effort and in some cases money to the project. So we may have overreacted slightly by deciding that ALL API access requires a license, rather than the first thousand circuits being free.

So, in version 2.2 - we’re simplifying:

• For up to 1,000 circuits - the shipped version of LibreQoS will provide the API.
• Above 1,000 circuits, you can buy an API-only license or Insight.

And our prices are pretty great. They are also localized - we’re trying very hard to subsidize the parts of the world that need us the most. $150 USD for 1,000 active circuits (note “active” - we don’t bill for circuits with zero traffic), or $0.15 per subscriber is really cheap in the US market. Firmly in “if you aren’t making $0.15 per subscriber, you might consider buying a taco truck” territory. It’s even lower in developing markets such as the Philippines.

You can, of course, reach out to us about pricing at any time. We love to work with people who have a genuine hardship and need.

Solution: A Stable API

Rather than shackle development of LibreQoS by freezing internal APIs/bus calls, a better middle-ground is to provide a stable, versioned API. It comes with the implicit contract that we’ll only break it for emergencies such as security vulnerabilities. It covers most of the system, and is being expanded. An API can let you have your own monitoring and graphing if you want it. It’s come a long way since our first API post, and ships with every binary distribution of LibreQoS.

Together with the relaxed licensing requirements and the low price - hopefully this middle-ground will work for ISPs worldwide.

Wrapping-Up

We’re working hard to make sure that the API that ships with LibreQoS is comprehensive. Retrieve stats for pretty much everything we track. Control circuits. Interact with the dynamic circuit system. And so on. We’re absolutely committed to remaining Free as in Freedom - you have the source, you have the power. And we’ve listened; API and Insight are on the same access tiers, so small ISPs aren’t penalized - and we grow with you.