I really wished to see an OTP-first design. Unfortunately for me, the code is almost procedural as it's touching ETS or Application, which is built on ETS, in nearly every operation.
If the author wishes to learn how to design services in Elixir, or any BEAM language, with OTP, they can take a look at "Designing Elixir Systems with OTP" by by James Edward Gray and Bruce Tate, and "Functional Web Development with Elixir, OTP, and Phoenix" by Lance Halvorsen.
guywithahat 7 hours ago [-]
There's something about C++ developers that makes them love Go and Elixir (and I include myself in this demographic). I think it's something about the people who are attracted to C++ for performance are attracted to Go/Elixir for its multithreaded performance. Really cool project
uncircle 3 hours ago [-]
Not sure about C++ devs, but Erlang/Elixir are great to handle parsing of protocols, with its implementation of pattern matching. Also, makes the code much cleaner because pattern matching basically eliminates most branching and thus depth of the code base.
The let it crash philosophy allows you to ignore most corner cases with the knowledge that, if they are encountered or a cosmic ray flips a bit, the crash is localised to a single client. I have worked with Elixir almost a decade at this point, and I have never seen an unexpected downtime of the apps I deployed. Aside of maintenance and updates, they all have 100% uptime. How cool is that?
This is how I sell it to clients. “Will you be using Python, Go?” Me: “What about Elixir and the promise that your service won’t ever crash? And you get cool dashboards with it.” Them: “Sold.”
I wish there was a systems language that allows you to pattern match on structs and enums, and in function signatures like Elixir
dahrkael 1 hours ago [-]
Indeed. when your daily job is tracking down memory stomps, deadlocks, invalid pointers and unexpected state in very big codebases then using Elixir feels like "why is this so easy? it just works?". Also i'm a network programmer so the binary pattern matching is very much appreciated.
arch-choot 2 hours ago [-]
Interesting! I'd done something similar in Typescript to learn more about BT, and then redid it in rust to learn rust (https://github.com/ckcr4lyf/kiryuu).
However I decided to just use redis as the DB. It sounds like your entire DB is in memory? Any interesting design decisions you made and/or problems faced in doing so?
(My redis solution isn't great since it does not randomize peers in subsequent announces afaik)
dahrkael 1 hours ago [-]
in my case using the in-memory ETS has been the best decision, it lets me read&write the peer's data concurrently each on its own process so contention and latency are minimal. the only sequential part is when a new swarm is initially created but that doesn't happen a lot so its fine.
there's sadly no native support for taking random rows directly from the tables, so for now i grab the whole swarm and then take a random subset (https://github.com/Dahrkael/ExTracker/blob/master/lib/ex_tra...)
interesting! why did you decide on a private tracker specifically?
abrookewood 8 hours ago [-]
Hey congrats on the launch! Can you provide any details on how it runs compared to opentracker? I'm really interested in the performance etc.
dahrkael 1 hours ago [-]
For small trackers opentracker is probably faster and use a bit less memory.
Where extracker is gonna shine compared to it is when core count starts having 2 digits.
I still have to do a proper benchmark though.
voicedYoda 7 hours ago [-]
Well done. Couple quick notes, move to a logger instead of using IO.puts. Also consider adding OTel.
solid_fuel 6 hours ago [-]
I'll second this, just using the built in Logger [0] and Telemetry [1] applications would be fine, opentelemetry or anything else can be added to the telemetry hooks easily to export the metrics later.
Love Elixir so much, building a kick-ass notification engine with it now. Its so so good.
mikehostetler 7 hours ago [-]
nice, private or OSS? Elixir needs a better notification engine badly
rhgraysonii 4 hours ago [-]
What do you mean exactly? If you need a notification engine, reaching for a pubsub implementation is very easy with phoenix’s popularity and quite battle tested. I’ve implemented notifications at scale a few times in the ecosystem. What problems are you encountering that you don’t feel you have a tool in the shed to work with in this case?
rhgraysonii 4 hours ago [-]
And if you wanna buy over build, check out https://knock.app - also built in elixir
vivzkestrel 6 hours ago [-]
- how did you start
- did you refer to other projects
- how long did it take
- how much functionality do you think works compared to say qbittorrent?
dahrkael 38 minutes ago [-]
I started because I needed a tracker for another project but the tracker turned to be more fun to make.
I did glance over other trackers code but their code tends to be either overly complex or too simple so not very useful.
So far its been 3 months of revenge bedtime procrastination.
While this is not a client like qbittorrent I have ideas for a seedbox-oriented client project in the future.
lionkor 4 hours ago [-]
it's a tracker, not a torrenting client.
NooneAtAll3 3 hours ago [-]
what does tracker mean?
devoutsalsa 3 hours ago [-]
A torrent tracker is basically the world’s most antisocial matchmaking service that knows who has what files but refuses to actually store anything itself, like that friend who always knows where the party is but never hosts one. When your BitTorrent client asks “hey who’s got that Linux ISO,” the tracker dumps a list of IP addresses faster than a startup pivoting after their Series A falls through. Your client then connects to these strangers (seeders with complete files and leechers still downloading) and starts exchanging data while the tracker pretends nothing happened. It’s like Tinder but for file sharing, except everyone’s anonymous and probably downloading something weird at 3am.
vjerancrnjak 2 hours ago [-]
not anonymous at all, while interacting with the tracker can be done with https, all of the communication between peers is unencrypted.
but it seems to work. I would've liked to see HTTP stats too but I guess UDP is fine (though I have it disabled)
bill876 1 hours ago [-]
The "paused" event is part of BEP 21. Clients send it to the tracker to let it know that the client is still incomplete, but won't download anymore. For example, because a user only wants some files from the torrent.
Readme of the project shows that support for BEP 21 is not implemented.
dahrkael 47 minutes ago [-]
Telemetry for the HTTP side is in my ToDo list yes, since I'm using a 3rd party library for the webserver I still need to figure out how to do it right.
For HTTPS to work you need to provide a valid certificate path in :https_keyfile but right now I would recommend sticking Caddy or Nginx in front of the tracker if you want HTTPS.
I have certbot integration planned but is not a priority since most of the torrent peers use UDP.
jhgg 8 hours ago [-]
Really cool! You looking to write Elixir as your main job?
dahrkael 52 minutes ago [-]
It's one of my options yes, I'm sure I would enjoy it more than C++.
pdimitar 42 minutes ago [-]
Not OP but I am working with Elixir for 9 years and 2 months now. Know Rust and Golang as well. You hiring?
IlikeKitties 10 hours ago [-]
Now that's neat. The Beam VM sounds like a natural fit for a torrent tracker
dahrkael 53 minutes ago [-]
I feel like ETS has been the real killer feature to pull this, being able to concurrently read and write from protected tables makes the whole thing incredibly parallel
If the author wishes to learn how to design services in Elixir, or any BEAM language, with OTP, they can take a look at "Designing Elixir Systems with OTP" by by James Edward Gray and Bruce Tate, and "Functional Web Development with Elixir, OTP, and Phoenix" by Lance Halvorsen.
The let it crash philosophy allows you to ignore most corner cases with the knowledge that, if they are encountered or a cosmic ray flips a bit, the crash is localised to a single client. I have worked with Elixir almost a decade at this point, and I have never seen an unexpected downtime of the apps I deployed. Aside of maintenance and updates, they all have 100% uptime. How cool is that?
This is how I sell it to clients. “Will you be using Python, Go?” Me: “What about Elixir and the promise that your service won’t ever crash? And you get cool dashboards with it.” Them: “Sold.”
I wish there was a systems language that allows you to pattern match on structs and enums, and in function signatures like Elixir
However I decided to just use redis as the DB. It sounds like your entire DB is in memory? Any interesting design decisions you made and/or problems faced in doing so?
(My redis solution isn't great since it does not randomize peers in subsequent announces afaik)
I wrote a basic tracker in Elixir a few years ago, here's the code: https://github.com/aalin/mr_torrent
[0] https://hexdocs.pm/logger/1.18.4/Logger.html [1] https://hexdocs.pm/telemetry/readme.html
Also my console gets spammed with:
04:43:20.160 [warning] invalid 'event' parameter: size: 6 value: "paused"
but it seems to work. I would've liked to see HTTP stats too but I guess UDP is fine (though I have it disabled)