Posts tagged "self-hosting"

  • Onion routing and services

    The other week I saw [this toot][tor-toot] from the Tor Project:
    Jul 9
    The Tor Project                                @torproject@mastodon.social
    
    Starting today, we're running a month-long campaign to raise awareness
    about onion sites, and if you enable Onion-Location on your site, you
    could win Tor swag! Join us to make a more secure web! #MoreOnionsPorFavor
    
    Read our blog for all the details: https://blog.torproject.org/more-onions-porfavor
    
    I've been interested in the privacy oriented Tor network for many years, but I haven't served any content on there before. I've used the Tor browser just for fun a few times (and it's easy to get going with), but I haven't made regular use of it. What they are trying to do with this campaign addresses something I've been missing from it - more just regular content, served straight within the Tor network itself, without the need to have exit nodes to "reach outside". So... I gave it a shot. It was really simple to set up. I set it up as described here: [https://community.torproject.org/onion-services/advanced/onion-location][onion-location] The linked article is actually for how to let visitors - connecting the old fashioned way - know that the content is available over Tor, but the article also covers how to configure the other server details. All this was simple to set up, but I've stumbled upon one isue, which is that nginx just... stopped working after a few days. This blog went down, and nginx started saying something like this in the logs:
    "/var/sock/tor-xhrpb.com.sock failed (98: Address already in use)"
    
    ... which I guess has to do with the fact that the Tor server regularly restarts itself in order for clients to reach it in a different way or something. As you can tell I haven't really researched this. Anyway, restarting tor@default.service and then restarting nginx.service seemed to solve the issue. I don't really get why I'm having this issue though, as I think the Tor service should create the socket and nginx should just set up a listener for it. There shouldn't really be a conflict. My current workaround (not at all a proper solution) is this: /etc/systemd/system/nginx.service.d/override.conf:
    [Unit]
    PartOf=tor@default.service
    
    /etc/systemd/system/tor@default.service.d/override.conf:
    [Service]
    ExecStartPre=/bin/rm -f /var/run/tor-xhrpb.com.sock
    
    The first one, in short, makes nginx.service restart when tor@default.service restarts. The second one makes sure to remove the socket before tor@default.service starts. The simple way to make changes to systemd services is to run "systemctl edit ", that way systemd creates the proper files in the right places and you don't have to run systemctl daemon-reload after manually changing the files. The reason for me sharing this is just to share what some in-the-moment troubleshooting can look like. The proper solution, the ideal, is to figure out exactly what isn't working right, but in reality one doesn't always have the time to do that. For the services I'm running, the "solution" above is perfectly adequate for me. Personally, the most important thing when making workarounds like this is to document it properly, so that one might come back and fix it later. This is especially important if you're working in teams with other people. Lastly, back to the topic at hand; What the added HTTP header does is that if you're browsing with the Tor browser and visits this site at https://xhrpb.com, the browser will let you know that it's also available on this address: [http://hppdzw5resw4k7v66yiuiooriqkh3mhnn4jxkm4mzy4rjyxvstmzgoad.onion][tor-blog] It then gives you the choice to browse on that address instead. I hope I'll see a lot of Tor sites in the future. [onion-location]: https://community.torproject.org/onion-services/advanced/onion-location/ [tor-blog]: http://hppdzw5resw4k7v66yiuiooriqkh3mhnn4jxkm4mzy4rjyxvstmzgoad.onion [tor-toot]: https://mastodon.social/@torproject/104480576144280489
  • Datacenter diaries: Added route, VRRP, follow-up

    Dear diary, First: This is not a tutorial. This is how I've set it up now, and I'm learning. I'm not a network engineer - I'm a curious sysadmin getting my feet wet by jumping in on the deep end. What I wrote about [the other day][post-added-route] turned out wasn't as mysterious as I first thought. I was confused, and it came to my attention that I also had to do some VRRP configuration. From their end it looked like both of my routers said they were the primary router, so I had to fix that. So, here's the same logic diagram as in the previous post, with a few changes. Note: As before, the 172.16.-IPs are examples and in reality they are public IPs. The 10.0.0.0 network is as in the diagram and configuration - a private network.
    +------------------------------------------------+
    |                  The internet                  |
    +------------------------------------------------+
          |                                   |
    +------------+                      +------------+
    | 172.16.0.2 | <--> negotiates <--> | 172.16.0.3 |
    +------------+          |           +------------+
    ISP primary             |           ISP secondary
                            v
                      +------------+
                      | 172.16.0.1 | ISP VIP
                      +------------+
                            ^
                            |
                            v
                      +------------+
                      | 172.16.0.4 | My VIP
                      +------------+
                            ^
    My primary              |           My secondary
      router                |             router
    +------------+          |           +------------+
    | 172.16.0.5 | <--> negotiates <--> | 172.16.0.6 |
    |   10.0.0.1 |                      | 10.0.0.2   |
    +------------+                      +------------+
          |                                   |
    +------------------------------------------------+
    |                   My network                   |
    +------------------------------------------------+
    
    ### Configuring VRRP This is on a Ubiquiti Edgerouter 6p. The configuration below is for the primary router.
    # Create a bridge with both the public facing interface (eth5) and the
    # interface that's connected directly to the other router (eth1)
    set interfaces ethernet eth1 bridge-group bridge br0
    set interfaces ethernet eth5 bridge-group bridge br0
    
    # Set the private (first) address and the public (second) address on the
    # bridge. The private address is used for negotiation, the public for
    # public access. These are "static" addresses not affected by VRRP.
    set interfaces bridge br0 description vrrp-bridge
    set interfaces bridge br0 address 10.0.0.1/30
    set interfaces bridge br0 address 172.16.0.5/29
    
    # Configure VRRP, group ID picked at random. Has to be the same on both my
    # routers, and should not (as I understand it) conflict with what the
    # other two routers by my datacenter operators are using on their end.
    # Whatever they have.
    # Higher number equals higher priority.
    set interfaces bridge br0 vrrp vrrp-group 12 priority 200
    
    # Configure the "floating" IP - the one that will be taken over by the
    # secondary router if the first one goes down.
    set interfaces bridge br0 vrrp vrrp-group 12 virtual-address 172.16.0.4/29
    
    The secondary router has the same configuration, with these exceptions: * priority 100 * IP 172.16.0.6/29 and 10.0.0.2/30 respectively ... and that's it. It started working and all is fine. :) The eth1 interface on each router is important because the routers need a way to talk to each other if the upstream network isn't working properly. Why they have to be on a bridge with the eth5 interface isn't clear to me right now, but when it becomes more clear to me I'll try to share it here. [post-added-route]: /2020/07/05/added-route.html
  • Datacenter diaries: Added route

    I went back to the datacenter two days ago and added a default route to the new router, so I can actually access it. It works great now. But the other router I found nothing wrong with, but something is up. So, each router has a fiber connection to my provider, and the provider has set up a VRRP solution that works like this: * Example network: 172.16.0.0/29 (but actually it's a public address) * Provider router/switch #1 has IP 172.16.0.2 * Provider router/switch #2 has IP 172.16.0.3 * One of the provider devices above also has IP 172.16.0.1, and they negotiate who has this adress between themselves, hopefully one that has proper internet access * My router #1 has IP 172.16.0.4 * My router #2 has IP 172.16.0.5 * I set 172.16.0.1 as the default gateway/next-hop on my routers
    +------------------------------------------------+   |
    |                  The internet                  |   |
    +------------------------------------------------+   |
          |                                   |          |
    +------------+                      +------------+   | Provider
    | 172.16.0.2 | <--> negotiates <--> | 172.16.0.3 |   |  network
    +------------+          |           +------------+   |   infra
                            |                            |
                            v                            |
                      +------------+                     |
                      | 172.16.0.1 |                     | Provider VRRP IP
                      +------------+                     |
                        ^        ^                       |
                        |        |                       |
                        v        v                       |
             +------------+    +------------+            | My routers with
             | 172.16.0.4 |<-->| 172.16.0.5 |            |   172.16.0.1
             +------------+    +------------+            |     as next-hop
    
    I have some theoretical experience with a setup like this, but very limited practical experience. I'm not sure exactly what it's supposed to work like, but this is what's happening now: * My router #1 can access the internet properly * My router #2 can't access the internet at all * My router #1 can ping 172.16.0.1 and 172.16.0.2, but not 172.16.0.3 * My router #2 can ping 172.16.0.1 and 172.16.0.3, but not 172.16.0.2 What I expect to see is that both my routers can ping all addresses on 172.16.0.0/29 and beyond it. I can also add that they can't ping each other on that network. If you're curious of how I can access the second router that can't reach the internet it's because I also have a cable connecting the routers straight to each other. I've sent an email to their support and I'm sure I will have answers or clarification in the next few days. Also worth noting is that there isn't much behind all this infrastructure of mine yet, so it's not like this is stopping anything as it is now. Over the next few days that will change, though. I'm looking forward to that. :)
  • Datacenter diaries: Redundancy

    Hello diary, Today I went to the datacenter again. I haven't been since my last update due to reasons, but it's nothing I've given up on or anything. So today I thew in a second router and a second switch - now we have redundancy! Or should have, at least. In theory. You know. After tightening the screws and putting the cables in place I just had to plug in my laptop and check if things worked alright, but unfortunately I had left my adapter thing at home. Gone are the days of laptops with RJ45 connectors... So I did what I could; I packed up, went home, and discovered nothing's working. So I guess I'll be heading back tomorrow. No worries.
  • Datacenter diaries: 13U and a Raspberry Pi

    Dear diary, Except that I actually have an internet connection in my rack now, I don't really have much to say, but I'll say what has happened since yesterday. So yesterday I went to the datacenter in the evening to reconfigure the router I had there with a new IP, mount the router properly in the rack (as I forgot the rack mounting kit the day before), put up a switch (Ubiquiti EdgeSwitch24 Lite) and two Raspberry Pi computers (just to test things). The actual servers going into the cabinet won't be there for another week or something. Anyway, yesterday, I couldn't get in. My access card and pin code didn't work. So I went back home, with all the stuff. No worries, it only takes me 30 minutes to get there and I wasn't in any rush. I was wondering if maybe I didn't have 24/7 access like I thought. The pin got reset today, and I hade no issues coming in this afternoon, and yes, I have 24/7 access. So today I did all the things I had planned for yesterday. I have the one router via one (of two) fiber connections in the cabinet (another router will be installed later for redundancy), one switch hooked up to the router, and then one Pi connected directly to one of the ports of the router and the other Pi connected to a port on the switch. So, now I'm sitting at home, and I can access all the stuff remotely just as expected, except for the Pi connected directly to the router - probably a misconfigured IP address on the Pi or something. I didn't bother testing it when I was there because I still had some network configuration to do, and I was sure at least one of them would work. I wanted to install [WireGuard][wireguard] (awesome software, I love it) on the EdgeRouter 6P as a temporary VPN solution (see below), but ended up installing it on one of the Raspberry Pi's using [this instruction on github.com/adrianmihalko/raspberrypiwireguard][rpi-wg], and it's working great. The router forwards connections to the Pi and then I'm on the inside. ### WireGuard and Ubiquiti I believe some hardware and software should be treated with a bit more caution than other. While Debian unstable currently ships Wireguard (and it'll soon be in the kernel itself!), this is not the case with Ubiquiti. Installing it on Ubiquiti means manually updating it, and the updates provided by Lochnair [here][github-ubnt-wg] have unfortunately been lagging behind, which is a bit unfortunate with software that deals with something so security oriented. Not that I have any issues with this person for not maintaining it anymore -- it's their right -- I very much appreciate the work put in. Thank you. Uploading stuff like that and helping the community by sharing solutions should not be seen as a lifelong promise to keep it updated, as some people tend to assume it is... Manually updating it (on the case of the EdgeRouter) is also not a good idea from a maintainability perspective. Those things tend to be forgotten. I just want to set things to automatically update on a regular basis and then forget about it, and focus on monitoring the services and server health instead. And of course monitoring that the system doesn't have uninstalled updates. So, with this installed on the Raspberry Pi for the time being, I feel very content. The Pi automatically updates with Debian's unattended-upgrades, and I'm not running unofficial software on something as important as a router on the edge. Less software, fewer bugs, better stability. I guess that's it for today. Happy smiley face. [wireguard]: https://www.wireguard.io [rpi-wg]: https://github.com/adrianmihalko/raspberrypiwireguard [github-ubnt-wg]: https://github.com/Lochnair/vyatta-wireguard
  • Datacenter diaries: First day and fiber assumptions

    Me and a couple of friends have shared a server in a colocation space for quite a few years now, but I've been wanting to rent a few more units (unit = data center term for space, basically) to put in some network equipment and do some other fun stuff with it. However, based on what it costs, it just hasn't made sense. At all. But now I've made it work out. I've started a company with the goal of providing colocation services myself, ish, but with some other fun stuff on top of it. Small services that make a big difference. Like some network connectivity stuff so the client doesn't need to expose IPMI on the internet but still have it available, for instance. I've got a client already, so the cost of this one-third rack (~14 units) I've just gotten access to will be covered, and I can do all the stuff I've wanted to do now! Amazing. ### Fiber fun Today's goal was just to put in a router (Ubiquiti Edgerouter 6P), connect it via fiber and see if it works. I didn't get quite that far though, because even if I'm kind of comfortable in datacenters and working with servers - including some network stuff - when it comes to working with fiber and network setup datacenter network engineers work with I'm at a total loss. #### Assumption 1: SFPs and 10G -> 1G compatibility First thing: When I ordered the SFPs I assumed that fiber modules were backward compatible to lower speeds just as you can use a CAT6 network cable for CAT5 speeds. This is apparently not true. Not in all cases, anyway. There are probably ways to make it work, but it's not as simple as for "typical network cables". Why, I'm not sure, but I'm guessing it has to do with silly things like physics and light and stuff. Right now I'm happy just knowing that this is a thing. #### Assumption 2: Multi-mode and single mode So, SFPs and fiber tech in general comes in two types when it comes to wavelengths, if not more; The ones I know about in this regard are multi-mode and single mode. I bought MM-modules, when I needed SM modules. I'm not sure what I was thinking when I ordered them a few weeks ago, but maybe I thought "back in the day" single mode was what was available (like when I cut and crimped [is that the word for fiber? I'm at such a loss] connectors to fiber cables in school 15 years ago). And MM was then newer thing. Maybe I thought that SM was two fiber wires sending in one direction, and MM was when one sent in each direction. I'm not sure. Anyway, I got the wrong one, but the people working in the datacenter were very kind to lend me a SFP module for the time being. I'll make a follow-up post about what SFP modules I'll end up using. #### End of the day At least i got LC connectors on the modules, so I got that right. Ubiquiti doesn't apparently sell 1GB SM connectors though, so I need to source them from somewhere else. Brands are not super important for SFP modules as I understand it, unless you're using something like Cisco, where they want you to buy their modules. Juniper is nicer than Cisco in that regard, according to the engineer I talked to. I'm guessing and hoping Ubiquiti is on the nicer end as well. I also forgot the rack mounting kit for the router at home... so... it's just lying in the bottom of the rack now. I'm both looking forward to and not looking forward to the fun cable management that comes later. So my router is connected now, and running, anyway, but they discovered a configuration issue they're going to fix tomorrow. I said no worries, I'm in no rush. Let's see if it responds to ping tomorrow. Ps. I just want to write "SPF" instead of "SFP" aaaall the time, maybe because "SFP" looks way to much like "sftp" (as in SSH FTP) for me.